 Hi everyone. So welcome back. So now we've got the first of our research paper sessions. So in this session there will be three different research papers looking at different ways to measure data and make it estimations. Each talk is going to last for 15 minutes with five minutes immediately afterwards for questions. So first up we've got Paul Fisher who will be presenting Assessing Earnings and Income Data from a Shure Web Survey. Paul, if you'd like to start showing your screen whilst I introduce you. So Paul is a research fellow and economist at the Institute of Social and Economic Research at the University of Essex. He is a co-investigator of the Understanding Society Survey and is the Survey Topic Champion for Income. So over to you Paul. Great. I'll just check you can hear me and see the screen. Yes I can. Great. Okay so thanks. It's nice to be presenting at this event again. So this year the title is Assessing Earnings and Income Data from a Shure Web Survey and this is joint work with Tom Crosley who's at EUI and Aima Hussein who's also at the University of Essex. So I'll jump straight into the background. So during the COVID-19 pandemic many web surveys tracked population income using short question sets. So a good example of this is Understanding Society COVID-19 study but there were many others. And so short question sets what we have in mind is that they ask about income totals not the individual sources. So the question just asks each individual for total household earnings or the total household income. And this is in contrast to detailed question sets which ask about individual income sources. So they might ask each individual about income from different jobs you know all the benefit income they get different pensions. And then to get to the totals the researcher has to aggregate over all of the individual sources and over the individuals in the household. So this is more like the FRS approach and also is done in Understanding Society main study. So just to say that these short question sets this is not something which just come out of the pandemic. Short question sets are also used in other surveys that prioritise other content domains. So you know just one example but there's many is Health Survey for England. So it's a specialist health survey but researchers also want to have some controls or some information about income and so they ask these short question sets about income. So relative to the detailed question sets the short ones are quicker to develop and feel in a crisis like the pandemic. They save interview space and they also have this lower burden on the respondent because they're just quick questions. But on the other side the short question sets kind of violate the best practice guidelines for data collection and the best practice guidelines say to use the detailed question sets. So there's little evidence which is kind of assessed the performance of these short question sets and we think this is really surprising given how much they're used in policy and research and how many surveys are using these questions. And so we think it's imperative to understand the quality of the data collected in this way given that they're being used very extensively. So what we do is we study the quality of earnings and income data collected with short question sets in the Understanding Society COVID-19 study and we compare individuals. How can we do it? So we can compare individuals who are interviewed twice in a short time period once with a detailed set of income questions in the Understanding Society main study and once with a short set of income questions in the Understanding Society COVID-19 study. And essentially we use the detailed responses as a validation source for the short question sets. So empirically what we do is we estimate a model of measurement error and this is the Binglian-Martinello model. And so unlike the kind of the early validation studies that kind of more or less just treated their validation source as being true, this method allows us to allow for errors. So we're going to allow for errors in the Understanding Society main study in the detailed question sets. We allow for some types of errors in that. We don't assume the validation sources is the truth. And this method allows us to compare the quality of the different data sources. So as I said, we're using the Understanding Society. So we link the Understanding Society main study with the COVID-19 study at the individual level. So just to kind of summarize the data sets, the Understanding Society main study has annual interviewing of participants. It's a 45-minute interview. It has a mixed mode design. During the pandemic, it was completely online. The sample size is quite large for a panel. So for example, at Wave 11, there was 32,000 individuals. And it does its best to practice income data collection. So it asks for detailed income questions by source and individual. So then we move on to the COVID-19 study. So what happened here is all of the main study participants were invited to participate in the COVID-19 study. The response rate at Wave 1 was about 41%. And in contrast to the main study, the interviews are much shorter. So the interviews were just 20 minutes long, 20-minute web surveys. And people were interviewed in various months. And we work with Wave 2 to 6, which were in May, June, July, September, November, 2020, and March 21. So the COVID-19 study has short interviews. And so the questions asked about income were of the short question set type. So there's three kind of questions we're working with. One about individual earnings and the other two about household earnings. So each individual was asked, what is your household earnings and what is your household income? So our research design. So the main thing to say is what we do is we take people who had their COVID study interview with the short question sets, and we say, if they had a main study interview within 14 days of that, then we're going to match them. And they're going to be in our sample. And the reason we have this kind of 14-day period is because we want to match, we want to have, we want the income, the reference period for the income variables to match in both sources. So we pick people who had the interviews at a close in time. And this figure here just shows graphically what we do. So the top half of the panel there we can see on the horizontal axis is time, on the vertical axis is the number of interviews. And so the gray observations there are the main study. And we can see every month, you know, some respondents get issued to the field for interview. And that happens at the start of each month. There's a biggest spike at the start of each month. And then the dark color interview, the dark colored bars there are the COVID-19 study. And you can see these happen in a selection of months. And they tend to happen at the end of the month. So the bottom part of the panel just zooms in on May and June. And what we can see is if we assume this person, this dashed line in the middle here, if we assume someone was interviewed in their COVID interview at the end of May, then we go 14 days this way, 14 days that way. And if they had a main survey interview in that period, then we match them. And so that leaves us with 3669 matched individuals in our sample. So I'll first just give some descriptive comparisons just before moving on to the measurement error models. So we have some results on item missing data, but I'm not going to talk about that unless anyone's got questions maybe at the end. So these show income and earnings, CDS for our three income variables, individual earnings, household earnings, household income. So the dark black solid line is from the detail questions in the main study. And the dashed line is the responses from the short questions in the COVID survey. And so I think the kind of most striking thing to take away from the start is that these distributions look really similar. So we get kind of similar answers in both surveys, which is reassuring. If we look at individual earnings, we can see that they basically overlap each other. So the both surveys are telling us the same thing, whether we use the detailed or the short question sets. Whereas for the household variables, we see there is some deviation. So typically, the COVID survey line, you know, runs above the main survey line. So we're getting lower estimates of income from the COVID survey for household earnings. And it looks like the gap is a bit bigger for household income. So we get lower estimates of household income from the COVID survey relative to the main study. So already, you know, the surveys look similar, but we see some under, well, less reporting in earnings and income in the COVID study. So I'll move on to the measurement error model and results. So as I said, we estimate this model of Bingley and Martinello. And I'm just going to try, I'm just going to kind of emphasize the main features that we need to interpret the results. So there's these six parameters that we're going to estimate here. And I'm going to mainly focus on the first two. I'll just run through what these parameters are. So C means COVID and M means main. So KC is the mean under reporting in the COVID-19 study. Okay. On average, how much people under report? Rho C is the relationship of the errors with the true values. So are the, are the errors that people make correlated with the true values of their income? And then here we have mu y is just the mean of the true income sigma squared y is the variance of the true income. Sigma squared M is the variance of the main study income error. And sigma squared C is the error variance of the COVID-19 study error variance. And you know, all, so we want to estimate these parameters and this is going to help us say something about the quality of the data. So I'll just give you the kind of two main assumptions we're making in this method, which just, just to kind of, you know, be transparent. So the first thing we're assuming is we're not assuming that the main study, the detail questions in the main study, we're not assuming that they're error free. We're allowing them to have errors, but we're just assuming that the errors are classical. So that basically means that people report their true income. There's just some noise when they report. There's just random errors added on to their, to their reports. And the second assumption is that we need to have instrumental variables that are correlated with true income, but uncorrelated with the measurement error in both surveys. So we need some variables that predict true income, but they're uncorrelated with the errors in both surveys. And I'll show you what we use in the next slides there to do that. So I'm going to show you the results for house on earnings and income. I'll just focus on those because I think they're the most interesting. So the instruments we use just to say is the main thing we're using is this, the house on council tax liability. And the reason we use this as an instrument is because it's externally reported. So it comes from administrative sources. It's linked to administrative sources basically. So that means that there's no areas from the respondent. The respondent doesn't have to report this, so there's no areas. And if there are areas, they're not correlated with the ones the respondent would make. And we also use the number of cars in the house on the number of rooms in the house on because we think these are reported pretty cleanly. So the first two columns here show the results for house on earnings and the second two for house on income. And here we see the parameter estimate status size. So if we just run through these, if we look at column two, we see that KC, if you remember, was the under reporting. So KC is the mean under reporting. And we see that house on earnings is reported on the single income questions. Rosie tells us about whether the reporting areas are related to the true values. And because this is significant, we see that they are. So that the areas that people make are correlated to their true, related with their true earnings, true house on earnings. That was house on earnings. For house on income, if we look at column three. So in column three, we first of all, in column three, we assume that the detailed questions in the main study are true. We assume that the truth. And what we find is that people, yes, they under report KC is negative. So they under report on average their income. And also that the the reporting areas are related to their true values of the incomes. Okay. So this is when we assume the main study is true, but we can relax that and allow there to be an error variance in the main study. So we allow the main study to have errors. And what happens then is the under reporting here for KC still remains, but we find out that the estimate of row is now insignificant. So once we account for the fact that there's errors in the detailed questions, actually there's under reporting, but the under reporting is not correlated with the true values of income. So one kind of question is, you know, how can we kind of, what do we learn about using these types of data from these estimates? And so I just want to finish and give you a kind of, kind of example of, you know, what these kind of estimates imply for the types of analysis that people might do. So one thing people often do is some kind of simple regression analysis and let's imagine here that they had something like they took the health survey of England, they got the quick income questions and they wanted to know, they wanted to know if there's an income gradient in some outcome. So we assume that the outcome could be whether people have COVID and you want to know, I'd do rich people, more likely to get COVID than poor people. So they run this regression here and we know that the income comes from the short question sets. Well, this is just a result from the literature that we know that if we have to do this regression, the estimate of B that we get is equal to the true one multiplied by this attenuation factor. And this is going to be something between zero and one. So the estimate we get is going to be downward biased by this factor. And all of these parameters here are just things we just estimated in the previous table. So we can just calculate what this attenuation factor is. And the other thing that people sometimes do is they estimate this model by instrumental variables to try and correct for the they say, OK, there's errors in the income will estimate by instrumental variables to correct for these correct for these errors. And in that case, the attenuation is given by this attenuation factor here. OK. And again, we can get our estimate to row and calculate what these attenuation factors are. So we can let's do that and see what happens. So here we see for. So if we if we run that regression with individual earnings, what we're going to find is actually there's not too much attenuation. Basically, this tells us that if we take the main stage data, we run that regression, we would get our estimate of beta would be ninety two percent of the true value. OK. And if we did it in the covid survey, it would be eighty seven percent of the true value. When we move to household earnings and household income for household earnings, we see there's more attenuation. So in the covid survey, we'd only get our estimate of beta would only be seventy percent of the true value. And the main survey would do a bit better. And then we move to household income and, you know, we can see the gaps even bigger that the covid survey would would get even more attenuation for household income compared to the main. Then what people do is, you know, they try and do corrections using an instrumental variable so we can see what happens if we have an instrumental variable. Well, if we did the instrumental variables on using the covid survey, actually this corrects completely the attenuation and our estimate of beta would equal the true one because it equals one. And similarly for household income, you know, this corrects. Instrumental variables would be successful in correcting the bias there. And the reason that it's it doesn't correct for household earnings is because that the estimate of row for household earnings was positive. It wasn't zero. So it works to correct individual earnings, household income. So just to kind of sum up because I think I'm running out of time. So we've presented evidence on the reliability of income data collected with short question sets using the Understanding Society covid-19 study is a test case. Individual earnings collected in a short web survey is of comparable quality to those collected in the more detailed survey. But for household earnings and income, short question sets produce measures that are more noisy and suffer from systematic under-reporting. So these should be born in mind if you're going to try and estimate poverty rates using short question sets. For example, the household income, there's no evidence that the measurement errors in the covid-19 data are related to the true values. So if we're doing regression analysis, instrumental variables actually is going to correct the attenuation bias. And we also have some results on are you missing this, which I can talk about if anyone's interested. And finally, just to say that what we think we learned from all of this is that the short question sets on earnings and income are useful content for short surveys or even in longer surveys that prioritize other content domains. OK, and that's it. We'll move on to the next presentation, which is from Pauline Brown and the presentation is on differences in measures of family poverty and their association with educational outcomes. So Pauline, if you're there, would you like to share your screen whilst I introduce you? So thanks. So Pauline is a part-time PhD student at the University of Manchester. She's an experienced teacher and senior leader leading on pupil premium and inclusion. Pauline currently works in a multi-academy trust and her research interests are exploring the links between poverty, place and education. Over to you. Hi, can you hear me? OK. Yes. Thank you. And you can see my presentation. Yes, thank you. Thank you. OK. I'm just going to start by contextualizing socioeconomic disadvantage before we go into looking at the data set. Obviously, it's a known huge impact on children's educational outcomes and has been a focus for policy since 2010, particularly around the diminishing the gap narrative. But it's quite a complex concept, which includes family income, parental education and occupation. And these concepts are obviously interdependent and operate in different and cumulative ways and educational children's educational trajectories. There's lots of previous studies done on the effectiveness of free school meals, notably vineyards et al. And they have highlighted that income deprivation is a reasonable socioeconomic proxy for parental education and occupation. But again, it is because data is difficult to obtain in these other in these other areas. So socioeconomic disadvantage and education can contextualize as income deprivation. This form of material deprivation is currently proxied by free school meals. And this is a passported benefit that's predominantly now determined by universal credit eligibility. And it's important to recognize the passported benefit is one that families have to further apply for at the school and is determined by local local authorities. And the backdrop to this is there has been significant welfare benefit reform since 2010, which has led to quite a lot of volatility in FSM eligibility, such as the current protections that in implementing universal credit that are about to run out at the end of this year and also receive the increase in FSM due to the pandemic. So it's quite a volatile cohort. The research questions then are to what extent are available poverty proxies identifying children in poverty and what is the measure that is best you should best be used in identifying educational risk derived from poverty. So operationalizing income poverty is quite challenging, but we're going to focus I'm going to focus on the income derived concepts, which are obviously determined by welfare eligibility. People premium is defined by FSM eligibility and it's at any point in the past six years. And this study is seeking to operationalize poverty concepts comparing them, comparing the theoretical basis for their conceptualizations and the direct looking at how those are. If it how those play out in poverty variables that are available in the millennium cohort study. So that the methodological approach is that income deprivation is the aspect of social economic status that policy can best ascertain and compensate for in real time. If families lack funds redistributing funding could be implemented through a range of levers. And that's effectively what the current pupil premium funding does in school that redistributes school funding based on children who are eligible for pupil free premium. But identifying poverty and defining it is about understanding the availability and deprivation of family resource. And measures of poverty should seek to evaluate the availability of a family resource based on financial capability. And this builds on the work of our Marta Sends theory of capability where we consider the available family resources that are actually there considering the resource pool. And in terms of an economic evaluative space this could be represented by net income looking at after housing costs and or equalised income. And ideally net income would consider all inescapable costs such as childcare, debt, work travel much of which is spatially varied. And I just want to highlight the work of the Social Metrics Commission on developing a new approach to poverty measurement that does try to encapsulate these inescapable costs as well. And after housing costs and equalised income are difficult to operationalise but we've seen the DWP's work on households below average income use both of these measures. So in this study a multi-level model was implemented because the NIL model highlighted that there was an intra-class correlation of 0.31 for the Key Stage 4 attainment which is the outcome variable showing that there was clustering at school level. Base model then was created with known predictors of Key Stage 4 attainment which is what we're looking at of Key Stage 2 of private attainment which has the biggest impact gender, ethnicity and different variables for social economic disadvantage. These are widely established in the literature, the work of Gorad, Vignoles and also Lecky and Goldstein. I've identified these and repeatedly shown that these are the most important control variables. Then the RAP, Heskett and Skondrol R-squared Level 2 methodology was implemented looking at the proportional reduction in error comparing eight different models where different poverty variables were changed. Three of these poverty variables crucially operationalised types of equalised income variables. The data set then is the Millennium Cohort Study which is a birth cohort study which is fantastic for this particular piece of research because it oversampled in the disadvantage stratum as well as the ethnic stratum. And the particular data that I was looking at or I'm looking at is this is sweep six. It's based on the young person aged 14 with then their linked their linked NPD educational histories available right the way through from early years through now tracking through to post 16 and into destination data. Crucially sweep 16 contains several derived income variables whereby missing data was crucially replaced using multiple imputation from previous sweeps. So there's a really there's very little missing data. There's no missing data in fact in the derived incomes that I am looking at although there is some data that is not yet is not available. So the analysis of cohort members were based on successful NPD linkage and productive outcome achieved at sweep six and there is significant attrition in this study but due to the oversampling the final matched complete case data is representative of national in terms of the disadvantaged population. And we were looking at an overall population of just short of 6000. The different variables available in the MCS data set represented different theoretical dimensions. And I think this is really important for my study to show the differences. Well, often when we talk about poverty it's all it's it's quite unclear actually what we mean. Some I'm sure colleagues here are aware of those differences but it's important just to highlight that some of these are obviously continuous variables. Some of them are binary poverty line variables that are being operationalised. And then we've got two that are looking at concepts around place. Just a little quick discussion of equalisation. The MCS uses the OECD equalisation methodology which is standard amongst many data sets. And this is what I will be looking at in the three poverty variables that are equalised. So just to go back to universal credit which obviously FSM is derived from and this is just highlights that there is a form of equalisation I suppose, evident in the monthly standard allowance in which calculated you can see that it takes into account economies of scale and also age which is what the principles of equalisation utilise. However, we've got obviously implemented now a benefit cap which doesn't follow that methodology or theory of equalisation which is obviously impacting on children in families. And just to illustrate really how that could play out I don't we don't have the individual family situations in terms of the other inescapable costs housing costs for example or other income but just to show you where there is a maximum benefit cap how that could look like in terms of an equalisation when we look at how income is shared out in different sized families. So who is in poverty in terms of the different variables what we can see is that there are differences in the numbers of children that are identified in poverty but crucially comparing to people premium and FSM but people premium being the main marker of disadvantage it is actually crucially identifying different children in poverty and I just want to flag up the number of people premium children compared to the OECD and 60% median relative poverty similar numbers of children but very different representation or a third of those children are non-PP. So the multi-level models implementing a proportional reduction in error highlight then for each of the models that the variables that were representing a criminalised income showed a greatest reduction proportional reduction in error you can see it was 0.628 for the criminalised continuous income for the binary relative income 0.614 and then the quintiles 0.629 so these were highlighting that more educational risk could be identified through using through using these variables and it was more successful in reducing the variance. So the limitations of this work are that unequivalised income was is unavailable at this stage so looking at the comparison that would be significant and obviously those huge limitations in actually defining those inescapable costs and further define resource capability and that's obviously been a theme that's come through in a lot of the other presentations this morning. I was not able to compare parental education and occupation data this was significant had significant amounts of data missing although I would point to the previous work that Vinyals et al have done around this and what I want to do next is obviously to test the coloniality of the poverty variables in the model this is a piece of ongoing PhD work. So just discussion to highlight really as we saw the number of children vary significantly depending on poverty metrics adopted and that's not always a clear I think when we talk about poverty that as we said the equalised income variables explain most of the variants and then also just to flag up that we use key stage two and prior attainment when we are in the models commonly because there is prior to key stage of prior attainment that the primary data is less reliable to use in these models and I would pose the question and to what extent is this data prior key stage of priority also dependent on socioeconomic disadvantage given that the disadvantage gap does widen from the early years and is this in fact you know is the impact on income deprivation in fact greater? So conclusions are I think that pupil premium and FSM are no longer effective proxies for poverty and socioeconomic disadvantage and if you are late study looking at this talks about using a basket of measures for identifying socioeconomic socioeconomic disadvantage in schools changes to welfare policy matter in the education domain we've got lots of larger families now that are subject to the benefit cap that was introduced in 2017 those children are five and they are coming and they will be entering into school and the depth of the deprivation that those children are in in terms of family poverty will not be will not be seen by the current proxies all other volatility I mentioned about the transitional protections to universal credit obviously that's going to have an impact on things like persistent poverty and long-term disadvantage and being able to use those measures and I suppose a further prediction of this paper is the FSM eligibility and it's the value of using data that is taken from the political domain and into the education domain is not helping us to identify the children that most are at risk of socioeconomic disadvantage to target resources so I'll just stop there and I'll see if there are any questions thank you so the final presentation of this session is from sorry Lina Killian and it is estimating neighborhood greenhouse gas emissions using the living costs and food survey so Lina is a third year PhD student at the University of Leeds and she is part of the ESRD funded Centre for Data Analytics and Society and researches consumption based emissions of UK households security yeah can you see my screen and hear me all right yes I am all right perfect I'll get started then so in recent years local actors have become increasingly involved in climate change mitigation efforts and moreover the focus on emissions has shifted from focusing only on production to incorporating consumption based approaches as well however to enable such local approaches to reduce consumption based emissions we need to generate data at a subnational level on such emissions and research like this needs household consumption micro data of where existing research often relies on commercial expenditure data which can be less accessible and also less transparent because often the kind of data generation processes aren't shared fully so in this research we aim to see if we can do something similar and create this similar data set using the living costs and food survey oh it's not okay so yes as mentioned before I'm a PhD student at the University of Leeds a part of the Data Analytics and Society's CDT and my PhD focuses on estimating and evaluating neighborhood greenhouse gas emissions in the UK and so in this particular talk I want to talk about the part of the PhD which estimates these using the living costs and food survey so first I think there were already some buzz words that might not be familiar to everybody I'll go through some definitions then the method which will be the main part of the talk some validity checks we did or a brief summary of those and finally some brief conclusions so first of all the big question I guess what are consumption based emissions and why are they important consumption based emissions are emissions measured throughout the supply chain that are attributed to the kind of final demand user rather than the producer so they include indirect emissions like buying vegetables and they also include direct emissions like from driving a car and here in this paper we're looking at UK households as the kind of end users but then aggregate this to a neighborhood level now that we know what consumption based emissions are why should we care about them developed countries can decrease their emissions from production by outsourcing their production to developing countries and then in some cases actually if the industries elsewhere are more carbon heavy consumption based emissions might increase while production based emissions go down so they might present or complement production based emissions with a kind of fair approach that looks at where products and services are actually consumed second they can offer demand site mitigation advice so by knowing where things are consumed we can try to focus on maybe behavior change policy or infrastructure developments like building bike lanes to try to reduce car emissions in high car emission areas for example and finally consumption based emissions allow us to look sorry at carbon inequality both internationally but also within countries so this means we can look at not only where we need to reduce emissions and where it might be easiest to reduce emissions for most effective but also where we need to redistribute emissions and this can help with things like reducing fuel poverty and such like that like we need to increase emissions potentially to reduce those effects so to calculate the consumption based emissions sub-nationally we need two types of data we need consumption expenditure data where here we're using the living cost and food survey and we also need something called MRIO data which is multi-regional input output data we're here we're using the UK's MRIO data as the UK has its own model we're using data from 2016 as this is the last year where comparison data are available to us at least so yeah we then conduct something called an input output analysis which is a method that originates from economics but we can add an environmental extension to this to calculate emissions that occurred throughout the global supply chain and we can also use this to isolate emissions that occurred across countries and sectors due to final demand so this means that we can calculate emissions from products sold throughout the supply chain including trade between industries as well MRIO data is structured in the following way so we have a transaction matrix which shows inter-industry trades so the inputs and outputs between different industries of different global regions and for example it means that if the agricultural sector uses a truck from the automotive industry it will be an output for the automotive industry and an input from for the agricultural sector so we kind of get this inter-industry interaction next we have a final demand vector which captures sales to final demand by different countries and also by different kind of actors so government is one of them but we here look at households and finally the important thing also for this research is the environmental extension which captures total emissions from different industries and essentially we use all this data to calculate conversion factors which are emissions in our case greenhouse gas emissions per great british pound spend so for each british pound spend on different types of products what were the emissions associated with that and then we can use these conversion factors and multiply them by neighborhood expenditure estimates to estimate household emissions sub-nationally now where do we get the neighborhood expenditure from and for this we use the living cost and food survey why it has a transparent data generation process it's free to access for research and it contains a lot of detail on different types of expenditures so first what we do is we adjust the expenditure for different products in the living cost and food survey by the final demand matrix from the MRIO data to make sure that they match and that there is some consistency between those and then to go from five to six six thousand households to the entire UK we use the regional and the output area classification data within the living cost and food survey to create sub-regional expenditure profiles so the OAC data essentially is a classification of different neighborhoods by different types so there are different levels of this one contains eight different types one 26 and I believe one 76 but yeah so we kind of combine this information to try to create these regional neighborhood profiles with different expenditures to try and get a more detailed geographic overview this is a summary which looks very messy but essentially what's happening is is that we create groups so that each output area so the smallest census geography is made up of at least 10 different observations or each sorry each profile that we create is made up of at least 10 different observations these are matched to the output areas and then we aggregate these to larger geographies so here we're using middle super output areas and lower super output areas to make sure that we have a larger number of households within each neighborhood to try and reduce the effects of outliers and also some bias that might come from individual observations then obviously we have no idea how this compares to the estimates that we created with this to other estimates created with other data sets so we did some validity testing which I will summarize here very briefly so we compare the estimates we get from using this method to estimates we get from using the output area classification which also comes with with expenditure profiles attached to it we're using the group level which means we have 26 different neighborhood types across the whole UK and we also compare it to emissions created from an expenditure micro data set from the credit institute institute trans-union and we want to see how similar the emission estimates are to the emission estimates from those two other data sets and I'll briefly just show the distribution so first of all we see that the living cost and food survey distribution which is the continuous line is very similar to the trans-union distribution across so this is across all UK neighborhoods at both LSOA and MSOA levels so at both the lower and the larger neighborhood levels the OAC data is a bit different but we think that probably also the multiple peaks come from having just 26 different types of neighborhoods so here this might indicate that the living cost and food survey way we generated these profiles might give us actually more geographic detail but then this is for total footprints so we also want to see what happens at individual products and here we find that so we run different tests and then said they pass or they fail so we run correlation tests for all three data set comparisons and distributional tests and we said that if the effect size of the distributional test indicates no effect that then we pass the test as the distributions are similar and likewise if there was at least a weak correlation so correlation coefficient of 0.3 or larger that we considered them to have passed the correlation test so we can see that for apart from the oh yeah sorry the different product levels one contains 39 different categories so these are larger categories like food and one contains 134 different categories which are more detailed like bread, rice and cereals so apart from the most disaggregated one which is the LSOA 134 category the majority of the footprint from households passes all three tests so products from over 50% of emissions products that make up over 50% of emissions across the different neighborhoods pass all tests so this gives us some kind of promising outlook that the living cost and food survey can be used for this and especially if we look at the individual comparisons rather than kind of the past all test columns the emissions covered by this are a bit higher but of course we can also see that there is some room for improvement here and when we looked at it in more detail we found that especially for household gas and electricity use the emission estimates were vastly different across the three data sets as this is the highest emitting category for consumption-based emissions of households it's very important to try and find maybe other data potentially using some physical data here where this is available could really improve our trust in this so yeah to kind of summarize and conclude um we think that the living cost and food survey provides a good like open method for this analysis and of course what we found some differences between the data sets and some areas for improvement using the living cost and food survey has the main advantage that we know where the uncertainties lie because we know how the data are generated so we know where they create uncertainties with the commercial data unfortunately we don't know this so although we have the estimates we're unable to say why there are differences between the data sets based on the trans-union data so yes so we find the living cost and food survey method very promising for future research and for generating these these profiles I think that it's possible to do it with an open method as well rather than just commercially created expenditure data the full paper is available on the link on the screen and also my email if you have any questions I want to email me in the future and please feel free to do so thank you very much hi so welcome back for the speakers this morning there have been some lovely comments in the chat and you can also find copies of most of this morning's presentations on the event page so there's a link there too and we'll now start our session on food insecurity and I'm very pleased to introduce the session which will include presentations from two speakers so we've got Rachel Lupstra and Ben Bambergiger and we're going to have both presentations and then open up for discussion comments and questions so first of all we'll have Rachel so if you want to set up your screen or anything while I introduce you so Rachel completed her PhD in nutritional sciences at the University of Toronto where her research focused on understanding drivers of food insecurity and food bank use in the Canadian context and then as a postdoctoral researcher in the Department of Sociology at the University of Oxford she examined relationships between austerity and social protection programs in relation to food bank use homelessness, food hardship in the UK and across the EU and then from November 2016 she was a lecturer in nutrition in the Department of Nutritional Sciences at King's College London developing a profile of research focused on macro policies relating to health inequalities chiefly household food insecurity in the UK and cross-nationally and then in June 2020 she started as a senior lecturer position in the Department of Public Health Policy and Systems at the University of Liverpool and she's an expert advisor to the Food Foundation and has provided advice on food insecurity and food bank use monitoring measurement to the food standards agency Department of Worth and Pensions the Trussell Trust among others so and today Rachel will be speaking about making sense of food insecurity data in the UK so over to Rachel sorry I was just saying I just recently joined the University of Liverpool just a couple weeks ago actually so just have rebranded my slides that way and thanks so much again Jen for inviting me to the conference today to present so I wanted to start off the presentation with a couple of different headlines that we saw come out this past spring so first of all we had the data released from 2020-21 from the Family Resources Survey stating a minority of households were food insecure 3% experiencing low food security and 3% experiencing very low food security but then just a couple months later in May we had a headline come out from the Food Foundation that one in seven adults were either skipping meals reducing their meal sizes or going hungry just a week later we had another headline saying one in four people were skipping meals and then we also had conflicting data on food bank use with the Trust of Trust reporting 2.1 million food parcels going out which equates to about 3% of the population if you count that on a one-to-one basis and then data from the Food Standards Agency reporting that about one in six people were using food banks in March 2022 so lots of different headlines and certainly lots of different interest in this problem of people not having enough food to eat and so my thought for the presentation today was to go over some basics let's return to talking about what is it what is the problem that we're trying to get a handle on and understand what we're measuring to have a look with you at the different surveys that are measuring food insecurity in the UK and then have some reflections on the different methodologies and how that could then be influencing our estimates and then hopefully I'll briefly have time to talk about what food bank statistics are telling us about food insecurity as well so first of all if we look at the construct and when it was developed in high income country context we can actually go back to the US in the 1980s where they had had a program of looking at undernourishment in the population using typically biologically defined measures of malnutrition and they had found that largely they had eradicated the problem of nutrient deficiencies but yet in the 80s they saw this large growth in the number of people seeking help from food charities and so not so similar of course from the UK story that we've seen here in the past 10 years and so different researchers began a program of work to try to better understand what were the lived experiences of people seeking help for food assistance not having enough food to eat in their homes what did it look like and what is the problem of insecure food access in a high income country context because typical measures of nutrition weren't clearly not capturing this problem so the quality of work in this area there's been quite a lot of it from both the United States and from Canada over the 1980s and 1990s and these are some of the quotes from that research so experiences of families having food depletion so their food supply is running low and experiencing the anxiety that comes with that not being able to make up a proper meal not being able to offer their children a variety of fresh fruits and vegetables being limited in the items that they could select to those that were low cost or on sale or to what they were receiving from charities and then more extreme examples of well as well so going to bed hungry knowing that they had not had enough to eat and some researcher by Radimer and colleagues came up with this chart kind of depicting the different core components of food insecurity highlighting that experience at the household level and observing people talking about their household food supplies so food depletion the types of foods that their household was relying on and experiencing anxiety about those supplies and having to rely on charities or unacceptable means of acquiring food but then also that food insecurity could be experienced by different individuals within the household in different ways so people talked about their own experiences being different in particular from their children so parents talking about their own insufficiency of intake or their inadequate diets and then also the psychological components of that of feeling deprived and not keeping up with kind of normally socially prescribed eating patterns like eating breakfast lunch and dinner so and other observations from this work is that the experience is range and severity it then has multi different dimensions and it's not equally experienced by everyone in the household so out of this qualitative work the United States Department of Agriculture spent quite a lot of work to develop a measure that they could put into their current population survey to regularly measure and monitor food insecurity in the US population and to develop this module they took 40 items based on this body of qualitative research and tested different scales combining different items to try to come up with kind of the best indicator to capture this underlying core construct of food insecurity and I won't go into the details of how they came up with it but essentially out of this work they've come up with a scale that has 10 questions aimed at the household and adults in the household and then a separate scale of eight questions focusing on the experiences of children in the household and each of the questions tie the experiences to financial resource constraint and this is an acknowledgement that they are not wanting to capture people skipping meals because they just didn't have time or were dieting for example but this is related to the concept of enforced lack so this is not by choice people are going without food because they don't have the financial means to get it and also reflecting that in high-income countries if you have enough money you cannot only afford the foods you need but also have the resources to get to the shops or in today's times order if you need to order online groceries the questions also specify time periods and as I've already mentioned differentiate experiences adults and children so this is what the household or an adult food security questions look like so the first three questions are more around worry and anxiety food supplies running low and not being able to afford balanced meals and again that's a socially or a personally defined question so what's right for you and what you think would be a balanced meal and to reduce the amount of time that people need to take to do this questionnaire in surveys a screener is introduced and often applied so that if a respondent doesn't answer the first three questions they then don't go on to answer the rest of the questions and that's just to save time really and it's also reflecting however this idea that food insecurity ranges in severity with some people just experiencing more marginal experiences others experiencing more severe but that usually if you're experiencing severe experiences you've also experienced the worry and concern about food running out for example and then the operational definition emerging from the scale that the USDA has defined is that it's inadequate and insecure access to food due to financial constraints so very much tied again to a lack of financial resources in addition to this scale we had the food and agriculture organization work to develop the scale that could be used in countries across the world and a lot of the questions are actually very similar to the USDA scale and they use that scale to build their scale and this is now in use in the Gallup world poll and we're also seeing it in use in the UK in a couple of different contexts so I wanted to highlight this scale as well it has only eight items and the responses are only ever yes no whereas the USDA scale includes different response options for the first three questions often or sometimes getting that different temporal scales so based on the number of items that are affirmed on either the USDA or the FAO scale that I just showed you households are classified into three different categories so I should say four actually food securities if they answer no questions affirmatively then we have a marginal or mild food insecurity category low food security or moderate and then very low or what's also known as severe and I tend to use the labels on the bottom because in the Canadian context that's what they use and that was where my training was so I'll be referring to moderate and severe food insecurity but in the USDA and the language that's being used in the family resources survey it's low or very low food security now I was involved with quite a bit of work to try to push for food insecurity scales to be included in measures in the UK as were a number of different organizations and others other researchers as well and I wanted to just briefly touch on this because of course we know that material deprivation measures have been included in the family resources survey for quite some time however we never we didn't have the food insecurity module included regularly in UK surveys but I think it's important because compared to other kind of material deprivation questions it's capturing something that's quite unique so first of all we don't need to argue about what's essential food is essential for everyone and so therefore to have people saying that they're not able to eat foods that they need or even insufficient quantities of food we can all know that that's a standard of living that's not acceptable the other thing about food is that because it's so personal food spending is a very flexible budget item and therefore it's quite sensitive to income budget or income fluctuation so it's a sensitive measure of poverty in that way so people will quickly substitute their how much food they're eating because it doesn't have any immediate consequences they're not going to go into rent arrears, bill arrears they're not going to have someone knocking down their door asking for their payment that they were due last month so it's something that people I think more quickly start cutting back on and so it's very sensitive then for when households are starting to experience financial struggles so it's easier for people to make short-term trade-offs easier to hide and the other thing to note is that it is also then transient so we see it manifesting in different ways because it's more easier to remedy the other thing to reflect on is that it's not affected by things like seasonality so the same way that maybe heating costs go up of course in colder weather so I was delighted when the food standards agency started including the USDA measure in their survey Food and New Survey since 2016 and then we had the family resources survey add the module from 2019 and we saw the understanding society survey includes some measures in their COVID-19 survey and I believe it's planned to continue some measures going in in ways 13 and 15 and hopefully forward from there and then we also have had the Scottish Health Survey and the Food Foundation also monitoring with UGov polls so even though we have all these different measures coming out from this graph you can quickly see that is coming up what we're seeing is very different prevalence estimates so in Scotland we had a prevalence of around 4% of people and reporting eating less in September 2020 then over that same time period the data from the Food Foundation suggested around 9% again over that is about the same time period data from the FAO suggesting 16% of adults were experiencing moderate and severe food insecurity and then we had the data from the FRS for again the same period 2020 to 2021 suggesting only around 7% of households so we're having these different data now come out from these surveys and I think again it's just creating quite a confusing picture so I just wanted to reflect on some of the reasons why we're seeing such different prevalence estimates come out from these surveys so this is a busy slide I realized there's a lot of information on it but I just wanted to draw attention to kind of some important sources of differences so the first thing to reflect on is the different sampling methods so who are these surveys representative of so a lot of them are based on trying to on the postal address file so based on households so who are we missing in those surveys of course are going to be homeless populations people with no fixed address and also institutionalized populations and we know that those populations have higher risk of food insecurity then we also look at Ugov data that's a web panel based on quota sampling and we also have to think about who then has access to the internet and who's going to be more likely to participate in that kind of survey different data collection methods have also been used so whether or not it's an interview administered questionnaire or self-completion and we don't have a lot of data really on on how these different methods are influencing response rates but when we think about sensitive questions like food insecurity questions we can probably guess that there's going to be some influence on how you answer those questions based on whether you're answering them yourself or whether someone's asking them and the last question I have is how different fieldwork periods may also affect prevalence so in the food and use survey in particular they're collecting data just over selected months of the year and again if there's seasonal patterns of poverty when different employment patterns we're going to have a different prevalence based on different months as opposed to surveys that are then representative each month of the year then of course each of these surveys are going to have their own response rates and we have to think about who completed each one but I wanted to share some data on how different components of the survey methodology we can look at how different survey methodologies are actually influencing prevalence estimates so I've flagged here the first thing I'm going to share with you is how the different respondents answering the food insecurity questions might influence the prevalence estimates that we're seeing so across these different surveys we're seeing different adults responding to the questions so in food and use they've asked any two adults from a household to complete the questions in the family resources survey the adult with the best knowledge of food preparation or shopping is supposed to answer the questions and then in the other surveys we have all adults from the household answering them so how does who is reporting on food insecurity with influence the responses so some early research was done in Bangladesh published in 2010 that's began to explore this question so in this study it was male and female partners who were answering the question and what these researchers found was that when people are answering experiences for themselves things like did they skip meals did they go hungry but were unable to eat there was greater discordance than in how household food insecurity was classified and that's because individuals in the household can experience those things definitely and there was definitely a gender kind of component to this so this study then raised questions about the assumption that you can ask any individual in the household to represent the household and their recommendation coming out of this was that we should look at food security as representative of individuals not attach an individual's responses to the household now this is different than how we're seeing the food security questions be applied in the family resources context where they're asking the person the most responsible for food and this might be a way around that because that might be the person who's actually best placed then to answer the questions for the household however I've just started an analysis this is very preliminary looking at data from the food and use survey where we have different adults answering the food security questions and there's also a lot of information in that survey about responsibility for food preparation and cooking so the first thing I wanted to highlight is that adults have different ideas about who's responsible for cooking so 32 percent of respondents said that they were responsible but this didn't match the other adults' perception where only 20 percent of them said someone else was responsible so how do we even identify who's responsible for food preparation and cooking and then the other thing is that 41 percent reported shared responsibility so again then who should answer those questions in a household survey I also just wanted to flag some quick data on discordance then so 17 percent of households were classified different into a different food security category crossing one boundary and then over 5 percent crossed into two or more boundaries so that's going to lead to different prevalence estimates of the different levels of food insecurity and then a last finding from from this preliminary analysis is that there was a significantly higher prevalence of severe food insecurity among respondents who are indicating that they're responsible for food preparation so suggesting then the influence of who's answering the questions in the household other thing I wanted to highlight from different surveys and how they're using measures from food insecurity scale is that quite a few surveys in the UK are just selecting a couple of items from the scale even though it's recommended practice to use the full scale and that's because every item contributes information that then allows a household to be classified this said there are a couple of different validated shorter scales that have been used so the two items scale based on less severe circumstances which will capture kind of the vast majority of people who then experience food insecurity and the six item scale as well however in some of the the UK surveys we're just seeing a focus on this cutting size of meals question or one or another item so when we look at the question did you ever cut or skip meal sizes only about 5% of households respond affirmatively to that question but that then contrasts with the overall prevalence of moderate and severe food insecurity so you can see why then a single item would underestimate the prevalence of food insecurity in the population that's because people experience different food insecurity in different ways so then if we if we just look at this one item only 64% of food insecure households are captured by the single item but a significant proportion are then not captured by this item so just caution then about using any single item on the scale I just want to flag then understanding society questionnaire from the COVID survey I did just use some individual items so again concerned that there was an underestimation other ways then that these questions have been modified also has lacked that key clause about lack of finances so reporting when someone experienced hunger but did not eat it's hard to know was that just because they forgot to eat there might be times when someone experiences hunger but has gone without eating and that was then rectified later in the survey where they included follow-up questions trying to get at why people had experience going hungry and not eating the other thing to mention here is this time period of just the last week being used and this is relevant because in the family resources survey we have a 30 day recall period whereas in the food and use survey we use a 12 month recall period and we can see that based on data from these two different surveys so from the FRS data and the FSA data there's a very large difference in prevalence even over the same time period and we can look at sub-questions within the USDA scale based on the food and use survey the FSA data where they're talking about whether they experience these questions sometimes or often and we see that less than a third of adult experience worry about food often so this does suggest that there's a fair degree of transience so that shows that it's important to measure over a longer time scale because in a single month food insecurity may be not experienced by many households who then experience it at other times of the year so just a couple notes on using food insecurity in the family resources survey we need to reflect on kind of who's answering the question so is it more appropriate to do a household level analysis or an individual level analysis and we also need to reflect the 30-day recall period in our reporting of these experiences and then I also just wanted to mention something that I've noticed in using the data is that we need to pay attention to missing responses because currently in the data release files so far people who are missing are counted as zeros and we need to think about that because of course that might be then underestimating severe food insecurity I think I need to pass it over to Ben so I just wanted to flag one last slide on the difference in food insecurity from food bank use and this is data from the food and use survey which measures both food bank use and food insecurity and I just wanted to flag that food bank use is severely underestimating prevalence of food insecurity and we see that even when we measure or look at food bank use amongst people experiencing most severe levels of food insecurity so another piece of work that I've been working on is trying to then understand kind of what is what explains the difference between food insecurity and food bank use and of course there are a number of different factors that are going to be influencing that relationship and so again we just need to have an immense amount of caution when we're using food bank use as an indicator of food insecurity and I am delighted to know that the family resources survey is going to be measuring food bank use I think already from this year so we'll be able to study this relationship in more detail I'll now thank you very much for your time and I'll turn it over to Ben for him to share some of the work we've been doing with family resources so Ben Thank you very much Rachel and so yes Ben if you want to get yourself ready so Ben is a reader in sociology and social policy at the University of Kent who from August he's moving to the Centre for Society and Mental Health at Kings College London and he's co-lead of The Welfare at a Social Distance Project a large study looking at the benefit system during Covid-19 and that was funded by the ESRC as part of the rapid response to the pandemic and he's had got interests in disability the benefit system and the nature of work and today his presentation is going to address the question did the £20 per week uplift in universal credit reduce food insecurity among claimants in 2020-21 Brilliant thanks for the intro and sorry for having such a wordy intro that I thank you it turns out listening to it back Firstly my apologies I wasn't around earlier on today my son is ill and off nursery today so I had to miss this morning because of childcare but Rachel did send me some notes which was really helpful and I will listen back to the recordings where I can do so apologies if I go over anything that was discussed earlier and also I should say that what I'm about to present to you is joint work with Rachel who's just on a fantastic presentation on her wider work and Aaron Reeves and the bit that I'm mostly going to do in this project is not the bit that's been done so far so most of what I'm going to show you is Rachel and Aaron's work so far The other thing I want to say to you before sort of really getting stuck into this is that these are really really preliminary results that I'm going to give you about the impact of the £20 universal credit uplift on food insecurity Rachel and Aaron and I ask you not to tweet about these results not to kind of publicise them or cite them at this stage we are working really hard to get this out as quickly as we can do but we want your feedback on whether these results seem plausible and robust so that we can do something that we're really really confident of so a caveat at this stage and it would be really interesting to get a conversation with you about which things you found convincing and which not convincing but the FRS data that we're using here only came out a few weeks ago and some of the other data that we're using came out even more recently so that's why this is sort of very very preliminary results so having done a lot of initial bits at the start let me get to the heart of what we're doing here so there's obviously a lot of debates about the cost of living crisis at the moment and also many debates about benefits adequacy and whether the level of benefit is really sufficient for people to be avoiding not just kind of technical income poverty but quite severe need and destitution within this the sort of two key moments I suppose that in this project here we're really focused on the first one is that as COVID-19 hit in March 2020 the very first budget in response to it raised the basic element of the standard allowance in universal credit and also working tax credits by 20 pounds a week so about a thousand pounds a year which is it was a noticeable raise to the the basic element was about 75 pounds per week at the time and though there's all sorts of other additions that different people receive so that's the first moment and that's that raise of 20 pounds a week is known as to many people as the uplift and then in October 2021 this uplift was removed but given a context of the cost of living in crisis and a lot of political pressure about a third of the cost of that was given back to some claimants through increased rewards for working partly through the taper rate so how quickly universal credit is withdrawn as you earn more money and partly the allowance so the amount that you can earn before you have to start getting it reduced by the taper so those are sort of two key policy moments around this 20 pounds a week uplift and a lot of the conversation about benefits adequacy has been around this we could come back into the discussion about you know whether we should be having a wider conversation about benefits adequacy but still these two changes allow us to have a look at the link between our benefits generosity and food insecurity in the context of the current system now some of you may be thinking well why is this interesting because it is clearly obvious that we would expect the amount of income that people receive particularly people towards the bottom of the income distribution receiving universal credit to be linked to food insecurity but as Rachel has been explaining in detail you know that we're really interested in food insecurity as a marker of deep poverty there's a series of reasons why you know the extent of the link between the 20 pounds a week uplift and food insecurity does matter for current debates and does matter for policy even if obviously we're not expecting it to have a positive effect on well so we're not expecting the removal of the uplift to have a positive effect or the addition of the uplift to have a negative effect we have a clear idea of the direction but the extent is something that is important so to try and have a look at the impact of the uplift on food insecurity we're using a quasi-experimental approach using two different datasets so the first dataset is obviously enough for today the family resources survey 2019-20 and 2020-21 looking at the introduction of the uplift and we're also looking at the welfare at a social distance survey of claimants which was May June 2021 and May June 2022 to look at the removal of the uplift the welfare at a social distance project is one that I'm co-leading and was doing large surveys via Ugov of benefit claimants in 2020-2021 and 2022 and I'll come back to that a little bit later on but primarily here we're going to be talking about the FRS results so just the introduction of the uplift rather than its removal and the overall logic that we're using is that people claiming universal credit and working tax credit receive the uplift but legacy benefit claimants so in these analysis that's job seekers allowance employment and support allowance housing benefit income support and carers allowance didn't receive the uplift unless they were also receiving UC so we have a good comparison group we can have a look at changes for universal credit claimants over time versus changes in legacy benefit claimants over time in their food insecurity but we also need to account for the changing composition of claims during COVID so just to give you a bit of a feel of this this is work that we've been doing using the FRS data and these are current universal credit claimants and their characteristics in 2019-20 versus 2020-21 and you can see for example that there is a very sharp rise in proportion of universal credit claimants that are cohabiting there's a marked rise in the number that have degrees and a marked rise in the number of owner occupiers together with a lesser rise in the number of private renters so there's a lot of different compositional effects here that we need to take account of so the way in which we are trying to make this comparison valid to get at the actual impact of the £20 uplift is a kind of a doubly robust method sometimes known as a doubly robust method where firstly we're doing matching over time on key variables so the idea is we're getting rid of a certain proportion of the sample where we can't find somebody in 2020-21 who is similar to somebody in 2019-20 and the ways in which we're deciding if people are similar are based on six variables so ethnicity marital status children qualifications work status and housing and those are all things that we see particularly sharp changes between 2019-20 and 2020-2021 and again we can come back to this in the discussion but there's a lot of different ways of doing matching we're here using exact matching so we're only comparing people that are sort of exactly the same as each other according to these combinations of categories and the result of that is that 40% of the original sample is unmatched so we end up with a sample of about 1900 people rather than I think about three and a half thousand so that was the first step in the doubly robust method we're only looking at people that are sort of broadly similar to each other on these characteristics and then within our matched group we're doing a regression analysis so we're controlling for all of those things above but also some additional confounders including sex, age group, disability and occupational group the reason that we're not matching on all of these variables is because we're using exact matching and if you use too many variables with exact matching you basically end up losing most of your sample so like in lots of different matching methods you end up with a bit of a trade-off about how exactly doing you're doing the matching and making sure you have enough sample left in order to get reasonably precise estimates so again we can come back to this and we are very interested in whether you think this general approach is convincing or not so with that in mind well let me just firstly before getting the results just mention some of the challenges around using the FRS here and I'm really sorry for having missed the discussion earlier where I'm sure there was much more discussion around this there's sort of two general issues with the FRS that we obviously have to bear in mind one of them is the changing survey mode the other is the changing response rate both of these I'm sure have been mentioned over time it should it's worth noting that the welfare to social distance survey that we have doesn't have a changing survey mode or a changing response rate over the period we're looking at but on the other hand it has a much worse sampling method it's using a UGov kind of advanced quota sample in a way rather than a stratified random sample so both of our two data sets have limitations but they're just different limitations which hopefully means that they're complementary but I hope is that there's no real reason to expect that this is going to bias our kind of difference in difference estimates in the end no reason to think that changing survey mode is going to impact on legacy benefits differently to have it impact on universal credit claimants there's also some limitations around FRS in terms of benefits claim variables and as far as I know this was only briefly touched on earlier one of the issues is that there's relatively limited information on universal credit in FRS in particular FRS as far as we can tell asks different questions about claimants of different benefits and because there's lots of things we need to know about UC people are not asked when they claimed which is a bit frustrating because it would be useful for us to you know to have a look at pre-pandemic universal credit claimants would be helpful if somebody in the FRS team is thinking no that data was actually there please do let us know because there's some slightly puzzling variables in their benefits dataset but as far as we know there's a bit of limited information on universal credit and there's also a really big issue that I think everybody doing survey work on universal credit needs to bear in mind is that it's actually very difficult to know what a claimant is for universal credit partly because a large minority of people who are technically on universal credit get no money in any given month and that was particularly true given the pandemic where they allowed people to claim no money for longer before closing their claim and also because people have five weeks before their application for universal credit and receiving any money unless they get an advance so what people mean when they're responding to questions about their claims is is a bit unclear and administrative data here would be enormously helpful as in when we're able to get that but we don't have that in this analysis at the moment so just very quickly then in my final so the five minutes or so four or five minutes just to show what we've found and to give you some questions so the raw data suggests that the introduction of the 20 pounds a week uplift for universal credit had a big impact on moderate or severe food insecurity among clients now I'm firstly going to show you this raw data this is from some work that Rachel put out on Twitter there's a link in slides when they first came out so you can see in the raw data among households receiving universal credit about 55% were marginal, moderate or severe severely food insecure each month in 2019-20 going down to about 40% in 2020-2021 and you can see among legacy benefits claimants in this case housing benefit claimants or USA claimants there was no change so this is really really suggested that the 20 pounds a week had a big impact but as we said there's a lot of compositional changes going on here that we need to take account of so after we count for all of these compositional changes in our doubly robust method what we find is pretty much the same thing so this is now moderate and severe food insecurity we're excluding marginal food insecurity but we'll come back to sensitivity analysis that do that but this is our main analysis and as you see among legacy benefit claimants there is very little change in food insecurity over time but among universal credit claimants there is a really really sharp drop we estimate that the effect is 13% percentage point difference and the confidence interval are quite wide around that I think about from 3 to 20% or so but so we are finding that there does seem to be an impact a large impact of the 20 pounds a week uplift on food insecurity so saying this please do remember the caveats at the start this is the first time we have told anybody outside of our team about what our results are so don't tweet about this or tell others about this stage but we really do want your feedback on how convincing this is and any other things that we need to take into account before we publish these results and in particular who maybe sort of just three things to focus on firstly in these analyses so far we are only looking at one adult in the household and that is the person that has says they're responsible for food purchasing and consumption in the household and we're only looking at their benefit claims so Rachel was talking a little bit in the previous presentation about going between the individual and the household level and how we do this in FRS so your views on whether we need to do something differently here in how we relate the individual and the household is really helpful secondly about in general whether our causal claim is convincing given the response and mode changes in FRSA for this period and giving compositional changes in who is claiming benefits and finally you know to what extent are any concerns you've got addressed by the fact that we have this complimentary data from the welfare social distance project which is worse in terms of its sampling methodology but better in terms of its consistency over time and which also enables us to look at the withdrawal of the 20 pounds a week uplift as well as its introduction so as I said we're really really interested in your comments about this and also Rachel's earlier presentation and a massive thanks to Rachel and Aaron for working with me on this collaboration and really interested in the discussion