 Yn y cwmhreifft, ddim yn fawr i'r cyfrannu ar y cwmhreifft ar y cwmhreifft. Yn y cwmhreifft, mae'r cyfrannu data ac mae gennym ni'n cyfrannu ar y O&S. Mae'r ystafell yn ymdweud o'r cyfrannu cyfrannu cyfrannu ar y Llyfrgell. A'r cyfrannu cyfrannu ar y Llyfrgell ystafell. Felly, rydyn ni'n gweithio'r cyfrannu ar y Martina ac Petia. Felly Martina Helm yn y cyfrannu cyfrannu ar y O&S, y cyfrannu cyfrannu ar y Llyfrgell a'r Cyfrannu Cyfrannu. A Petia, y cofnodd, yn ymdweud y cyfrannu cyfrannu ar y Llyfrgell ystafell. Rydyn ni'n gweithio'r cyfrannu cyfrannu ar y Llyfrgell ystafell. Ond mae'r cyfrannu sylwagaeth fadeol, yn cyn() o'r cyfrannu cyfrannu cyfrannu. Felly, oherwydd i'r rhan o'r rhan oed yn ydych chi, rwyf wedi'u cyfraith yn 2020, yn ydych chi, sydd ar y cyfraith o'r pwn i'n gweithio yma, ychydig iawn i'r cyfraith a'r gynghwyl wedi'u ddweud ydw i, nid i'r cyfraith o'r cyfraith a'r gyffredinill yn y ddym ni'n gweld yn hynny. Rwyf y gallwn ar y cyfraith o'r cyfraith o'r cyfraith o'r cyfraith o'r cyfraith o'r cyfraith o'r cyfraith. A gael hynny i ddim nesaf o'r ddweud yn gweithio'r gweithio, dwi'n ddwy'r ddweud o'i cyfle gysylltu Gwyrdd LFS yn y Cynllun 2020. Dyma, dwi'n ddwy'n ddweud o'r gweithio bobl pan ddwy'n ddweud. Ond oherwydd mae'n ddweud am y celfon yw'r interview ac mae'n ddweud o'r ddweud o'r ddweud o'r ddweud, mae'n amlwg hynny o'r ddweud o'r ddweud o'r ddweud o'i ddweud o'r ddweud. a we did that through various different ways. Previously, we telemaged only addresses that located north of the Caledonian Canal for telephone interviewing, so we extended that to the entire sample that only gave us about 67% of phone contact details for about 67% of cases. So, we then adapted our advanced material as well and implemented an online portal asking residents of our sampled addresses to provide us with phone contact details through the portal. And then interviews could follow up with telephone interviews. We also got phone contact details through people contacting our survey inquiry line or by contacting our interviews directly, gave us another 67%. But the biggest part of phone contact details we actually obtained through a sort of field tactic that we refer to as Nocturnage. We gained through about 55% to 60% of phone contact details for our cases. So, I wanted to explain a little bit more what Nocturnage is. So, we introduced this in April 2021 after testing it on a couple of our other household surveys, which showed to be quite successful in increasing our response rates. And as part of Nocturnage, we basically sent interviews or face-to-face interviews to cases where we weren't able to obtain any phone contact details and interviews will then make up to three call attempts to obtain a phone number at the doorstep and then follow that up with the telephone interview successful. And we started Nocturnage only on Wave 1 cases to start with, but then extended that to Wave 2 plus cases as well, where these were sort of in the area of Wave 1 cases or where we weren't able to make a successful contact because the phone contact details were perhaps wrong or they were supplied late in the last wave. And Nocturnage, I wanted to show you on the next few slides what success we had with using Nocturnage. So, here's the slide that shows the response rates over time in 2019-2020. So, you can see that before the pandemic, we had a fairly healthy response rate in the mid-50s that obviously then drastically dropped with the start of the pandemic to the mid-20s and stayed around that level until we introduced Nocturnage in April 2021, which we achieved around a 10 percentage point increase at the time and that level then around the late 30s over the summer. You can see a bit of a dip towards the end of last year, but that was not down to Nocturnage losing its effectiveness. That was down to Omicron really playing a part here. Obviously interviewers couldn't go out due to isolation rules and the capacity was stretched for that reason, so that's why we're seeing a bit of a dip there again. But in general, obviously Nocturnage did help bringing our response rate up a bit. Obviously not back up to where it was before as we didn't do it to the whole sample just to a part of the sample where we didn't have any phone contact details or where we failed to make contact. What impact did it have on our achieved samples? So pre-pandemic, you can see here the sample size for the LFS wave 1, again dropping drastically in March 2020. As I said earlier, we then tweaked our sample or doubled it actually with July 2020 and that meant that we actually then had a sample size that was a bit bigger than before the pandemic, so we tweaked it a bit towards the end of the year to bring it down to about the same level. And Nocturnage really helped us to maintain that so far, which was great. Now that showed us at the moment the quantitative impact that Nocturnage had. I also wanted to show you a few slides now with the qualitative impact that it had. So on this slide here, it shows the distribution of tenure, which is normally something where you don't see a lot of movement over time. So the left bars on the side basically showed a distribution of tenure before the pandemic and the green middle bar shows what the distribution of tenure was as we switched to telephone mode only. And we can see here that we got more owner occupiers and fewer renters in the sample in wave 1. And when we then introduced Nocturnage to the mix, this is the purple bar in the middle, we can see that we are getting more of the harder to reach sample members back into the sample, so more renters and fewer of the owner occupiers. And when we look exclusively at those cases that were subject to Nocturnage, which is the right dark ring bar, we can see that the proportion is pretty close to the pre-pandemic level. Similar movement can be observed with individual characteristics such as age, which shows that we got more older and fewer younger people in the sample as we switched to telephone mode and again Nocturnage helped us bring that back into the right direction. Similarly NSSEC, we had more higher skilled and fewer routine workers in the sample and again Nocturnage helped to bring that back to the previous level. Material status, similar picture, fewer single, more married respondents. And nationality, which we had a big debate over as well, we can see that we had more UK and fewer non UK respondents in the sample. And the movement that we observed in tenure as well as the one with nationality and country of birth, both of these led to some adjustment in the waiting. And this is why I hope I can hand over now to Petia to talk us through the adjustments that we made to our samples. So just checking now whether Petia can use her audio successfully. Hi, can you hear me? Yes. Fantastic. Amazing. Thank you. Okay. Thanks Petia. Can you see the slides? Okay. Yes, perfect. Everything else is audio. Fantastic. So I'll go over to the next slide then and over to you. Thank you so much. So yeah, I'm just going to go over the methodology adjustments that we introduced following the changes in the data collection and the pandemic that introduced quite a few biases into surveys as well. So the first thing we did was to add a non-responsive adjustment prior to the change in data collection that LFS did not have a non-responsive adjustment. So what we did now is to adjust the design ways of the household that joined after the change in the data collection. So from week 10 of quarter one of 2020 onwards, we adjusted these design ways by calculating probabilities of responding. We used logistic regression to calculate these probabilities and we used GB region and aggregate level census factors of index of multiple deprivation, output area and open ruler as covariate to calculate these probabilities. Just to note, this was only done for GB as Northern Ireland postcodes were not available last year. So we weren't able to link Northern Ireland to the census factors. We needed the covariate. But now during the 2022 reweighting, which we're going to be starting just recently, we will include Northern Ireland as well. So this was the first thing that we looked at. Next slide, please. The other thing we wanted to see if there's any variables that we can actually use to help us mitigate the biases introduced in the long term. And tenure particularly was key here because what other factors around labour markets or migration or health awareness will eventually cause significant change over time in LFS output. The proportions of the housing stock that are rental or owner occupied are far more stable historically. As you can see on the screen, the distribution for tenure from 2018, the beginning of 2018 up to the end of 2018 is fairly stable. Because it's relatively constant, IA is relatively predictable for us. This is a suitable variable to be used in mitigating action in the short term and the medium term as well that was introduced and introduced biases into the sample. Indeed, we noticed differences in proportions responding to the sample with other characteristics as well such as ethnicity, disability status, nationality, country of birth. So it wasn't just tenure, but we focused on tenure because it was so significantly affected by the changing collection and the pandemic. And because as a variable, it is far less susceptible to large changes compared to some of the other variables we collect. Furthermore, the distribution of other variables of concern are actually correlated with tenure. So if we address tenure, we are also impacting other outputs as well. So what we did is to use the average LFS tenure distribution across the four quarters of 2019 as a basis for waiting in addition to our prior already existing calibration plans. We started doing this from JM 20 onwards. The general effect of the introduction of the tenure waste is that those characteristics that are more present in owned outright housing will be reduced in the weighted estimate. Will those characteristics more prevalent in rented housing will increase? And this will affect not just personal characteristics, but also employment status and type of employment as well. Just to note that we are aware of the caveat that the way it comes from the 2019 survey. And without an external source to update them, they're not really suitable for use in the long term. But in the short term, they did have a good impact. Next slide please. So the impact of the tenure waste was as expected. So once we introduced them, we could see that the unadjusted unemployment rate got higher and the adjusted employment rate got lower as we expected. Now, what we also notice as well is that the introduction of the tenure waste, both other characteristics closer to the long term friends such as the number of people born in the UK, number of people meeting definition of disability, et cetera. So the estimates produced across a range of variables after we introduced the tenure waste became more consistent and credible with other external information and long term expected trends as well. However, we wanted to see if there's anything further we can do. Next slide please. We noticed that country of birth is a particular concern as well. So the tuition rates decreased in the UK born after the pandemic, but they increased in non-UK born. And we observed this across all age bands that we used. The EU born started dropping out of the sample more than the EU, non-EU born as well. Luckily for us, there was actually an admin source that we could use to try to mitigate this. Next slide please. So we identified the RTI administrative source. This is real time information. It comes from the HMRC. And RTI is the complete coverage of payroll and foils including binationality. So this was a good opportunity for us to have an admin source that we can compare to the RTI data and see if first we can actually identify the bias and then if we can mitigate it. So LFS based growth rates for year-on-year percentage changes for non-UK decreased much more between October, December 2019 and July, September 2020 compared to the RTI based growth rates. So this shows us that the LFS estimates suffer from bias stemming from differential non-response between UK and non-UK born. And the additional tenure constraint that we included hasn't really reduced this bias efficiently. Next slide please. So what we wanted to do is to find a way to use the RTI data to estimate year-on-year population growth rates for EU and non-EU born subpopulations. And we needed to get that for each rolling quarter in 2020 onwards. There were very few observations from the RTI data set and it wasn't really suitable to fit a statistical model based on these few observations. So we had to use very simple assumptions to derive an expression of change in the population growth rates in terms of the change in RTI based employee growth rates. We wanted to see if we can track the RTI based employee growth rates and see whether this is going to translate to population growth rate changes. In order for us to actually validate the estimator and validate and prove the assumptions that we made, we need to choose base periods as well. So we looked at October, December 2019 because this is the last period before the pandemic where the population growth is known. The assumptions we made were that any change in the population growth rate of the non-UK population is going to be in the same direction as the change in the RTI employee growth rate. So we wouldn't expect RTI employee growth rates for non-UK born to be increasing while the population growth rates of the non-UK born is decreasing. We also expected that the magnitude of change in the population growth rate does not exceed that of change in the RTI employee growth rate. Because the RTI is only of the employees whereas the population will include employees, self-employed, unemployed, etc. So we wouldn't expect the magnitude to actually be disproportionate. So given these assumptions, next slide please. We selected a very kind of quick method which is three steps and we essentially wanted to check if we can estimate the population growth rate based on the RTI growth rate. So if we let TETA denote the RTI employee growth rate of the EU born population between JS20, so July, December 20, and JS19, we also adjusted RTI year-on-year percentage changes in any subpopulation by differentiating the UK nationals. So in the equations if we take TETA RTI for JS20 for EU for example, the adjusted version of that is the TETA RTI employee growth rate for JS20 for EU minus the TETA RTI employee growth rate for the UK. We removed the UK because we wanted to account for background change in employment in the country. So if we let Y denote the population growth rate of the EU born population, we would actually assume that the population growth rate for EU born in the pandemic, the YJS20 EU adjusted, is going to be proportionate to the growth rate in the RTI. So the population growth rate, the last equation on the slide, the population growth rate for JS20 for EU born adjusted minus that from the base period, which is all D19 for the EU, will be proportionate to the RTI employee growth rate for the same period minus that base period. So essentially we are looking at proportionally estimating this and we have published on our own website proof of this estimation method and we did show that the change of the population growth rate between a base period and a period from 2020 is approximately proportional to the change in RTI employee growth rates. We also choose a proportionality factor so B to be set to 0.5 because this will be a positive number, but we set it to 0.5 to minimize mean prediction errors. Now after we've decided on the method and after we did proof of it, we wanted to compare it to external sources. Next slide please. We first looked at LTIM, so this is the Long Term International Migration Estimates. All the numbers that you see on the slide are presented in the thousands. So we compared the methods and estimates to the LTIM and this is year on year changes. We noticed that the year on year changes between RTI based estimates and the LTIM are actually fairly close. And the mean percentage deviation for the estimate that we produced on just the RTI growth rate is very close to 0 and the mean deviation itself is only 2. So this gave us the confidence that the developed estimator is actually approximately unbiased. The bottom table shows the year on year change in the population total of the UK born and non UK born populations and of the whole population using RTI and then only the LFS without the RTI adjustment. We can see that in the first quarter there is very little difference between the two, however that difference increases with time. The differences between estimates by country of birth however are much larger and they also increase over time. So it is clear that the unadjusted LFS responses under represent non UK born. So in order for us to fix this, the introduction of the RTI based estimates did bring our estimates closer to other external sources. And it fixed that under represents the number present in the data at least to an extent. Thank you. I just wanted to follow up with just a few notes on what's ahead of us now in 2020. So I explained earlier about our field strategy and obviously we're still exclusively telephone interviewing on the LFS. The UNS has started some face-to-face trials end of last year on a small scale looking at the practicalities around that going into people's houses again. And the first findings were quite promising. So a further more large scale trial is planned in spring this year. Although that won't be done on the LFS so it will be done on some of our other surveys and we then will consider the outcomes of that before this might be rolled out on the labour force survey. Petia briefly mentioned earlier about a reweighting over the course of 2022. So obviously we completed the reweighting exercise based on the RTI data over the course of last year and we're planning an update to that based on the same methodology over the course of this year on using the latest RTI data. And as further sources for estimates of the population become available then from the census later this year we'll review the methodology further and further reweighting may take place possibly next year. But plans around that haven't really taken shape yet so there's still a lot of debate over that. And with that at the end of our presentation apologies again for the technical hiccups we experienced but hopefully you were all able to follow the content regardless. Fantastic, great. Well thank you very much and I think we shall move on then to the next talk. Great, so this is the transformation of labour market statistics with James Harris. Okay, and James joined the ONS working for a few years in population statistics on both estimates and projections for moving on to work with local authorities across the country and all types of sub-national statistics, including businesses, trade, jobs, wages, health and demographics for almost a decade. He is now leading the labour force survey team as they work through the transformation to new methods of collecting labour market data and this is what he's going to be updating us on today. Okay, over to you then James. Great. Can you hear me now? Yes I can. There we are, I'm on hooray. Right well hello everybody. So thank you very much for the introduction. Good to see you all. Glad to see so many people here. So my name is James Harris I was previously working in sub-national statistics now head of the labour force survey. Just letting you know a couple of my colleagues on the line as well so obviously you've heard from Martina who's in charge of the LFS production. You also have my colleagues Jason Zawadski. So he's the divisional director of our whole division now. So my manager formally in charge of the census 2021 operations. So very, very aware of what's necessary for the operations of running surveys and data collection. And that's now of course slowly tying up and he's now moved across to work with us in terms of the transformation of labour market statistics. And also all a phrase on the line who's in charge of parts of the transformation. So looking off the labour market statistics transformation aspects in particular looking at the quality and the design aspects of everything. So what I'm going to talk about today is the transformation of labour market statistics. But more importantly, this is what I'm going to cover so our vision for the transformation. So what we're going to be delivering so what you can expect to see and then touching on what to expect to come next. So first off our vision for transforming the labour market statistics where we're going what we're doing why we're doing it. So there's been a very long history of labour market statistics as I'm sure most of you do know we've been running the labour force survey for almost 50 years. Technically we're heading into our 50th year I think now we're producing regular cross section on labour market statistics and estimates. So this has been in production for a very long time and you've already heard from from Martina the developments that we're undertaking to make sure that this continues that we're able to continue producing these numbers. But more importantly than that perhaps enabling a wide variety of articles and analysis of workers and their characteristics and circumstances. So internally with NONS producing outputs and analysis and breakdowns think an awful lot of activity of course to the current pandemic we've been producing over the last few months. Our colleagues all across the rest of government so DWP, DFE looking at aspects of the labour market monitoring the developments and so forth. And perhaps more importantly all of you on the line or use of the data of the information of the available resources that we have to produce your own articles your own analysis like we had presentation from Nye earlier on looking at the breakdowns of the labour market about the population about the different aspects of society going on at the moment. And of course speaking of society we're trying to evolve to over time to the developing changing needs and the shape of society so adding questions subtracting questions when things become important. So, you know, over the course of time that last few years we've had Brexit on the horizon that we've had the COVID pandemic. We've had an awful lot of changes the new industrial classifications new occupational classifications just to example so making sure that we're keeping it and following what's happening over the course of time. But through that through all our engagements through conversations with with you with everybody with colleagues internally recognising all the various questions suggestions issues and is that not only our colleagues but you the users have been raising over the years through the different format through the different forums that we have. The communities and local information partnerships groups and all the different outreach programmes that we have so an awful lot of information that you want want faster and more frequent outputs. Ideally want monthly estimates or indeed the RTI information theoretically it's real time but we'll see what we can do but ideally with monthly as a core target right now having more robust and more detailed data on the character particularly on characteristics of interest. So you know the personal characteristics of who the people and who the workers are and what what what their social is having being able to be more flexible being able to faster to respond to the changing needs of the day. So what the policies on the horizon and changes that are happening what what information do we need about the changing labour market on a you know day to day basis and what's changing over the course of time. Big of course aspect in terms of high quality data quality takes all sorts of different aspects that we will touch on more in a moment but just fundamentally higher quality better confidence intervals so that you're more certain more sure of what we're actually producing. Of course you also want it easier for the public part so in parts to improve the response rates to reduce the respondent burden but fundamentally so that the user the respondents to the survey actually are able to participate were able to understand the questions they're able to respond more easily more quickly and and then feedback their thoughts on what the correction is. And the resilience of what we're doing so making sure that what we're doing is sustainable and resilient so we're able to continue production we're able to continue the collection we're able to do what needs to be done to produce and publish the day. Especially during times of uncertainty like covid so martin has already been through a couple of the approaches that we've had introduced to make sure that the election is is robust and working effectively. And of course we're trying to tell up and change things to make sure that that's consistent in future as well. So what's our broad vision so the transformation fundamentally we're looking at delivering statistics for the public good that's the vision for the whole whole department but we're looking in particular at integrating surveys sensors and administrative data as the broad aim right at the top but beneath that in terms of specifically Labour market statistics we're looking at producing more coherent more granular more timely statistics in particular being responsive to the user needs. So what the what variables people need one information people need what we need to be monitoring in terms of the labour market and of course trying to reduce the costs and the burden both for government for businesses and households and make it easier to respond to. Now I mentioned in here of course trying to add more sources as well so trying to add administrative data trying to add new linked information so that we understand people and the society even better. It's a complicated process trying to link up all the different administrative sources you know the HMRC PAYE data the real time information information of the health records and all these different things trying to look at what what's situation people are in trying to make sure we're in. So that we have as much information as we possibly can from all these different records but of course that that comes over the course of time with an awful lot of research. We're making sure that this is an online first responsive design so we're trying to transform the surveys to make them more effective more responsive so that fundamentally we can improve the quality of them so trying to gather a larger overall sample size with you know more robust processing systems so that we're able to produce the information faster and more effectively trying to build the modular design to integrate more question blocks so you know when users come along when colleagues in the government come along wanting to know more information different information new questions new new policies on the horizon new topics of interest. Trying to build things effectively so that we're able to add in new modules add in more questions in a simpler manner in a faster manner as well and being therefore being more flexible more able to respond to the fast changes that people actually need in particular that the speed aspect to it so that we know it doesn't take massively long to add new questions that we're able to discuss and negotiate and figure out what's necessary from a question point of view and trying to add that in in a matter of months a week. Rather than years that can sometime to happen and of course things I have been changing over the last few years I mentioned the industrial and occupational classifications as two basic examples but making sure that we are updating and upgrading the questions and the response categories that people have so that you know so that we're up to date we're current we know current situation is with the society and population. We're using the correct definitions we're using the harmonized classifications wherever possible changes with things like the census so that we're consistent with other sources as well. And at the bottom of the slide here in particular looking at the improved ability to monitor progress so making sure that the collection is effective and appropriate and the things like the knock to nudge processes are effective and working and moving responses in the right direction. We're building into our longer term aim I think called I packs and perhaps it's an internal term but the integrated population and characteristics survey. Fundamentally trying to build all these sources together into one coherent source of information that we can understand the whole population and society so trying to build a stable core system. All on this this common capability so that all these sources are built in similar fashion that we can link the information together more easily that they're capturing all the relevant variables that need to be that we're meeting the survey needs not just your current needs not just what we're currently producing but also looking at future needs you know the the government's leveling up agenda for example on on on the cards right now trying to collect make sure that we have the information that we need not just now but trying to plan for that. The next six months a year 10 years if necessary. But that that's the kind of the future aspects of what we're going hopefully to be delivering over the coming year few years but specifically what we're working on right now in terms of this particular transformation so moving from the vision to the actual delivery. So we're going to be obviously continuously producing the labour market data continue continuously delivering it we've been producing this information for as I said some decades already but making sure that we're continuing the production as as as normal. Trying to provide similar data sets that you're all currently receiving so it may not be exactly the same. It might be a slightly different format so see it rather than SPSS for example maybe slight differences in the variable names like differences in the content that you're getting. Hopefully when I say differences in sample size hopefully a larger sample size so a larger sample size hopefully leading to larger number of responses, meaning that you're able to do more granular granular analysis larger larger proportions of people in each category that you're then able to draw better conclusions from. And of course you'll still be getting your your person your household and your longitudinal views. So all of that continues to be produced and it's a part of the transformation designs. We have, of course, refreshed and updated the content of variables for the latest definitions so making sure that we have the correct you know ethnic categories that match up with the census the correct religious categories the correct response categories to the industrial and occupational classifications and everything else. Trying to you know give robust and detailed information about people's character personal characteristics and of course some things will have changed so you know age and sex and ethnicity will not have changed radically over the course of time, but some variables will have come and gone and updated over the course of time. So well we're reviewing the effectiveness of all of them and making sure that we're asking the right questions to get the right answers. And of course, fundamentally this is all leading towards having higher quality data so not just the sample size, but making sure that we have higher quality data. So we have three core targets that we're aiming for and all is on the line if I say anything wrong here please do correct me but fundamentally trying to reduce the bias so achieving a representative sample that we make sure we're capturing the whole of the population in the right way that we're representing representing all the different categories and groups of people. Focusing on both national and local improvements so trying to minimize the variability between different regions between different indices of multiple deprivation categories between different output area classification categories to make sure that we're not mistargeting anything disproportionately. And as I say trying to make sure that we have proportional samples by all the different personal characteristics so age, sex, disability, tenure, ethnicity, the key variables that we know everybody needs. Trying to reduce the attrition as well so making sure that it's not just wave one responses that we're getting but that once you're in the survey hopefully we're continuing to get your continued engagement through wave one to two to three up all the way up to five. So hoping to reduce the drop off in response rates between waves, trying to ensure that we have a sufficient sample size by the time we get to wave five, and whether or not we get a sufficient sample size, trying to make sure that there's no bias in the changes to the sample that it's still the same age, sex, disability groups and so forth that we're representing the population appropriately through the process. And of course hopefully try to improve the response rate throughout the whole process so reducing the operational complexity reducing the respondent burden such that we're still able to meet all these quality targets that we're able to increase the response rates and hopefully not introduce any new biases. And of course, over the course of time, building the capability not just now but for the future so I've touched on what we're building right now that we're building, you know, transformed system building a better survey better way of sampling and so forth. And over the course of time we're aiming to add more functionality to the survey so hopefully, as I said earlier, faster response to emerging needs that we're able to capture new questions and new topics of concern as they come up engagement with users. Such as yourselves, trying to add the functionality for additional modules making trying to make things more frequent. So we may not necessarily be delivering that in the coming few months or year or so, but adding the capability that we will be able to add modules and add more frequent data and trying to better integrate things into different production systems. I won't give you all the acronyms, but we have a bunch of different production systems within on s and across government and I'm sure some of you have your own production systems that use the LFS data as well. So trying to make things more more effective more more tech friendly as well. And working with our publishing colleagues so the web dissemination side especially trying to build more exciting ways of viewing and interacting with the data. New means of accessing the information new means of presenting the information new data visualizations for example new data science techniques trying to look at the information in new and exciting ways. So over the course of time trying to build these capabilities. I mentioned in terms of the questions so looking from a first principles approach so not just taking a question at face value and just sticking on the survey, making sure that we're answering the right question that we're meeting the user need. It's not just a definition of a variable. It's actually meeting the what people need to know that we're asking the right question or multiple questions to get to that final requirement at the end, and engaging in an awful lot of cognitive testing with respondents with people answering the survey. To make sure that we're getting the responses that we expect that they understand it that it's received in the right way that it's all understood. And we have extensive systems and flow testing to make sure that the questionnaire is designed in the right way that the questions are asked in the right order that the routing is correct and that we're getting the right responses that we're expecting and the people aren't randomly missing questions or skipping questions or whatever. So making sure that we have all these extensive systems and testing all in place and wherever possible trying to maintain consistency with ons standards with gds standards with office for stats regulation standards with standards set for for example with the authorities office, all the different international standards as well in terms of industrial and occupational classifications. So wherever possible, trying to maintain consistent with all the different standards that are available, and indeed hopefully with the time series so you don't see a much radical change over the course of all the different time series in terms of the data sets that we produce. One touching briefly on the data collection flow because this is one change that's rather significant so currently or this is changing a little over the course of the pandemic but the currently the process is that we issue the survey to the respondents, and either they respond face to face or they respond by telephone obviously over the course of the pandemic that slightly changed so this is a very simplified model here, but fundamentally, somebody turns up at the door knocks the door you know. Hi, we're from the ons. We'd like you to engage in this survey, and before the pandemic and hopefully after the pandemic, either we have a little face to face interview there. With the person so we get all the information as accurate as possible, or as it has been over the last few months, engaging in the telephone capture so making sure that they're able to answer the telephone survey by the telephone, making sure that they have all the information that they need, making sure that, you know, encouraging them to engage with the process. And then, once we've got that that telephone capture, we either have a response or a refusal or a failure out of the end of it but trying to actively promote engagement with the survey and with the collection. Over the course of time it was normally with the LFS it used to be face to face first in wave one and then following up by telephone, but now we're going for an online first approach. So, again issuing the survey, hoping that people respond online, hoping that we get the right responses. But if they're not responding online going through the process of telephone capture, whether it's the telematching that Martina mentioned, or whether it's a knock to nudge process to find people's telephone numbers. Maybe it's they responded in wave one and then we're phoning them up for wave two just to make sure that they understood and they're able to respond. But some sort of process here trying to get an online response or a telephone response possibly not with a knock to nudge process trying to encourage that. And in the course of time, hopefully we're aiming to add active field interviews into the process as the final option available to respondents if they wanted to. Fundamentally trying to get a response out of users. So, essentially the same process, but if I just skip back previously there wasn't an online option it was just feel a face first and then telephone first, but now it's online first telephone second and field third. So, that's what we're hoping to deliver, but what to expect next what's coming so obviously we're delivering this transformation so we're going to make iterative improvements over the course of the year. So, we have an online mode that's already live, we're adding telephone mode coming up very soon so in the next month or so hopefully adding the telephone mode into the process, then adding more additional questions, then adding a field mode in later 2022, whether that's just not not or an active field mode we're still working through the exact details, but that's what's happening over the course of this year. And of course throughout that but fundamentally aiming at an upgraded production system that we're able to produce these statistics robustly and quickly and accurately, and building with the new design that we have coming forward. And of course it doesn't end there this ongoing upgrade so it's not just that we're producing this thing now we're going to have to make sure that the systems are still working the questions are updated and upgraded and make sure that everything's functioning appropriately. And of course actively monitoring the progress and effectiveness of this overtime so looking at the response rates looking at the quality of the information coming forward. So we'll be dual running both the new design and the current LFS collection dual running those over the course of a good number of months that they're both live concurrently at the same times that we're able to compare the two sources and make sure that they're delivering what we expect them to deliver and continuing the engagement with the regulatory bodies so for the office for stats regulation so making sure that we're still complying with all the requirements all the needs all the badging necessities to make sure that we're collecting the right information in the right way that it can be still retain the badge if possible. And of course we have the intention to release the indicative results in late 2022. So hopefully later this year you should see some initial results from this newly transformed system so that we understand what what the label market looks like and that the systems are all functioning as we expect them to. And the target is that all of this so the online mode telephone mode field mode everything is all in place come autumn 2023 so all these improvements will hopefully be in train in action working by autumn next year. And what's happened with the LFS during this time you might ask well nothing radically is changing we're continuing production of the LFS data until at least mid 2023 you know all the production all the collection is still continuing as normal. No substantive changes to the content or design so all the variables in the LFS are all staying there we're not adding or subtracting anything in particular. No big changes to that the collection you know not to nudge process or the exact sample size might change a little bit as as Martina mentioned but that's for the purposes of making sure that we're collecting what we need to collect. Continuing to maintain all the materials so the user guidance and the data sets all being produced and published and updated as they normally are on a regular basis. And of course continuing that our customer contact services, if any of you have dealt with the Darts team for example, making sure that we're still maintaining the continued access through that process and archiving the current data and so making sure it's in the current system so that the ways of accessing the data in the data archive and other places that you're still able to retain the access for future analysis. Of course we're all taking this journey together it's it's nice talking about the transformation but you're a part of this as well so making sure that we're continuing to engage with with all of you all of our users and producers as well, continuing the engagement events including our labour market working groups. There'll be a feedback exercise in the spring regards that how how the transformation is going will be publishing blogs and updates and I mentioned the experimental results coming hopefully later this year. So you should be getting, I can't say regular but regular implies it'll be on a monthly basis. You'll be getting updates as and when we have information to disseminate. And of course engaging in conference events like this for example, but giving updates and developments as they come along when when there are conferences to attend and making sure that over the course of time you have sufficient guidance and materials and information so all the user guidance or the documentation snippets of code if we have them available, making sure that you have all the information you need about the transformation and about what's coming that you're able to take this on board. And remember we're further developing the 50 year legacy so we have a long history of good quality good good labour market statistics all there in the public domain for the for the public good and we're making sure that we're developing this improving this making sure that you have the information you need and trying to bring it all up to date with the modern technologies and modern descriptions and definitions. Just on the final slide here so you know making sure that you're part of the engagement. So if you have questions if you have issues if you have things things you want to know more about if you want to be kept in touch with through the process. The email address here social surveys at ons.gov.uk so do get in touch if there's something that you want to know or something that you want to follow up on and I'd welcome any any additional questions. So I see number five in the Q&A section so I'll stop there and head towards questions. Thank you very much.