 Welcome back to everyone and especially welcome to our second session which is dedicated to the data producer that is the Office for National Statistics talk and updates about on the one hand the existing labor force and annual cooperation survey so that will be our first presentation by Martina Helm in just a moment and then yes indeed an update on the preparations for the transformed labor force survey by James Harris. So in terms of how we are planning to organize this session so I think Martina and James will be talking between 15 and 20 minutes each we will have a space for questions immediately after the presentation and then if there are additional question we will or discussion we will have them after both presentations have individual so now to introduce the first presentation I think it's obvious to most of us that even if we are moving to a transformed labor force survey there with there is still going to be a labor force survey for at least a few months and there will be many users of the labor force survey for the foreseeable future so it's really necessary and useful to have an update on the latest developments for by the team working on this and this will be given by Martina I hope I'm pronouncing your surname when Helm who works at the social survey directorate of the Office for National Statistics and she's the survey manager for the LFS and APS so Martina the floor is yours sorry thank you very much Pierre and also welcome from me to this conference today it's really great to see it's so well attended there's obviously still a big interest in the state and widely used across users which is fantastic to see so I'm hoping I'll you know cover some interesting information for you in this presentation today which is meant to give you a bit of an update of guess what happened on the LFS APS since the last time we presented at this conference so before I start can I just check that you can see my screen yes we can see the wait a second yeah fantastic okay thank you so what I was planning to cover was giving a bit of an update of what happened in the data collection space since we last spoke to you what has been happening on the re-waiting and the discussions we're having in that space I wanted to bring up an issue we identified on the on the SOC coding and also wanted to talk a little bit about the transition to the transformed LFS although I'm not going to go into that much detail on that because obviously James is going cover more on that in his slot in a bit so going straight into the data collection at the last conference I spoke to you about the measures we've taken since the start of the pandemic to improve and maintain our response rates and so whilst I won't go into all that detail again I just wanted to give you a bit of a more of an up-to-date view on how this picture has evolved since then so on this slide here you can see the response rates since 2019 with the grey line at the top representing 2019 and the green line for 2020 shows obviously where response is dropped to so obviously half almost in March 2020 for the onset of the pandemic and the blue line represents 2021 and you can see there the increase in April when we introduced Nook to Natch where we basically brought our field interviewers out again and asked them to sort of knock on doors and try to encourage the public to participate again not doing face-to-face interviews at that point but basically trying to encourage them to participate in telephone interviews and that bedded in quite well as you can see but it started to decrease again and you can see that in the red line for 2022 so last year it was difficult to maintain the level of response as obviously across all our surveys we started to roll out or test face-to-face interviews in one way or another and it was difficult to maintain that level with all the different competing priorities and field strategies so at the last conference I spoke about the measures we had taken to improve response rate and one of the things we did was we tried to increase our achieved samples so our issued sample size to address the drop in response so here again the grey line representing 2019 achieved sample sizes and the green line what it looked like when the pandemic started so you can see the decrease in April straight away as response rates halved and we basically doubled our sample size in July that year and you can see the achieved sample going up straight away which was fantastic and we thereafter tweaked the sample size it was a slightly quarter on quarter to see you know to make sure we sort of keep keep close to what we achieved pre-pandemic but as you can see obviously the red line representing last year it was difficult to maintain that level with everything else that was going on and so I just wanted to mention obviously that whilst we have rolled out face-to-face on some of our other surveys in one way or another and promoting it as a sort of default option others trialling it to start with so we sort of had to feel our way through that and see what works once so we were allowed to do that again on the LFS we decided to not fully roll out face-to-face interviews so we decided to just operate basically where it is necessary for inclusivity and where we would otherwise lose the interview but we largely wanted to keep the mode constant with mainly telephone interviews as we are obviously dual running this survey now with the transformed LFS and we are trying to compare estimates and by changing things constantly it would make that a lot more difficult so trying to demonstrate this with this slide here where you can see that whilst the proportion of face-to-face interviews has slightly increased towards the second half of last year where we relaxed this rule a little bit the vast majority of our wave 1 interviews is still done over the phone and we continue this strategy until we transition to the transformed LFS so what is impacting our response so there's a lot of external factors like expectations of respondents having changed since the pandemic more people actually asking for online completion and this is what we're trying to cater for with the transformed LFS then expectations of interviews to some extent have also changed since the pandemic so it's somewhat more challenging to retain them or also to recruit new interviewers there's other external factors that are outside our control like just recently the royal male strike end of last year which meant that advance letters landed later and meant that it was much more difficult obviously then to engage the public in participating within that short window of data collection that we have and obviously capacities are strapped to the market for interviewer recruits has changed since the pandemic the logistics for interviews is more complex because of different approaches for returning to face-to-face interviewing and we have competing priorities also now with so James will mention that later about the transformed labor force so we're also introducing or has it has introduced a field mode so you know we obviously have to carefully manage the various data collection priorities that we've gotten yeah that's that's definitely you know showing an event so what to expect going forward so something that I just wanted to make users aware of that is when you are using APS datasets for lower level analysis then you'll probably find that the sample overall has been decreasing because whilst we adjusted an increased main sample since the pandemic to address the drop in response rates the both sample has not changed and therefore suffered more from the drop in response the ONS is currently reviewing its priorities for face-to-face data collection to see how we can manage our field force also more efficiently and effectively and we obviously also put measures in place to improve our recruitment and onboarding of interviewers and training them you know to make sure we use our resource more effectively and and on our field data collection priorities then moving on to reweighting so again there's a lot of on the methodology side I think that we covered in the in last year's conference so I don't want to go into all that detail again but I can provide some links in the chat later to articles be published in that space so just sort of to summarise I guess where we are with the reweighting prior to the pandemic our data was reweighted every two years using the latest population projections obviously since the pandemic the mode change from face-to-face to telephone interviews for wave one has introduced some some bias to the data we've got we found that we had more own occupiers for example then renters and that participated they split between UK born and non- UK born population and it looked odd so we had to tweak our waiting process to account for that so we have since 2020 reweighted annually adjusting the calibration with tenure in 2020 so that was our first reweighting exercise and then in 2021 and 2022 we used population growth rates that were based on the RTI data, the income tax data from the HMRC which has its limitations but that was basically the best available data for us to adjust our population growth rates and the latest round of reweighting is almost complete so we started that probably second half of last year we just released the five quarter longitudinal data sets again with the revised weights in mid-January and in mid-March we're going to release the reweighted APS wave one weight as well and so that reweighting exercise is then completed. What's going to happen going forward is that so we are considering obviously what we're doing next there's new RTI data expected to be released in spring and we're having discussions about potential further RTI based reweighting exercise that might happen post transition to the transformed LFS. We're also considering obviously whether and when we would do a census based reweighting exercise once population projections for England Wales and Scotland are available but this is not expected before 2024 so somewhat after the plan transitioned to the transformed LFS which means that any further reweighting plans really need to be considered alongside the transformed LFS and its new estimation methodologies. We can't look at it in isolation for the current LFS we need to look at that longer term picture really from a user perspective as well. So that's the news on the reweighting and then I wanted to draw your attention to an issue as I said earlier that we identified in the collection of our occupational data. We published two announcements in July and August last year to make users aware that we had identified an error with the collection of occupational data not just affecting the LFS it's across our social surveys and at that point we knew what the problem was but obviously didn't know quite enough about the extent of it or the impact so we did a lot more work on that and then we published a paper in September last year which you've got a link for in the slides while which will be sent around after or will be published on the UKDS website I believe and in that article we basically presented approaches that we took to understand the impact and also the level of impact and the timeline we estimate that it takes us to fix this problem. So just a sort of a brief overview of the work we did to assess the impact so we reviewed the SOC coding index that interviewers use basically to code occupational data as part of the interview we compared as of time series of data at a very low level to understand what happened with the switch from 2010 to 2020 SOC codes we compared the data also or the LFS data also with the sensors although there is a different coding methodology because sensors use as an automated coding approach whereas on the LFS it's interview let coding and we had then as often initially exercised done of using automated as well as clerical recoding and try to assess the outcome of that as well and then we basically published this paper in September where we included a list of all the 412 four-digit unit groups and the estimated impact for this group so this table here is a summary of the impact showing that about half of the codes are expected to be impacted to high level by this error and then we mentioned the paper was though about a timeline of when we expect sorry before I go to that so this is the impact at four digit level and at one digit level the estimated net change is expected to be fairly small as you can see here the group that we would estimate to be impacted at the largest would be group two which is professional occupations for around 0.3 percentage point difference so the work the recording of SOC codes since so for SOC codes from January 2021 onwards is well underway and we estimated that the work would take us about six months and we are still on track with that obviously once all the SOC codes have been recorded then all the relevant derived variables will have to be revised and micro data sets will be re-released as well as the aggregated tables that we published on our website will also be corrected accordingly so just to be mindful of to apply caution when using the SOC occupational data from January 2021 onwards at the moment in particular if you're planning to use SOC codes that we highlighted as having a high impact or being highly impacted by this SOC coding error and with that I'm on to my last topic and that is the transition to the transformed LFS so I'm not going to go into detail about the transformed LFS so I leave that to James who will tell you a lot more about that in a minute but when we transition to the new LFS the aim is to decommission the current LFS APS and that is all waves at once with the end of June this year however before it comes to that we expect a decision to be made by the office and that is expected by March April time this year once we get to that point of decommissioning we obviously then inform all our respondents that are still part of the panel that they won't be contacted for any further waves so it's not that they would then move to the transformed LFS that's a separate sample and from a user perspective I thought it would be useful for you to be aware that obviously we will process and produce all relevant datasets covering the periods up until April to June 2023 but that will mean that certain datasets cannot be produced as we're not doing a full year in 2023 so there won't be an APS household dataset January to December 2023 for example or an APS two year longitudinal dataset for 2022 to 2023 or an APS three year pooled dataset including 2023. Just off high level timeline in terms of what this year would look like now or will look like now from our perspective so obviously all the you know a data collection and outputs production goes ahead as normal until mid this year depending the decision in March April we're obviously supporting the transformation and preparing for the transition alongside that and then we'll transition to using the transformed LFS in all our processes and outputs production until mid this year we should have the SOC revision concluded and obviously the reweighting as I mentioned earlier should conclude too much earlier we hope to have this all done by mid of March although we're obviously thinking about potentially a further reweighting exercise on then historical LFS data potentially later this year so to prepare for the transition uses our advice to read up on the already published user guides which we published on our website in November last year which includes sort of a high level background user guide variable mapping document and dummy dataset James will say a little bit more about that later we obviously also publish more in the user guide space over the course of the next few months around derived variables and also more about the datasets etc and there are also plans for you know block blocks and articles around comparing the estimates from the two sources over the course of the transition and that brings me to the end of my presentation so I hope I didn't take too long so perhaps we have a few minutes for questions before we can hand over to James I am now going to give the floor to James Harris so James Harris he's actually found some property leading the team behind the transform labor force survey so he has been working for a few years is in population statistics on estimates and projections at DNS before moving on to to the transformed LFS and he's indeed going to update us about the work being done and what to expect with the transformed LFS so James I'm giving you the the floor now fantastic thank you hopefully everybody can see me good to see you all I'm holding transformation I'm leading kind of the transition from the current survey to the new survey other people are actually creating the stuff James we the bandwidth doesn't seem to be very good so maybe you could switch off your video okay so so it's good to see everybody here and it's good to have direct interaction with all the people are going to be impacted by what we're doing here so the transformed LFS and what it means for all of you and essentially how things have progressed over the last year and over the year as well so as was mentioned a little earlier we're celebrating 50 years of the LFS and this is the 50th year since we started collecting these statistics which cover the labor market and demographics and all sorts of factors and features about people's everyday lives and the survey is continuing to evolve through this transformation so bringing some of the old kind of paper approach and things down on in actual hard paperwork and now moving towards the online first collection processes supported by telephone and and other features behind it as so bringing us actually into the digital age and it's in our 50th year that we're going through this so a very proud time for all of us so the last time I was here this was the truck timeline for the transformation so we were looking at ahead at the time to February and April 2022 and of course now things have moved on a little so if I click to the next slide so this is where we are currently you may be able to see that one or two things have moved a little bit to the right but fundamentally the timeline is essentially still the same so starting in February last year we did successfully introduce telephone mode into the survey and following telephone mode in November we also added the final mode the face to face mode as it were but at the moment that's only knocking to nudge so that is all three modes of the survey live active in in collection right now and in in between those times we also had an increase in the sample size and the addition of more content so in theory all the labor market content has been included since September last year so huge amounts of work and developments going on over the course of the last year leading us to where we are now during that time we have also been obviously running the dual run so both surveys have been active at the same time and sharing and comparing the differences between the two sets of results and the responses that we've been getting to them and over that time we've also had a user engagement exercise in March to June last year and some more targeted business change engagements with departments divisions and people since then about the impact and results this change in this survey is likely to have upon them and heading towards what we're going to be doing over the coming year but I'll come onto that in a second so what we're looking at in the months ahead of us and I stress the word months it's not years anymore heading towards around about March April time as Martina mentioned that the transformed LFS will hopefully meet the quality criteria to become the primary source for labor market statistics assuming that that is met and that those decisions are made heading towards then a little bit of further content being added to the transformed survey and the decommissioning of the existing survey so at the end of June we would theoretically decommission the current survey and then the new survey is the de facto source of data from that point onwards leading towards the kind of pink section which is from that point onwards it is all the TLFS it's all the new survey source from that point it's the first publication of data using that new source being in September and then from that point onwards all the other surveys and sources and publications eventually coming on board with the new survey as the data starts rolling out on a regular basis so what we've been doing so as I say we've increased the sample size and they are now at the target levels or at least for the time being and we've included all the key labor market content but there's an awful lot of further development of additional content so there are other questions response categories weeks and changes looking at topics like sexual orientation agenda identity trying to squeeze in as many of these key fundamental issues that people are interested in that we want to know about the the population and the workforce specifically so an awful lot of that content will hopefully be added in before decommissioning in June we're working through the testing of that right now so the questions have been developed but we're working through the actual testing and implementation of them as I speak so we've now as I say added the final mode of data collection that's the quote on quote field mode although it is only not to knowledge at the moment we will eventually move to actual face-to-face interviews if it's necessary in future but that's unlikely to happen this particular year and we're monitoring and adjusting based upon the current performance so looking at the response rates that we're getting looking at the results looking at the way people are answering the new questions in the new fashion making sure that people are understanding the questions that we're getting a sufficient response all across the country in the different categories of people in the different strata and breakdowns and in similar fashion looking at the results of the the survey so whether or not it's producing similar labor market outcomes to what we would expect or what the current LFS produces or ideally both so investigating what the results actually look like still very early days we're just about getting enough data with enough of the survey design implemented and everything through but the investigation has started internally and over the course of the next few months we'll then spread that out to other surveys I'm sorry other sources other breakdowns other publications making sure that it there is a consistent story with the the current and the transformed survey and hopefully that the bias or indeed the bias is reduced or the accuracy is increased as a result of this new survey and then some further methodological development has been happening not much has been published about that yet but we're hoping in the next couple of months that we'll be able to share a bit more information a bit more detail about what methodological changes and improvements have been made to the survey and indeed perhaps some of the changes we're expecting over the next 12 to 18 months if more work is needed beyond just that June decommissioning date so an awful lot has been happening very grateful to all the people involved various different divisions of ONS is an awful lot of people involved in delivering this so where we are with the response rate at the moment focusing on wave one only there's not enough of the TLFS to do much more but looking at wave one all the modes combined so it's changed over time we had online first and then telephone and then face to face but wherever possible whichever modes existed at the time you can see that as Martina mentioned earlier the current LFS the lighter blue bars have slowly been dropping over time which is somewhat unfortunate but the priorities to get the TLFS online and working and that has been very slowly increasing over time the aggregated response is about 41% at the latest count the wave one here it reports is around about 44% but on the whole with everything added together it's around about 41% at the moment so a good response and where it should be and importantly it is improving especially with the results of the knocking to nudge so if I move on to that looking at the response rates by the indices of multiple deprivation so the most deprived households on the left hand side and the least deprived households on the right hand side you can see the difference in between them so the most deprived households are far less likely to respond to the survey the least deprived households are the much more likely to respond to the survey and as a result of introducing the field modes of knocking people's houses encouraging them to respond to the survey reaching out to especially the more deprived areas and certain demographics of people that has now improved the differential so the least deprived has improved a little bit and some of those categories eight nine and ten have improved a little bit but more importantly the lower end the most deprived households have improved much more significantly bringing it bringing it much more in line a more common approach all across the country so hopefully improved levels of bias all across the board of course still very much a work in progress and I'll describe more about an improvement that's happening there in a moment so Martina mentioned the tlfs user guidance so I'll just briefly explain what's in there and what's been going on so we published the first version of user guidance material to provide everybody with information about the the transformed survey design the methodology and the content the situation as at october 2022 so this was published on the 14th of november that was the current state of play as is at the moment and it included three items uh I'll skip that and move straight to the first one here on the right hand side that was an introductory page to our current volume one but that's now changed very significantly that's the current version of our background user guide for the tlfs so this contains all the key background methodological information about the survey design and how to use it or at least the skeleton thereof will be adding much more information to it over the coming few months as more gets developed and written and approved and we're able to release more information uh so uh this is a much simplified version of the former lfs documentation so trying to focus on exactly what it is you need to know the key details the key aspects uh you know we do have a 50 year history but there's an awful lot of bloat you don't need to know necessarily what happened in the 1990s you have all the historic information in other documents if you need them this is focusing on the transformed survey and what the current and future state is going to look like hopefully making it much easier for you to digest and understand and a lot shorter to actually get through as well um but still working on going on on this this collection of information as I say around about april maytime hopefully an updated version of this will be coming out with more information especially in terms of the content speaking of content we also put out a dummy data set so this is what this the current data set looks like so you can see the design the shape the variable definitions what you could call the architecture of the file what you can expect the micro data to look like so you can see the the names of the current variables the the variable names the uh type of variable or in some cases the uh an example response for that variable so for example the case id is listed as a numeric variable seven characters long whereas the the month of the survey survey month is listed as number three because that was an actual number number three it's a mixture of those things so you can see what the data looks like and start preparing thinking how how the names of the variables may have changed and how that might affect you your systems your analysis at the moment it doesn't contain any actual data it's just that one example response so that you're able to test with your systems but over the next few months we'll be sharing the actual data sets so indicative results in in summarized form and uh micro data to actually work with so watch this space we're not quite there yet but we will be there in the coming few weeks and months also the third item was the mapping file so an excel file the transformed TLFS to LFS mapping document it was called which contains all the regular variables contained in the design as at october so it doesn't contain the derived variables so things like ilodeffer i-l-o-d-e-f-r is not in there just yet this is the regular variables so to speak where it's just a question on the survey that matched straight through to the data sets but it gives you an idea of what the current variable is and was called and was asking and then the new variable what it is and is asking and the response categories thereof so picked out a particular example here of whether or not people are looking for work so what was the main reason uh you did not look for work in this particular period and you can see the response values of the current survey on the right hand side the response values of the new survey on the left hand side largely the same things like retirement and studying are still there maybe in a different place but they are still valid response categories but we've added a couple of extra bits and pieces in there for example looking at child care and caring responsibilities so new key variables that people were interested in valid response categories that should give you a little bit more information than you've previously had so there will be changes like this over the place some very minor that you probably wouldn't even notice some a little bit more major and you can see the the blue bar in the middle gives you an idea of what whether it matches whether it's exactly the same because a lot of variables like age will largely not change or some of them like this whether it has been a change trying to describe what that change looks like and you can see that in the little blue bar in the middle of this document so hopefully a good amount of information in there but again we'll be updating these as we go through and then touching a little bit on the ongoing development so one particular thing that we're working on right now is adapting the survey design so very grateful to my colleague Maria Tortoriello for a bit of the information here the current data collection strategy the same fundamentally for all addresses we're sampling using a systematic random sample from a particular file and then for five quarters we go out to each address with an invitation to take part in the survey and if they haven't yet responded we'll encourage them either by telephone or by knocking the front door and saying please respond to the survey you could do so by telephone or by other means and we carry that out through 13 geographically representative cohorts run on a weekly basis so it's geographically representative largely in terms of region but obviously we do try and make adjustments at the local level as well so the response rate indicates what proportion of people are invited to take part and there's unfortunately an unequal distribution of responses what's called the differential non-response bias so for example during the pandemic and in the early stages it was online collection only there was a bias more towards the white male home owned men aged around 45 so that they were more likely to be responding to the survey than other demographics but of course we for the purposes of accuracy for the purposes of reducing bias making sure that we have good levels of confidence for your estimates and analysis trying to make sure that we capture all the other categories that we possibly can and trying to improve our survey design to reach out to all the other demographics of people and improve the response rates all across the board the statistical processing the methodological processes can enable waiting of the sample to account for some of this bias but fundamentally confidence in the estimates only improves if we improve the source data itself if we go out and ask the people themselves and get their actual responses so we have three key quality targets for the survey I won't tell them to deal but reducing the bias choosing the attrition and improving the response rate and really speaking so making sure we capture everybody all over the country making sure they stay in the survey for as many waves as possible and trying to get as many people as possible to actually respond to the survey so fundamentally trying to improve the quality of this survey how are we adapting the survey to meet those that's the key question i'm touching on well implementing what's called the adaptive survey design so dividing the whole sample into smaller groups and similar characteristics what called segmentation one size does not fit all so trying to apply different approaches to all these different categories of people and adapting the survey design for all those different groups so some of them it made more suitable by telephone some online some place to face it may mean that they need special printed materials for them or they're more likely to respond if you contact them in the morning or evening or at night or whenever it might be so different ways of approaching these people it may be that a different incentive would appeal to them more so some of them will respond to a financial voucher some may be a tote bag some of them may prefer printed materials like a notebook and pen trying different incentive approaches to make sure that we're capturing all the right people as best we can and of course changing the way which we follow up with people so it may be that they're not responding to the telephone they're more likely to respond online or they're more likely to respond with a knock on the door it may be that certain geographies are responding in a different way that you're more likely to get a response by knocking on the door in Cumbria than you are in London for example who knows looking at all the different aspects of how people are responding why they're responding or indeed not responding and adapting our desire to meet those changing needs so the objective here of course is to improve the survey outcomes but it is working within certain budgetary constraints obviously I would say an incentive of 50 pounds is far more likely to succeed than an incentive of five pounds but there are certain limitations on what we can do but working within those bounds doing what we can to improve the responses so I've listed here four key features to the adapt adaptivity of this design so dividing the sample into eight different strata so split by age around about the age of 45 so lower ages and higher ages segmenting by urban versus rural areas and segmenting by the indices of multiple deprivation so the more deprived versus the less deprived peoples and you end up with eight different strata and that was built using a logistic regression model using existing information that we already have so figuring out who is and is less who is more or less likely to respond to the survey using what we already know and adapting accordingly we've already talked about utilizing knock and nudge but also utilizing new indicators so representivity indicators looking at the quality and cost and the contrast between different respondents and non-respondents so building better monitoring processes behind it and of course a little bit of trial and error thrown in here as well so testing what works and what doesn't work and building on on what we're testing as we go through so if something works we then build upon that and take the next step and test new things built upon that so an awful lot of this improvement over time and of course this is the transformed survey so it's live but it's still you know it's in the field it's still happening it's still being collected every week month quarter year this will be developing over the course of time so development is still underway in particular for the adaptive design so this started rolling out as of November so when the the knock and nudge processes became live and we were undertaking continuous evaluation of what's actually going on how successful it is we're monitoring the effectiveness of the design with the management information dashboard which is almost collected in real time I know it's not minute by minute the hour by hour I would argue it updates and it's showing which parts of the country and which types of people are responding and we're able to adjust accordingly so sending out knock to nudge processes to different areas and adapting where the field force is actually targeting and we're continuing to explore auxiliary data sources to figure out what how what will work to encourage respondents to actually reply but that was very much on the adaptive design there's an awful lot more going on over the coming months there was a set already continuing and conducting analysis and investigation of what we've collected and if we are finding problems issues and errors then making some tweaks and adjustments to fix any problems that's not just in the data itself so the survey design or the content or the questionnaire but also operational issues where maybe the telephone operators or the interviewers have identified some issue or problem so making sure that we're working through this even things like the the royal male strikes that Martina mentioned having to factor in how best to respond to those kinds of problems monitoring and making adjustments to the quality still an awful lot of work going on with the methodological development things like the estimation approach the wave structures the exact content that's being added in there the approach to waving everything else all of those things are still under review and still being developed you should deal with developments over the next two three months and of course further delivery of questions so still going up to interested parties making sure that we have captured the things that they're interested in their requirements of what questions should be in the survey what content they'd like to see what policies and interventions which might work with so as they celebrating here the 50 years of the LFS you should expect a bunch of publications webinars over the next few months covering all the various aspects of the design the indicative results and updates on progress and plans throughout 2023 so a link there to one of our progress updates which we have now published on a quarterly basis we've we're further developing the user guidance and the quality and additional delivery of content as well so an awful lot of activity going on and of course we're trying to continue the legacy that Richard mentioned the vital source of survey data for all of your needs and we're trying to meet it in the best way we possibly can trying to continue this legacy not just for the current 50 years but for the next 50 years as well and if you need more information obviously you can contact me or Martina but you can also get in touch through the labour market transformation inbox link there if you have problems issues questions you want more information about methodology you want to be invited to any webinars or seminars or maybe you've been dropped off a newsletter list or whatever it is please get in touch and we'll be happy to engage and support you through this traditional period and that puts me to the end beer I'm almost on time close enough I hope enough for a couple of Q&A