 Right, then. So this is an overview of what we're going to be talking about today. So I'll start off with a brief bit of information about the UK data service and who we are, and then talk a bit about survey data and then documentation and tips. Then I'm going to do a tour of UK data service website. And if you do have any questions, do put them in at any point in the Q&A box and we will answer them later. Okay, so who are we? Well, we are a resource funded by the Economic and Social Research Council. And via our website, we are a single point of access to a wide range of social sciences data that people can use. And in addition to this, we provide support, training and guidance about using our data. So what data do we hold? Well, this session is about UK surveys, but we do have other kinds of data. So you can see we have longitudinal data, which is data that is similar in fact to the UK surveys, but with the longitudinal data, you every time the survey is run, they look at the same, rather they go back to the same people each time. We have international databases, UK census data, which has been collected every 10 years, and a small collection of business and qualitative data. So UK surveys, they tend to be data about individuals or households, and they are often commissioned by government departments and conducted by large organisations such as the Office for National Statistics or the National Centre for Social Research, otherwise known as NATSEN. And each of these data sets includes key characteristics such as, well for individuals, things like gender, level of education, etc. And they also have large sample sizes. So that means they tend to have sample sizes of around 3,000 plus. Some of them have up to 40,000 that they interview, so quite large. These are nationally representative, which means that you can use the sample to create estimates for the whole population. And the ones that I'm talking about here are the repeated cross-sectional surveys. They tend to have the same or very similar questionnaire with a new sample of people every time it's run. And some are run annually, some are run more often than that, and others of them are run less often than that, but they are often repeated regularly. Okay, so what do we get from survey data? We get survey microdata. Microdata looks like this. So if you look at the top half of the screen, the data are displayed in rows and columns, and each row represents the information from a single person in this instance, and each column represents the actual information, so to speak. So the first column is QR, sorry, Q Health R, and that's talking about whether or not a person has good health. R sex would be gender, usually age, probably an age group, etc. Okay, and if you look at the bottom half of the screen, you can see, for example, that the first person there says they have good health and they are female and they're age 25 to 44, etc. Okay, so how do you get the data? Well, generally speaking, you would download the data from the website, and that comes in whole datasets, and is included in it is the documentation and the documentation you can also see on the website anyway. And it tends to come in particular formats. So those designed for SPSS data, and it often also comes as tab format. An alternative is to access it via Nestar, which I will talk about later. I will show you when we do the website tour. And what this does is it allows you to browse the variables and also to see the metadata, which is the information about the data. And you can also with Nestar do simple data analyses and export tables and graphs and download subsets of the data. Okay, so key topics are shown here, and you can see beneath each topic there are some key datasets that cover those topic areas. These are not the only topic areas that we have. We have a lot of other, basically any topic that you can think of that is the social sciences topic we in all likelihood have some data about. So, for example, the employment and work, we have the labour force survey, which I'll talk about briefly later, and the annual population survey, which despite having a completely different name, is in fact based on the labour force survey plus extra surveys that they run. And then there are also international cross-national surveys, such as the European Working Conditions Survey. Okay, so now you should see on your screen a poll in a moment. So which of these variables do you think we hold? Do we have total daily expenditure for two weeks, severe reports, blood samples, number of sexual partners, and COVID-19 test results? You can choose more than one. So we'll just wait for people to advance to the poll. Okay, shall we end the poll now, Jill, and share the results? Okay then, so, yes, we do have total daily expenditure for two weeks. That's the one that most people have chosen. I see not many people think that there are blood samples and some of you have selected the other three. And of course you can imagine, I'm going to say this was a trick question, because the answer is that all of these do appear on some of our surveys. Okay, so the Living Costs and Food Surveys includes a food diary where people keep a note of everything that they buy over that two week period. There's the English Housing Survey, which does include a survey's report. Health Surveys for England includes a nurse visit to a sub-sample of participants where they take blood and saliva and height and weight measures. The National Survey for Sexual Attitudes and Lifestyle asks detailed questions about sexual experiences, behaviors and views. And Understanding Society also asks the COVID questions in the COVID waves and these include test results, vaccination, infection, etc. Okay, so that's a huge variety of different kinds of data that's also collected in UK Surveys. So a couple of examples of some commonly used surveys. The British Social Attitude Survey looks at public attitudes across Britain and it's good for looking at patterns of continuity and change over time because it is an annual survey that has been run every year, apart from two since 1983. The sample is approximately 3,000 people every year and it's made up of core questions which include background and classificatory questions and these questions are asked in all or most years and in addition it has a series of modules on various social, economic, political and moral issues and some of these, as again, are asked very regularly and some of them are asked intermittently. Okay, so we have another poll on gender equality. So the question is what percentage of people believed in traditional gender roles in 2017? So as you can see on the screen, 48% people believed that essentially women should stay at home and look after the children and the men should go out and work and earn the money in 1987. So as you can see from the little graph sort of pictogram thing that the numbers have clearly gone down. What proportion were they gone down to by 2017, which was I think the last time that this question was asked? Okay, so 28% is pretty popular. Some are saying 38, few 18 and just a few of you selecting 8%. Okay, shall we end the poll, Jill, and look at the results? Yeah, so they stayed pretty much as I said. So most people think it's about 28% and again 8%. So essentially those attitudes have kind of fallen off a cliff since it was asked first in 1987. Okay, so Labour Foster. So this is another very commonly used survey and it's reduced by the Office of National Statistics and it is the main source of data about the labour market in the UK and it measures employment, unemployment, economic activity and a lot of other related topics such as the kind of occupation you have, training, hours of work and a variety of personal characteristic of yourself and household members aged 16 years and above. And this is a household survey. The quarterly labour force survey began in 1992 and approximately 60,000 people are interviewed a quarter. However, it did exist prior to that. It started in 1971 as an annual survey and you can still get the earlier data. But obviously you want to take with a slight pinch of salt the trends over time a little bit because the survey was done in a slightly different way given that it was an annual survey then. So the data sets since 1992 are quarterly which you can get at the individual or the household level. And there are also longitudinal data sets which link five quarters up to five quarters or two quarters because the way it's arranged is that basically a household will join and stay in for five quarters before dropping out and being replaced. And in every quarter roughly a fifth of the respondents are new households. So that's how that works. And if you want to look at the chart over on the right-hand side you can see that it is very useful for looking at how trends over time. This in fact is from the Office of National Statistics website and it does start in 1971 and it shows the total actual weekly hours worked in millions amongst UK population. You can see the big crashing numbers over COVID. Okay, and looking at another survey the Health Survey for England. This covers general health, long-standing illness and smoking and alcohol use. And there is a core questionnaire plus a focus topic for the particular year as it's run annually. And it has been run annually since 1991 and there are currently about 13,000 interviews a year. And as well as the questionnaire there are also physical measurements and the analysis of blood sample as we've already seen. And there are reports that you can find on the Health Survey website and you can see over on the right-hand side some information that's derived from the 2018 Health Survey for England. Okay, so a little bit about survey documentation. So obviously when you create your own survey you would know exactly what you did and how you did it but when you're using a survey that has been created by someone else it's very important to look at the documentation because that will tell you what they did and how they created the data and you can't really understand and use the data without having that information. Okay, so what we've got here is an example from the understanding society COVID study from 2020 to 21 but you'll see if you go to the website that pretty much all documentation sort of looks something like this although obviously they do vary from study to study. The key documents that you will usually find is a user guide or some similar document that summarises the design the sampling that was used the derived variables how they were created. So derived variables are the ones that are created from the variables that were based on the raw data from the survey. How to use survey weights which I'll talk about briefly in a moment how the survey was funded how to link to the main survey where relevant and details of special access data sets again where relevant and there should always be a questionnaire for every part of the survey. There will be a data dictionary which is created by the UK data service as part of the ingest process and what that does is it lists which variables are in the data set and gives some extra information about how they're coded for example and also there's participant information. Okay, so understanding society questionnaire here's an example of one so it tends to show the questions that were asked the variable name and the routing information which means the conditions on whether the question is asked because obviously not all questions are going to be relevant to all respondents and sometimes surveys deliberately ask only certain sections of respondents certain questions so that they can cover more questions but of course that leads to slightly smaller sample sizes for those questions so that's something else to watch out for. Okay, there is a resources tab if you look over on the right-hand side of this the resources tab provides links to things like the case studies that analyse the data publications doesn't include all possible publications it's usually just a small number details of related studies and data sets and other relevant information. Okay, and just a brief note about survey weights and that is that most surveys do require you to use survey weights to make the sample data better represent the population and those surveys that need you to use survey weights will provide the weighting variable or variables in the data set and the details of those variables how they were created and how you're supposed to use them are generally included in the survey documentation so you should be able to find that quite easily and in case you're new to this survey weights are quite easy to use if you use the statistic package like SPSS data or R and if you'd like more information about what weighting is we have various things we have a what is weighting guide short introduction to using weight in social surveys video and various guides to commonly used software which also includes how to use weighting okay so how do you get the data what are the conditions under which you can get it so there are three levels of access there's open access which has very few restrictions and in terms of survey data there are only a small number of teaching data sets available from the UK data service which are open access data and teaching data sets wants to very small cut down versions of the larger data sets and they're generally slightly altered in terms of for example grouping some of the categories together so that they can be shared as open access and they tend to only have maybe 10 or 20 variables so those really only to be used for teaching the majority of the data that you would want to use for research or work or for your studies would be the safeguarded data and to get that data most of it is under the end user licence and you just need to register with the data service and then agree to the conditions and then you can download end user licence data straight from the website there may occasionally be additional conditions and there is a much smaller collection of safeguarded data which is under the special licence and that does require a few extra hoops to jump through but the vast majority is under the end user licence and you can just download that from the website after registration okay and the last section is access condition is the controlled data so to get controlled data you would need to make an application and have it checked and then you would access it through a secure access agreement and usually through a physical or virtual secure environment so ours is called secure lab and the difference between controlled and safeguarded data is essentially that there tends to be more detail in controlled data sets so these might be variables that are on sensitive topics for example information about people who've left care for example I think you would not really like to find that information under the end user licence but it might be available under controlled in secure lab another key difference is that lower levels of geography tend not to be available under the end user licence and maybe not the special licence either so for example generally speaking for end user licence data the lowest level of geography you can get is the regional level so north west of England London is a region Wales, Scotland south west etc and you can get down potentially to postcode level if you apply for the controlled access version of data if one such exists okay right then so okay now let's look at the website okay so I will go there right okay so so the website is just ukdataservice.ac.uk and it's set up so that you have the main things you'd want to do which is to find data potentially deposit data or learn about using the data or come to our training and events over on the left hand side and on the right hand side there's information about the data service, latest news impact which essentially is how people have used the data our help pages and contact and you can also search the website and log in here at the top right okay so let's look at finding data well you could type in just on the front page or let's go to find data okay so right the options here are as I say either you can search by clicking into the box and searching for something or you can browse to the data so let's have a look at the browsing so you can browse by theme so each of these are common themes that people might want to search for again this does not cover every single possible topic that might be of interest to you but if you are new to using data you might be interested to look at these note that these are not all survey data that's included this includes a range of different kinds of data okay if we go back we can see that you can browse by data type and UK surveys or cross national surveys might be the ones that you're interested in if you're interested in these kinds of data and there's also links to teaching data sets and more teaching data sets or other ways of browsing okay then so let's let's in fact do a search let's search for crime data this will take me through to the catalogue the data catalogue search results page so you can see at the top here is the search bar and you've got search tips here if you want some more information about how to do search is the kinds of things that you can and can't do and on the left hand side here and on the right hand side we have results which you can show by various numbers per page or by relevance or some other way of sorting and note here you have studies the first tab the default tab tells you studies that come up say for my search for crime and that's what we have here so each of these is an individual study that has come up so in terms of surveys particularly series you might be more interested to look at the other tab which is the series tab up here on the towards the top left but the second tab and that will take you immediately to the the survey series pages okay so let us go to a catalogue page so crime survey for England and Wales let's choose that one and this is the series page and in the series page you have frequently asked questions if you're completely new to the survey and some resources again if you're completely new to the survey right okay not okay and that shows you the other surveys that are related for example I'm most interested to access the data so once you're in the series page you can see there's two kinds of two versions you can get this which is the end user license version so the kind that you can download after registration okay and then you have the secure access so you can usually spot end user license data by the fact that it doesn't say what kind of access it is normally the other kinds of access if it's open to your special license it says so okay so let's look at the end user license version okay so let's choose the most recent one so obviously over COVID they moved from the usual face-to-face crime survey for England and Wales to the telephone operated crime survey for England and Wales so let's click on to that one okay and this takes me through to the catalog page okay so we're on the details tab by default that's the first tab and this has a lot of information which I think is worth reading this particularly if you're new to a particular survey so it tells you the title the study number which is the study number that has been provided by the UK data service and it says how the data can be accessed information on the access page as well okay and it tells you about the series and who created it there's a bit of information here about who collected the data and sponsored the collection and the citation and copyright information there's a bit of information about topics and you could use the Cessarist search on keywords I think the most interesting thing often is the abstract because that tells you everything that you might not know about survey when you're new to it about the background and why it was created for example and some related surveys so if you were to read all of this it would explain why the telephone operated crime survey for England came about and so on and a little bit about the history it talks a bit about the secure access version of the data and some new methodology for capping the number of incidents crime incidents and so on and then the main topics bit of information here coverage and methodology is below so this is a nice way of getting some kind of overview of the survey without using all the documentation and then at the bottom we have the addition history okay if we scroll back up to the top we can see then the very important documentation tab so here we have a user guide and then there are two technical reports and there's the citation the data dictionary is created by the UK data service and a read me file so if we look at the user guide okay so you can see that's been created by the data producer okay and it basically gives a lot of background information talks about the methodology, sample design etc it talks about the data sets that are available variable names etc it gives you some information about analysis and some methodological limitations and so on okay so you can search that using control F to go to the relevant bit that interests you okay this particular data set also has a couple of technical reports which I think covers a lot of the same information but probably in a bit more detail so this has been the data where I think collected by Kantar and so they have also produced technical reports and again this will include probably the same information but possibly in more detail okay and let's just keep going data dictionaries so what this is doing is it's creating a zip file and I can open that and if I double click on it that's a word document which is taking a moment to start up and there we go okay right then and so this is what a typical data dictionary looks like so there's information about the number of variables and number of cases and then variable level information so the first variable second third tells you what the variable name is and tells you what that means and it also shows the coding for that variable okay so let's go back to the website and the final thing to mention is the read me file which just contains bit of information about how the the data set was ingested most of the time I think this may not be a particular relevance to you but occasionally when a variable is unexpectedly missing for example but for good reason that might appear here in the read me file if we've got a lot of queries about that so it's kind of worth always checking I think the read me file just to see if there's anything interesting in there if you're using the data okay we have the resources tab so here are a few case studies so those are examples publications reports there are a lot more that use these data than just these two other studies so you can find related studies and any other information okay so these are useful websites related to the data and then accessing the data so we click on access and you will see the access conditions here okay and you can't add to the account without logging in so I'm going to log in and what you do is you are taken to a login page and you can type the name of your organisation if you are at a UK college or university or if you've been here before they might remember you based at the University of Manchester so it recalls that I'm from there if you are not based at a UK college or university that's fine just click on my organisation is not listed and there will be more information about how you can register which essentially means requesting a username which does take up to five working days to process but once you've got that then you can just sign on and you can okay so University of Manchester so I will log in a little more login on password okay and it's thinking about it and there we are so what it's done is it's taken me back to the same page that I was on before where I can access the data but you notice now at the top here on the top right it now auto log out okay so let's go down and add these data to my account so click on add to account okay great so now I can go to my account or I could continue to browse but let's go to my account and download the data so this is my account my account probably looks a tiny bit busier than many accounts because I work for the data service so I have a lot of projects and former projects and data and so on but essentially when you have added a data set to your account you will be able to find it here you should be able to not finding it let's go for a different one let's go for that one here and what this is doing is asking you to select the data sets that you're interested in and I know I know where you can find it it's the telephone operated one isn't it yes there we are okay let's choose the correct one and then add to project okay and you can add it to an existing project if you have one or you can create a new project you can also show how to create a new project so you need to choose this beautiful title you would select the type now for research purposes or student work or most purposes in fact a non-commercial and so you could select that if you're using it for teaching then obviously select teaching if you are going to use it for commercial purposes so essentially to make money then you would register this as a commercial project okay and then you would add the abstract which has to be a minimum of 100 characters and then you would create the project so it's that simple however let's go back to the data let's see sorry if I go back to data and add that to a project that I already have okay so I've got a project that already exists okay and then I will add it to that project and my internet is working slowly because I'm doing a live demo clearly it's not normally slow like this thinking about it right okay it doesn't seem to like that okay so let's find that's probably because I've got so many data sets this won't be as bad as this when you do this yourself okay here we are so our actions that I could do here I could download the data or I can go back to the catalogue page or I can just click on the little button on the right hand side and then download the selected so it's very simple and I can select which version I want okay so I can have the SPSS versions data or the tab version and I just click on say that one download selected and you can see on the top right it's creating opening it that's it it's a zip file and then I double click on that and I have a zip file and I have documentation in this folder and the data themselves are here and that's the document that's the data sorry itself okay so that was quite simple really to to get access to the data now let's go back to finding data so I'll search again at the beginning there are ways of refining your search really using topics which are defined the data type that isn't very surveyed data the access level that you're interested in and you can also in here you can see that other people have suggested other things that you might wish to search for or you can do a keyword search which should cut down the number of results that you get because the keywords are signed by the data service and by producers specifically for that particular kind of information okay so let us now look at so I'm now going to search for a different data search this is the British Social Attitude data that I talked about earlier and again we've gone through the series page here so you can see all the different data sets available and if we click on access data now you can see there is only an end user licensed version of the BSA there are no other versions so if we click on the button at the right it should bring up the different ones and we can either go straight to Nesta this way or we can go to the catalogue page and access data on the right hand side and scroll down and here we can add the data to your account so that's adding the whole data set as I just demonstrated or you can go using Nesta to have a look at the data online okay so this is what Nesta looks like and essentially we have a list of all the different data sets that are included in the Nesta catalogue not everything is included in Nesta it tends to be it is survey data and census micro data so that's the data that is a sample of the entire UK census that has been anonymized that looks a bit like survey data and you have some unrestricted access data sets at the bottom and some teaching data sets as well now because I came in via the BSA 2021 Nesta has gone straight there to that data set if I were to click on metadata I would get more information about the documentation again which I think you can already see on the catalogue page if I click on variable description this is where this becomes useful so I could click on internet usage and you see that there are three variables here so on average how often would you say you use sorry say you access the internet personal use and what this is doing is showing you the in a sense the raw data or the results from the raw data so these are unweighted frequencies from the data for that particular variable and this can be quite nice if you haven't got a computer that's powerful enough to download the data so you're using a very old computer or you just want to have a quick glance at a variable or if you want to teach your students about survey data but you don't want to teach them to use a software at that moment in time anyway you just want to talk about coding and more basic kinds of things this could be very useful it could use e-coding so the different values and the categories that relate to each of those values and the number and the percentages of those who selected them and the number of valid and missing cases okay so let's look at some other ones that are derived variables same sort of thing what is your sex and again here we have roughly 55% were female and 43% 44% male okay so what else can you do with this now everything I've done so far you can do without logging in however everything else you do need to be logged in it's just the UK data service login I should say that Nestar is it only works in Firefox or it tends to only work in Firefox let me look at the login sections and you also may see that it's a little bit yeah it doesn't always work as you expect entirely so we are in the process of getting an alternative to Nestar something that will do something similar and be more reliable but it does work for the time being but like I say if you're planning to use it go into Firefox and I suggest you log in by using it so that you can use all these other aspects so you can create cross tabs so let's see let's look at internet usage so I'll add that to the row this is where we have the login issue okay and it's just asking me to agree to some conditions that's the NGS license conditions same information as we had before but in a table but now we can also wait the data so Nestar knows which are the waiting variables and we can move that across and then click on okay and now these are the weighted results and we can see that the weight is on because it says so at the bottom okay and you could do a cross tab if you wanted to so should we add sex to a column and again we have to accept the conditions and here we are right so we have female male prefer not to answer total and these are the percentages okay so if you're interested in this you can play around with it yourself and see all the different things you can do you could produce some various bar charts based on the data that you've added can do various things you can also select subsets of the data and you can compute your own variables and so on you can also do other analyses like correlation and regression I have to say I wouldn't use this for regression you could use it for correlation if you want to however okay so that's briefly Nestar okay let's go back to the UK data service so there's one more thing I wanted to show you before I just showed you the whole website some other web finding data which might interest you is what this main catalogue search is doing is searching the information that you would see in the main catalogue page so that is the title and the abstract and other similar things about the data the variable in question bank is a different way of searching so it's searching specifically about the variables and the questions so for example say you're interested in bus travel as you're interested in BSA you could search for that, that's not where it says it but occupational classification well yes well no doubt bus is in there somewhere so that's useful information is it okay maybe reason you don't have a job okay so this has found a variety of different things none of them actually related to buses but anyway you can see that this could be useful to search for specific things that are of interest to you but as we're demonstrating here you have to be careful what you select otherwise you end up with a lot of stuff that perhaps wasn't what you were after okay and that was all I wanted to show you really about searching for data a couple of quick things as I said if you are collecting your own data at any point in your career you can deposit it with the UK data service so have a look at that information if you're thinking about that even if you're at the start of your collection it's a really good idea to think long term about what you would do about sharing it and this is a good place to come for information the learning hub has information about how to use survey data as well as other kinds of data and also about how to manage your research data if you are collecting your own data shows upcoming events it also shows past events if you want to look at information about past events which often includes slides and recordings from events the about pages is about the data service and news is what it sounds like impact is how the different datasets so that's quite nice to look at thing that's most likely to be relevant if you're new to the data services the help pages there's some really nice information here frequently asked questions really and something that's very important is near the bottom here there is help desk contact information the UK data service runs an email help desk and you can contact us by clicking on the say you wanted access and general enquiry okay you would fill in this web form with your information and submit it to us and it goes on to our help desk system and the query will find its way to the person who's best able to answer your question and in relation to surveys you can answer questions about how to find data how to find particular variables if you're struggling having looked at the documentation if you spot anything you think might be wrong with the data if you just want some general help in general with using surveys but what we are not able to do is to do your analyses for you so however we are willing to using weights for example if your needs that okay and I think that was everything I wanted to talk about