 There we are. Hello, everyone. Thank you very much for attending this research questions workshop. My name is Dr. Jules Kazmaier and my co-leader today, Nigel. We've both run workshops on research cycles and developing research questions in the past, so we thought we would join forces and present you a combined effort, hopefully that will make sense to everyone as we move through. And now I'm going to pass it off to Nigel who will walk you through the table of contents and he will start the content of the session. Okay. Hi everybody. So just to preliminary think about what the workshop is and what we think you're going to achieve. This is not trying to replace the relationship you have with your supervisor in discussing what you're studying in refining your research questions. It's just a kind of kickstart of a 10, I suppose. And we're going to introduce ourselves and how we make research questions. We do this slightly differently, so it's been an interesting exercise thinking about that. And we decided we would each present our different methods. I'm then going to go on and talk about the research cycle, some types of questions and some examples. And then we're going to move into a breakout session. In that breakout session, Jules will tell you more about that. We're going to use a tool called the Myro board and we selected up to six of the questions. So we will need to check if you submitted a question and it's on the board. So we know that you're there. So we will run the breakouts that people have sent into us. And then we'll kind of wrap up after that session. So that's the structure of the day. As Jules said earlier, if you have a question as we're going through, then pop it in the Q&A and we'll try and pick it up at an appropriate point. So here we go. Here's me first, I think. So I generally start with a substantive set of interests and most of my research is around housing race and migration. And just thinking about how you start, I suggest you think about first of all, what's the focus of your research? Who your research subjects are and any theoretical or policy frameworks that are applicable to the research you're going to do and then broadly follow the research cycle, which I'll cover in more detail in a little bit of time and over to Jules. That's right. It's me again. I want to give you a very brief walkthrough of my sort of academic experience. I started in linguistics and then I moved to philosophy, psychology and language and then to complex adaptive systems, which was technically at an engineering university. And now I do computational social science with the UK data service. That means I use computationally intensive methods to address new social science research questions that we could never address without the use of computationally intensive methods or to address old science, social science research questions, but in new ways that are enabled by the computationally intensive methods. And I don't have the same research cycle as Nigel will introduce next because my academic experience here, although it's very social science oriented being about linguistics and psychology and philosophy. Because of the computational methods, I'm much more quantitative focused, much more empirical research focused. And so I have an eight step research cycle, the first three of which are relevant to developing research questions, which is identify the problem to solve, explore the problem and formalize the concepts. So you kind of loop through these first three a couple of times. By the time you move on to four, five, six and so on, you probably have a pretty firm research question. So we can talk about the other a little bit more later on. But for now, I have another meant to meet our interaction for everyone. What topics are you working on? It's important to know that there is no correct answer here. You are working on the right things for you, I have to assume. But we are interested to know just broadly just a couple of words, maybe a couple of big themes or a couple of focus words. Like what's in general, what kind of topics are you on? Let's see. Oh, do I have to start? I don't know. Let's enter to start quiz. Okay. This is a new thing. I have set up this meant meter in a strange way that is giving us 30 seconds to enter the answers. For some reason has done this as a competitive quiz instead of just showing the answers. I have no idea what happened. Apologies. But it looks like five people, six events, eight. Good. So you are answering and they will show us the answers presumably when the timer runs out. Again, apologies for this. It would have been nice if it had just showed us the answers collectively as they were entered. Okay. So we've got some good ones here. Loneliness, gender equality, haptic tech, doctoral education, orangutan behavior. That's amazing. Business support policy, education, mental health nursing, sexuality and disability. Great. Some great broad themes. A couple that overlap in a few ways, but it's also a very diverse sort of set. So hopefully we can come together well enough in the breakout room section that we can give new perspective on each other's questions and that's a really beneficial outlet. So moving on to the next section, I will pass it back to Nigel because he's going to go through his research cycle. Okay. So this might be a bit more familiar to some of you, but I suppose I'm a traditional socialist scientist and that what I start with is kind of what we know already. So looking at the literature at the moment, I'm trying to get to grips with housing and racial capitalism. So I'm looking at theoretical concepts, empirical studies, not so much a policy because it's more contributing to a kind of way we think about it. I'm thinking about research questions. So what do we want to know? Where are the gaps? And the cycle goes around to where's the data? What methods am I going to use? How am I going to marginalize the evidence and how am I going to report them? But on each of those subsequent steps, we push back and this would be much more complicated. I was talking about the whole research cycle, but for example, I've got a research question. I go and find the data and it doesn't quite match what I want to find out. So what do I do? Well, I can go back and refine the research question or think about it again, maybe come back and think about different sources of data, maybe moving to primary data collection, as opposed to secondary data. Once I start to look at that data and develop the evidence, again, I might return to the research questions. And similarly with reporting, I might find gaps in what I'm finding. So go back to the research question. So this is quite an iterative process. That's probably why we spend three years doing a PhD or the best part of six months doing a master's dissertation and why those of us working in research will spend quite a lot of time here. I suppose as you get more experienced, the theoretical basis and the empirical studies start to become part of what we know. So this may be the first part shorted down, but the rest of it identify new knowledge and then thinking about how to gather that is continuous. And to talk about methods just a little bit. A lot of people who completed the information were talking about primary data collection. It's worth saying the UK data services is predominantly a space for secondary data, survey data and qualitative data. In terms of methods, probably most of our experience is qualitative, quantitative, though we do have people with qualitative experience and also mixed methods. So personally, I've got experience in mixed methods using a range of different qualitative methods as well as constituted ones. We could move on. And here's a kind of thinking about the process of describing things and then moving into action. And I've used two examples here. The first one is the evidence for equality national survey, which was developed and carried out by the central dynamics of ethnicity based on a number of universities but quite a strong presence in Manchester. And they set out to ask this set of questions. So what would a racial justice society look like? How close is Britain to being a racial justice society and has the COVID-19 pandemic taken Britain further away from racial justice and ethnic inequality. So quite a broad set of questions they produced an ebook which is free or you can buy it if you want to. Covering a whole set of areas of people's lives. So experiences of racism is probably the headline message that wasn't really there in lots of the survey data. So there's quite detailed discussion of people's experiences of racism in Britain in different types of settings. If we move on from there, this is a kind of, sorry, back to the last slide. The way Manchester then as a council decided to take forward their own understanding was to get a better understanding of their community based on a series of activities. But someone using the census to understand the demographics of the population, someone talking, engaging, working with voluntary sector partners, et cetera. And what they wanted to do with that was to take that understanding that they gain to support making better places where people get on better and treat each other with respect and consideration. And that's a kind of part of their desire to commission services that meet the needs of everyone. So if you move on from there. This is a paper I was involved in called slippery discrimination and here we used a range of different methods to think about housing disadvantages faced by migrants and ethnic minorities. I suppose the headline message is in the title slippery discrimination that discrimination operates, not on a rational basis, but on a kind of quite slippery basis. So we did some work to identify the legal policy and market forces that shape housing disadvantage, how they've changed over time, how they're manifested nationally and locally, and how they're responded to locally. And the aim of the paper was to push the housing studies agenda towards more research and policy to engage with local actors and develop ways of overcoming migrant housing disadvantage and child discrimination. So the three methods we use was historical policy analysis, statistical analysis, central micro data and a facilitated workshop with key stakeholders. So if we move on, I'll just give an example of the output that came from trying to understand the migration and legal changes over time. So along the top line, we mapped all of the immigration legislation that has been enacted in the UK up to about 2015, I think, because this article was produced in 2016. We then looked at citizenship. At the bottom we looked at housing changes. Now housing policy changes very often, so we just picked out key points from there. And then in the middle, we modeled net migration over those years and key other pieces of legislation. So the various race relations act, the creation of the Commission for Race Equality and Equality in Human Rights, the commitment to provide adequate housing under the UN covenant on social and economic responsibilities. And then we also included some patterns of migration that were experienced in Britain over time. It was quite a complex start, but what it gave us was three frames of time. So the first frame was where we had kind of relatively open borders, people could come, say, work and go and so on. We then moved on to external border controls with immigration acts and so on that that curtailed the rights of people to enter the country. And in the last phase, we look, we see internal borders. So if you work in a university, you will see the internal border operating probably somewhere in your building. Every so often I go downstairs and see it and there are cure students waiting to get their visa sign. That's the same is true in health. The same is true in housing, in access to public services and making a bank account. So there are a whole set of internal border controls that make it more and more difficult for people whose legal status changes for whatever reason. Okay, on to the next one. And then the last one I'm going to talk about is thinking about policy. So I don't work with the race equality foundation over time. And what they were interested in was thinking about housing for black and minority ethnic older people. And the first thing we did was forecast. So we looked at the demographic profile of the larger ethnic groups who are likely to contribute to the growth of the population. We identified housing deprivation of those groups by age and explore the factors that contributed to higher levels of housing deprivation experienced by some groups. And looked at the usage of care and residential homes and discuss the geographical concentrations of ethnic groups which are quite different from concentrations of older people. So typically if you look at a map which we'll look at in a minute, when you look at older people generally the move into suburbia or to the coast is quite a common pattern with retirement. So when you look at older ethnic minority groups they tend to be more concentrated in cities, but probably for reasons of kind of family kinship and cultural ties. So here we want to the next one some of the output will. Sorry, I didn't put the end so you won't see it. So other types of research. There's a classic statistical research where we start with empirical evidence. We set a hypothesis and a null hypothesis and we set about proving that using statistical methods. There's more trying to look at causal relationships that tends to use longitudinal data and looks at precursors of particular events or conditions or and so on or potentially looking at treatments and outcomes of those treatments. So what I've shown you is not an exhaustive list. It reflects my experience and looking at the things that people have submitted a lot of you are coming from quite different places. So if we move on to the next one. Got another quiz. Inexplicably it's decided to make these quizzes. I wanted just a free text entry, but this time tell us what methods you're using or planning to use. This doesn't necessarily to be about the same research question, but just what what methods do you use in general or hope to use. All right, so we'll get a count down here. Everyone gets max points. So that's encouraging. So we'll get a count down. Just wanted to say I forgot to mention earlier when I introduced my eight step cycle that if you do empirical or computationally intensive or very quantitative driven research, you might consider the becoming a computational social science workshop that we run twice a year through the UK data service in which the eight step process that I outlined briefly. We go through it in detail. So if that's interesting to you, there is ways to know more. Okay, we've got mixed methods, qualitative methods, interviews, document analysis, instrumental qualitative case studies, arts based methods such as drawing. That's interesting. I don't know that I've ever worked with that one. Sensory, ethnography, interviews, focus group discussions. Looks like there might be one that's off screen for some reason and I can't quite see it. All right, so we've got a few different options here. I can't say that I'm surprised that no one's put computationally intensive methods. It's a growing field, but so far it's fairly small. So moving on, now we're going to talk, this is me for the most part, although please do feel free to chime in if you have something to say Nigel about how to really turn what you have so far into a research question. Because as Nigel has explained so far, most of us start with an idea of what we want to know more about or there's a problem or a pattern we see in the society that we want to explore further. And you do then some literature research or some finding out what other people have done or things like that, maybe get a sense of what data is available, what methods you might want to use, and you start bringing all those together into a research question. But probably that will not be very well structured as a research question. So what you want to do is turn that initial maybe fuzzy kind of research question into something that is clear, focused and concise. These are the three things that I think are requirements for a good research question. So clear means using straightforward grammar, language appropriate for your audience, a sensible question. Focused means that it matches the time, resources and data to which the researcher has access. And concise means it's expressed in the fewest words needed. Now that's fewest words needed, not fewest words possible. I'll go through some more detailed explorations of each of these. So clear. For example, if someone started with an initial research question like how should social networking sites address the harm they cause. I would respond to say this question doesn't specify what type of social networking sites doesn't specify the harm at all or who is harmed. And it does ask your audience to trust that there is harm already, which is a bit ambiguous. It leaves a lot of room for interpretation. It's not the clearest way that this research might go about creating a research question. So I suggest something more specific like what actions should Facebook take to prevent vulnerable users from exposure to extremist propaganda. That is a clearer research question than the original. So we can move on to focused unless you want to say something about clear clarity Nigel. No, it's good. It's clear enough. So focused is what effect have anti climate change innovations had. While a very interesting question. This is not focused. It's just very, very broad. So what is, you know, it doesn't specify the effect. So what is being measured or how or what even is being studied. What do you define as an innovation doesn't measure doesn't specify the scope. So the space and time boundaries to consider. Are you talking about anti climate change innovations around the world for all of history or just in the UK in the last 20 years. It doesn't mention the details. So what level of observation are we talking about individual units, you know, the heat pumps that I want to install in my house or heat pumps in general, or any kind of greener heating technologies. You know, this is the things to consider about specificity and focus. So a more focused question would be what effect have UK government green grants had on heat pump installations since 2008. Clearly this is more focused. It's the kind of thing you might be able to answer in six months to a year. Whereas the original way that this was phrased, you might struggle to answer that in a career. Concise. This one's a little bit silly. So what measurable difference can be seen in pre and post COVID academic lockdown testing outcomes of human individuals between the ages of 13 and 19 when those individuals are domiciled within the country of Great Britain and Northern Ireland. Obviously this is overly academic and the effect of that is it doesn't make it seem more clever or more intellectual. It just makes it hard to follow. So a much more concise version would be what differences can be measured between pre and post COVID student test scores for UK residents 13 to 19. It's the same basic question. It's just in a more concise way. Any comments, Nigel, because otherwise I will just take a poll which people find the hardest. I suppose the one thing to say is this idea is something that's new and original. Yeah. Oh, that's coming up. We've got that next. Okay, I'll say it then. Okay. Okay, it's good. So we find some people are struggling with focus and absolutely that makes a lot of sense. We want to be ambitious. We want to answer big questions. We want to make changes in the world that we're interested in. It's not surprising that a lot of us struggle with focus, but we do have to be realistic about how much we can accomplish in a master's dissertation or a PhD project or a two year postdoc project or something like that. A few people struggle with concise and I understand that as well because there's a lot of academics who like to sound really hoity-toity. I believe it's the technical term for it. They like to sound clever. They like to use big words just for the sake of it, not because it's the right word. So nobody struggles with clear. So I appreciate that. We're all perfectly clear. However, we're wordy and overly ambitious. So here's four more things that I think research questions should have. These I've marked as desirable rather than required and novel. This is the one that Nigel was commenting on there just now. This should address a question or problem that is not yet fully or sufficiently addressed. Now, this could mean you take a well accepted method and a well accepted topic, but apply it to a new population or you maybe slightly change the method and see if you get different answers. Novelty is subjective. Don't feel bad if you're doing something that's very like what someone else has done as long as it is not completely the same or it even could be completely the same if your purpose is to verify the outcome. That's also useful science. This is why this is desirable rather than required. Arguable is just means that you can't answer it with a simple yes, no, a simple repetition of facts. It should be an idea that is interesting and scientifically arguable objective. So you shouldn't rely on good, bad or similar kind of judgment words. So if you want to talk about housing discrimination, for example, you need to say, you know, that people are being overcharged or that people are being put an inferior quality housing stock or these kinds of things that you can measure. The money that people are paying or the damp factors in the house or something like that, rather than just saying good housing, bad housing and appropriate means the question answers match not only the time and your resources for the researcher, but also that the question and the data and the method should all match together. So you shouldn't expect to answer a qualitative, sort of descriptive big question with some simple statistics. Likewise, you shouldn't, you know, it shouldn't approach a simple kind of straightforward comparison question with a wide ranging focus group, maybe statistics are the right answer for that. Any comments, Nigel? No, no, I think you picked up the point I was going to make. Yeah. Novelty, it's good to be novel, but it's more important to be appropriate for the research you're doing. So don't don't get hung up on novelty. And of those four that I I've considered desirable rather than essential, maybe you disagree, maybe some of these are absolute must haves for you. And that's fine. So think a little bit about it. Maybe which of these is most important to you? Yeah, see novelty, it's really important to a lot of people. People like their novelty. Arguable, I quite like that one. I think, I think for a good research question, I would agree, I would want to make it arguable. But again, it depends on the research focus, you know, the time you have that's allowable, it might be worth for you to have a very straightforward, simple to answer question that you've answered very well. Less, less love for objective and appropriate. Those are maybe a bit weaselier. Okay. Before we move to breakout, do we have any questions that you want, either myself or Nigel to answer?