 Hello to everybody who's already here. We're waiting just a few minutes for a late, well, on time joiners. We've got three more minutes until 10 o'clock when the session officially starts. Good morning. Morning, everyone starting just well, 25 seconds according to my watch. Let people get in sometimes it can take a little bit of time because we've got the waiting room enabled. It's gone 10 o'clock. Thank you everyone for joining. My name is Dr. Jules Kazmaier and I will be leading the session today with my colleague Nigel. We both have in the past led sessions on developing research questions or beginning the research cycle. And we have found it is not uncommon that people get to a certain part in their research career and realize maybe they don't exactly know what a research question is. And they feel a bit stuck. So we're hopefully going to help people with that sort of disentangle some of the issues and some practical advice on how to write good research questions. Today, we will be using Mentimeter. So you can use the Zoom chat, but that's sort of for technical questions for our facilitators, things like where is the Zoom? You know, will the slides be available? Where is the Mentimeter link? That kind of thing. You can ask those kinds of questions in the Zoom chat. All the other questions, so content questions, feel free to ask in the Q&A function through Mentimeter and all other interactions will be in Mentimeter, such as polls and short answers and things like that. So let's go ahead and demonstrate Mentimeter now for anyone who hasn't already done it. And the first poll is simply can you hear us? So you will go to menti.com and enter the code at the top of the screen. That's three, two, six, nine, three, seven, six, one. Looks like so far one person can hear us, which I'm counting as a success. Let a few more people join. Sometimes it can take a little bit of time to join. You're welcome to do the Menti interactions on a mobile phone or a tablet or a second computer if you want to keep this screen that you're that you're viewing this session on is completely separate. Alternately, you can switch back and forth between applications on one screen. You don't necessarily need a second screen. Okay, we have 35 participants and so far 10 people have contributed to the poll. Fortunately, all of those people can hear us. So I'll give this just another 30, 40 seconds to let people get in and enter the Menti code and start interacting with it because this hopefully shows that your interactions appear in real time, more or less on the screen, and also that they are anonymous. So you don't need to worry about anyone knowing who has said what. This, of course, is not very important in a poll like can you hear us, but it is more important potentially later when we talk about research fields or research methods or experiences or worries or things like that about research questions. Okay, I'm going to assume that because the numbers for this poll are still climbing, people are still joining. That's fine. I'm going to move on. You can still join the menti menti.com and participate. So let's move forward for anyone, although no one voted no on that poll, anyone who does have some audio problems who couldn't hear us, there are a few basic things. The important thing to know is that we are recording this webinar and we will post it on YouTube. And in fact, we are live streaming this webinar. It is appearing now on YouTube. The disadvantage to watching it on YouTube instead of here is that you won't be able to participate in any interactions that we have live. We're not monitoring the YouTube channel. So moving forward to our table of contents. First, we're going to introduce ourselves, talk about some research, talk about some types of questions, develop some examples of research questions. We are not doing a breakout because we have too many people attending this session today. We had yeah, it was a bit oversubscribed and we thought breakouts wouldn't work in such a large group. So instead, we'll demonstrate several more examples and then we'll give extra time for the Q&A at the end. So moving into introducing ourselves, I'll start with Nigel. Okay, so I suppose as a researcher, my focus is on areas of substantive interest, particularly on housing, race and migration. I'm thinking about how I've arrived at that. I'm quite a late comma to the academy. I started studying again in 2012. So I suggest the starting point is to think about what's the focus of your research, who your research subjects are, and the kind of theoretical and policy frameworks that might be applicable. So for some of us, the theoretical framework is applicable. For me, the policy framework because it's about housing is quite a strong component. And then I will go on to talk about the research cycle and the kind of I suppose I'm going to problematise it a bit by saying how messy it is, but I'm going to broadly talk about that and how that relates to research question. So over to Jules. Thank you very much. Let's move to my slides. Hi, I'm Jules. I want to tell you a little bit about my background. I started in linguistics, and then I moved to philosophy, psychology and language, and then to complex adaptive systems. And now I'm a computational social science researcher with the UK data service. And you think computational social science? Never heard of it, mate. I understand. It is using new computational methods or computationally intensive methods to address new social science research questions, questions that can't be addressed at all without computational means, or old, you know, classic social science research questions, but addressing them in new ways that were not previously available. And a little bit more about my approach to the research cycle is I have an eight step sort of it's an iterative looping kind of cycle, but more or less that you march through the steps, sometimes going back and then forward again. The first three in which you identify the problem, explore the problem and formalize the concepts, I think are the most relevant to developing the research questions. And if you are interested in knowing more about my eight step research cycle, you can sign up for my becoming a computational social scientist workshop. These happen twice a year in the UK data service. You can find, you know, and maybe Jill can share the link to the next one in the chat, the Zoom chat. So thank you very much for sitting patiently through our introductions. And now on to the next interaction. We want to know what topics you are working on. So again, this is done through mentee. And just a couple of words, this will appear as a word cloud. Tell us maybe housing or autism or urban regeneration. Very cool. I did a project on that. Aging, also cool. Gender, mental health, children, crowdfunding. Oh, we got some great, great options here, some good stuff. Maths, anxiety, pensions, genetics. Wow, this is filling up fast. Guys are amazing. Fantastic. Intercultural education, design, nurses, health and well-being, migrants, waste plastic. Fantastic. Well, we've got clearly very diverse topics that we're all working on. So hopefully, with all of the different examples that we're going to show you about research questions and all of the different features of research questions, you will find things that apply to your particular topic, because it can be a little bit hard to understand how abstract discussions about research questions might apply to your particular field of interest. But hopefully, we'll find something that helps you out. And also there's plenty of time at the end for Q&A. So next, I am going to pass it back to Nigel, who has some explanation of the research cycle and how messy it is. So just thinking about the research cycle, I suppose I'm fairly much a traditionalist and that comes probably from teaching on the graduates for a few years. So I would always advocate understanding what we know already. And that might be looking at academic literature. It might also be looking at policy literature and policy papers. And from there, beginning to think about what we want to know. And then thinking about the practicality of how to get to that. So what data and methods will we use to find that out? And that may take us back into an iteration because of what isn't available or what is available. We might either expand our questions because there's more data than we think that we could get hold of or reduce the scope because there's less. As we begin to look at that data and I'm pulled together some analysis, we might identify new topics. So for example, I did my PhD on housing focusing on the private rental sector. And I didn't include race as a characteristic. There's lots of evidence to say it's quite important. So it emerged. So my research questions then developed and saying what are the characteristics of people who are living in the private rental sector and describing those and age and race became quite significant factors in that. As we begin to produce outputs, similarly, we might find there's a hole in what we're saying and we need to return and go back. And I think one of the important things to say to most of you at least is you're doing this in a relationship already an existing relationship. If you're studying with a supervisor and they're at the center of these discussions, I mean, we're giving you a brief input which hopefully informs you and how you take on those discussions. But the key person, the key people are you and your supervisor and how you manage that relationship and how you work your way through this research cycle. I think it's probably fair to say that most people doing research never really get to the end of what they want to achieve. Our ambition is always a bit higher than what we achieve. So there is a kind of pragmatic element to this research cycle about where we draw a line and say, right, we've got enough. That's enough for me to get the bit of paper I want or to produce the report I want or to get paid for this piece of research I've done. Okay, next one. So I'm just going to go through some practical examples. So there's a research centre called the Centre on the Dynamics of Ethnicity and they made a proposal and got funding to carry out a survey on evidence on ethnic inequalities and produced a free ebook which you can access if you just search for evens. We also hold the data set, but it asks these three big questions. So what would a racially just society look like? How close is Britain to being a racially just society and has the Covid-19 pandemic taken Britain further away from racial justice and ethnic inequality? Ethnic equality. And that's a kind of quite broad topic area, set of topic areas within the analysis. They went into different aspects of living, so health, well-being, mental health, housing, education, et cetera, et cetera, employment skills. To take that into a kind of policy context, this is a statement by Manchester City Council and it's really about saying because we know more now because of the census data, we want to understand better what's going on in our communities. And there's still time in the community cohesion, which is another debate altogether. But in effect, they pull together all of the evidence they have. So the demographic evidence they can get from the census to understand their population and then the things they learn through the different types of engagements they go on. So if you were going to say this, this is quite an open set of questions to start with saying, well, what is our population like? What things are they facing? But then it moves from there into actions and commissioning services and policies that link to addressing kind of what they find in terms of fundamental inequalities within their citizens. So we'll go on to another one. So this is a piece of work I did with Sue Luke's from London and Lisa, who's now at St. Andrew's. And what we were looking at was housing disadvantage faced by migrants and ethnic minorities. I suppose the kind of rationale here was that housing studies is not great at looking at issues facing people because of migration and race. And migration and race are quite messy concepts in that, you know, my mother was a migrant for a while. I don't know what if she came to a point in their life where she wasn't a migrant anymore. So therefore I'm the child of a migrant and I'm from an ethnic minority. But in the way I'm treated in certain settings, those definitions don't mean anything. I might be treated as a migrant by people because of the way I look. I might be treated as an ethnic minority because of the way I look. So we kind of, there was a problem in there that we wanted to articulate the need for further focus and particularly a focus around this kind of messy interface between migration and race. We also looked at the legal policy and market forces that shape those, how they developed over time and how they were manifested both nationally and locally. So in this example, there's not a specific research question, but I think I've framed the questions we were looking at. The rationale for us doing it was we thought this area was under research and the kind of questions we wanted to look at what, how has this come about through the legal policy and market forces, how that's changed over time and how that operates differently at different scales. So if you think of housing, it is basically a local policy function. So it operates quite differently in different places and how people respond to them locally. There's a great workshop. I think the most interesting person for me there was probably the Bishop of Manchester, who is a strong advocate of social housing. And he basically just talked about his mailbox and what mail he got, which highlighted the issues that kind of framed a lot of what we, we learned from that element of it. So we looked at historical policy. We did some statistical analysis of census microdata and facilitated that workshop with academics, key kind of stakeholders in the housing field and voluntary sector organisations. So people like Joseph Rowntree were there, et cetera. So if we move on, I'm just going to show you an output from that, which was around the policy, legal challenges. So what this was trying to show was how things have changed over time. So in the middle of the graph is net migration. And it's got immigration and emigration. And so that frames that timescale, which was from 1964 to about 2016. We then looked at the kind of legal changes that had happened over that time in terms of immigration. So the way that immigration controls have kind of operated. So the first kind of example we came across is the 1905 Aliens Act, which was excluding Jews and then transferred to enemy aliens as well. That was repealed in 1919. And we begin to see them acts associated, I suppose, with elements of racism about feeling the country has been swamped. So from a period in which external borders were created to the current period in which we have internal borders. And for those of us in universities, we can see the internal border operating. So in my building, I walk in and there's a sign saying, these are controlled over here. So our internal border operates in my building. It has been in other universities I've worked in as well. I kind of struggle with it. We have our different points of view. The second thing that came along that we looked at was citizenship. So and this is a period where people like my mother were citizens of the British Empire. They were born into a British colony and inherited that status. When she came to Britain, she got naturalized. So she then became British. But us as and us as children of those generations of migrants inherited British citizenship by being born in that period. But from 1984 onwards, those rights are not automatic. And we've seen more recently how those rights can be taken away from people who are born in Britain. In the case of the Bengali girl who went to Syria. And then the final part of the legal framework was the way that housing has changed. And particularly the way we've moved from a process where housing and legislation really focused on housing as a right, moving to a focus where it became much more about the market and the asset of housing. So in particular, the 1988 act, I think, took away security of tenure and rent regulation. So we're now in this scenario where, excuse me, where many of you are possibly renting and facing significant challenges in sustaining your livelihoods in the face of the way that rents are behaving. And then the final bit of that is to put in some things around different acts and different events about which groups came in, which organizations were working. So the 1976 Commission for Race Equality, which framed a lot of the way that housing was shaped and housing became part of the remit of that through to the Equality and Human Rights Act and then patterns of migration. So that's that's a kind of piece of work that we did. And yeah, go on, move on. Another piece of work I did was looking at thinking about policy. So I've worked for quite a long time with an organization called the Race Equality Foundation in London on housing stock, and they work with the Housing Learning Improvement Network. And what they wanted to do was to see what how aging is going to affect different ethnic minority groups. So the first thing I did was to kind of think about the demographic profile using census data to see what the growth was because migrant groups tend to be younger when they arrive at the kind of pattern of migration is around the kind of late teens through to early thirties. So you have a much younger population when you have a group of new migrants coming in. So that was particularly true of the Commonwealth migration from the Caribbean from Africa and from India in the period of the 50s to the 60s, 40s to 60s. More recently, that's been true of other groups coming into Britain, so patterns of EU migration, etc. So looking at how those will contribute to a growing older population, looking at housing deprivation of those groups by age and thinking about why there are higher levels of housing deprivation experienced by minority groups that that partly is still kind of fairly sustained. To think about care and residential homes and to think about what those demographic changes mean for future demand and then to look at some of the geography of that concentration, which is quite different to the geography of older people more generally. So if you look at older people across Britain, there is a tendency to retire out of cities to move out of cities and older populations tend to live in areas like coastal and rural areas. Whereas for a lot of minority groups, that pattern isn't the same. So, okay, so on to the next slide. This one's me. So I wanted to explain a different kind of research. This is a project that I've been working on in fact that I'm hoping to submit for publication next week, if I can write the conclusions. The research question underpinning this research was what differences can we find in the way human geneticists use person first and identity first language. So an example of person first language is person with autism or child with ASD or something like that. An identity first language would be autistic person or diabetic children or something like that. So we looked at all of the abstracts submitted to the European Conference on Human Genetics between 2001 and 2020-21 that ended up being almost 40,000 abstracts. And we looked specifically in the context of autism because we wanted to narrow the focus down. So not just person first language and identity first language, but person first and identity first language around autism. And we wanted to see how the use changed over time, how the nouns used were different. So whether it was person or child or cohort or something like that. And we wanted to look at abstracts that used both kinds of language within a single abstract. And we did this through text mining and natural language processing methods. So this is a much more computationally intensive method, as is what you might expect from my research focus. And our conclusions were that roughly the same number of examples of person first and identity first language in relative to autism in the articles in the abstracts. And they were more or less as popular as sort of up going up and down more or less over time until 2019, in which they started one started being much more popular. And then the next year reversed and then the year after that reversed again. So they just instead of going together, indicating general popularity of autism as a topic, the language started really changing. And that was interesting. We also found there were fewer different nouns used in person first language. These were almost always people nouns. So boy, child, sibling, family, things like that. Whereas the identity first language used a greater variety of nouns and they were much more science nouns, things like cohort or population or subject. We also found that about 20 percent of the abstracts that had both at least one example of person first or identity first language around autism, 20 percent of those used both patterns, which is interesting, it shows that people probably are using them more or less interchangeably, or that they were using them in some ways like they might use identity first language for the cohort and then person first language for an individual. So this does answer our original research question, but then it motivates other questions like what happened in 2019 when the pattern started changing, or can we drill down into those patterns in the abstracts that use both to see which one is like how they're being used, a bit more of the context. And that might involve a lot more reading rather than computationally intensive approaches. But at least we know which 20 percent of the abstracts to read manually instead of trying to find them out of the original 40,000, which does not sound fun. OK, so we've we've given several different examples here with a couple of different kinds of research method. And we do want to point out that some types of research, namely like empirical or empirical styles of research will have hypotheses and no hypotheses. So, for example, my question about whether identity first or person first language is more common in this in this set of abstracts that, you know, we could answer that with a hypothesis in a null hypothesis. Other things like causative kinds of research or longitudinal research are less likely to have these kind of strict hypotheses approach. They will also have different kinds of methods, potentially different kinds of data. Nigel, did you want to say any more about this? What I'd say is I suppose we're we're presenting we're presenting ideal types and kind of looking at what people are submitted. We are doing people are doing quite complex things. And many of you at the beginning of that journey. So I would say I'm a kind of mixed methods researcher that I get paid for being a quantity researcher. Yeah, yeah, it's it's lovely to idealize these kinds of research, but they they often come out much messier and squishier. And I think for many of us being open to the kind of methods to find the data we want is a kind of useful attitude to to adopt. Agreed. So I wouldn't be frightened by the kind of complexity of some of these if they help answer your question. And you find that other people have been doing them. They may well be things that you will want to engage with more. Yeah, I agreed. I find that the research question, the methods to use and the data to use all kind of inform each other. They all have to play nice together. And you might end up changing your question. If you are not able to get the data that you wanted and you get something else instead or you might change your method or both or it all keeps going around. Everything's changing every time you look at it. Try and avoid that if you can. So we've got another interaction for you in this time. It's about methods. We'll have another word cloud. So pop in the kind of methods you use or that you want to use or that you typically use or that you absolutely hate using. I don't know. I mean, it's up to you. Talk to us about methods. OK, qualitative, longitudinal, mixed ethnographic observations, diaries. OK. Cognitive interviews, web surveys. All right. We've got comparative case studies. I have to say no one so far is using text binding and natural language processing. I feel deeply hurt. Just kidding. OK, focus groups, semi-structured interviews. Yeah, we've got quite a variety of observations there as well, quite a variety of methods here. And each of these will lend to themselves to creating different kinds of data or to using different kinds of data. And that will relate to the research questions that you can answer or that are a good match for these kind of methods. So you're not going to enter. You're going to struggle to answer certain kinds of, you know, empirical questions with hypotheses and all hypotheses with semi-structured interviews unless the questions are really basic, like, do these people talk about these things, which is not a very interesting research question. OK, got some great variety here. Thanks, everybody. Data scraping. Woohoo. Got a computational method in there. All right. Moving on. We are now going to talk about actually how you develop the research questions. And this is a little bit some focus on features that your good research question should have. And I do want to point out this is a good research question that is the end of the research question process. So you start with a bad research question and you make it good by the end of the process. By checking, for example, is it clear? Does the question use straightforward grammar, language appropriate for the audience? Is it a sensible question? Is it focused in that it matches the time, resources and data to which the researcher has access? And is it concise? Is it expressed in the fewest words needed? Note that's the fewest words needed, not the fewest words possible. So let's go into these in detail. Clarity, for example, you might start with a question like how should social networking sites address the harm they cause? Unfortunately, this question doesn't specify what type of social networking sites or what kind of harm or who is harmed. And it doesn't even necessarily support the existence or the extent of that harm. It's this question as it stands is not very clear. It's too ambiguous. It leaves too much room for interpretation. A better way to to write this question with more clarity would be what actions should Facebook take to prevent vulnerable users from exposure to extremist propaganda? So in this case, we clarify who is is likely to be harmed, what kind of harm it is and who we're talking about. So Facebook, in this case, what kind of social networking site? And the answer to this question would be a list of actions. So it's sort of specifying what the shape of the answer should look like. Next is focused. So you could have a very unfocused question like what effect have anti climate change innovations had? And that is absolutely huge. What you want to do is narrow that down. You want to specify what is to be measured and how. So how will you measure this effect? What are the space and time boundaries to consider? Do we want to look just at the UK? Do we want to look at Europe? Do we want to look at cities specifically? That kind of thing. Also, what is the timeframe? What is the level of observation? Are we looking at anti climate change innovations? You know, what effect have they had within a home, within a city, within a country, within an industry? So a better way to phrase this, you know, something that's much more focused would be what effect have UK government green grants had on heat pump installation since 2008? So the thing here to measure would be how many heat pumps have been installed? And there's a time frame. So 2008, you might compare that to before 2008. And it specifies a country as well. And it also specifies an interaction. So we wouldn't necessarily look at heat pump installations for which no green grant was linked. So that's what focus means. Finally, concise. What measurable difference can be seen in pre and post COVID academic lockdown testing outcomes of human individuals between the ages of 13 and 19 when those individuals are domiciled within the country of Great Britain and Northern Ireland? I made it all the way through on one breath. However, it was a challenge and that should let you know that maybe your question is a bit wordy, it's a bit academic, it's a bit hard to follow. To put this in fewer words without losing any specificity or clarity, what differences can be measured between pre and post COVID student test scores for UK residents 13 to 19? So this is much easier to say in one breath but if you compare them, it means the same thing. So this is just a little warning not to try and sound academic. Often we try and sound academic because we see lots of examples of academic people being quite clever and we maybe are a bit nervous about sounding not as clever but being overly academic in your language makes it harder to understand. So remember, concise is valuable. Which of these do you find the most difficult, shall we say? Which do you struggle with clarity, with focus or with conciseness? I tend to focus pretty well because I always set myself really narrow goals and then I might expand if time allows. Concise I like, I find clarity to be the hardest because I never understand what other people are gonna interpret something as. It seems really obvious to me and other people are like, I have no idea what you're talking about. So for me, clarity. And it looks like we've got a pretty good spread here. Everyone finds different things challenging and certainly this might be a skill that you pick up over time. You might get better at something that you used to struggle with. Looks like a pretty good spread here which suggests that we're all talented in different ways. This is why I'm working together with others works out well because we can each lend our different talents. So those three, clarity, focus and conciseness, I find are absolutely essential for a good research question. However, there's also things that are beneficial or useful or laudable in good research questions but maybe not entirely essential. And those are things like novelty. Should your research question, should address a question or a problem that is not yet fully or sufficiently addressed. This I find is beneficial but not essential because exactly what counts as novelty is quite debatable. You might use a well-established method on a new population and some people might say that's not novel enough and other people say that's plenty novel. It's a bit tricky. Not all research has to be groundbreaking new amazing stuff. There's a lot of value to be gained from consistently doing good work, chipping away little bits at unknowns. Arguable is another one. Your question should not be answerable with a simple yes or no or with a simple repetition of well-known facts. Objective is a good one. You should not rely on good or bad or these kinds of judgment words. Maybe if the topic you're researching is fatalities or something like that, you could get away with arguing that fatalities are always bad and therefore we want to reduce them. But you should probably still in your research question talk about what you're measuring and whether you want to reduce that number or something like that. Not just we wanna reduce the bad things and appropriate. So not only should your question and answer should match your time and resources but it should also match each other. So you don't want to ask a qualitative question and answer it with a bunch of statistical numbers. That doesn't quite match up. That said, if you have a mixed question you need more than one kind of answer and you can certainly give descriptive statistics as part of a qualitative answer but just be aware that you don't wanna ask a question and then give an answer that people can't really relate to the question. So given these desirable but not always essential features, which of these do you consider to be essential for your work or maybe that you think everybody should maybe get involved in this? These are pretty much essential for everyone according to your view on science. Okay, we got a good distribution so far. Somebody's gotta come in for objective. No. All right. So yeah, these will apply more or less to some fields of research or to some research methods more than others. And it's clear that we need to be open-minded about this. Not everyone is going to appreciate novelty or our ability if the work they're addressing is actually filling in tiny little gaps in a generally well-known field but there's a few pockets in which this method has not been applied to a particular population or to a particular question. Okay, so it looks pretty good distribution there, I think. And at this point, I'm going to pass it mostly back to Nigel but I will be chipping in as needed. We're gonna work through a few questions submitted by our attendees today. And I believe some of these attendees are present today which is great. You can feel free to answer some of the questions if something isn't clear, if you wanna clarify. You.