 Thank you everyone for joining. My name is Dr. Jules Kazmeier and I will be leading the session today with my colleague Nigel. We both have in the past led sessions on developing research questions or beginning the research cycle. And we have found it is not uncommon that people get to a certain part in their research career and realize maybe they don't exactly know what a research question is and they feel a bit stuck. So we're hopefully gonna help people with that sort of disentangle some of the issues and some practical advice on how to write good research questions. So moving forward to our table of contents. First we're going to introduce ourselves, talk about some research, talk about some types of questions, developed some examples of research questions. We are not doing a breakout because we have too many people attending this session today. We had, yeah, it was a bit oversubscribed and we thought breakouts wouldn't work in such a large group. So instead we'll demonstrate several more examples and then we'll give extra time for the Q and A at the end. So moving into introducing ourselves, I'll start with Nigel. Okay, so I suppose as a researcher, my focus is on areas of substantive interest, particularly on housing, race and migration. And I'm thinking about how I've arrived at that. I'm quite a late comma to the academy. I started studying again in 2012. So I suggest the starting point is to think about what's the focus of your research, who your research subjects are and the kind of theoretical and policy frameworks that might be applicable. So for some of us, the theoretical framework is applicable for me, the policy framework because it's about housing is quite a strong component. And then I will go on to talk about the research cycle and the kind of, I suppose I'm gonna problematize it a bit by saying how messy it is, but I'm gonna broadly talk about that and how that relates to the research question. Thank you very much. Let's move to my slides. Hi, I'm Jules. I wanna tell you a little bit about my background. I started in linguistics and then I moved to philosophy, psychology and language and then to complex adaptive systems. And now I'm a computational social science researcher with the UK data service. And you think computational social science never heard of it, mate, I understand. It is using new computational methods or computationally intensive methods to address new social science research questions, questions that can't be addressed at all without computational means or old classic social science research questions but addressing them in new ways that were not previously available. And a little bit more about my approach to the research cycle is I have an eight step sort of, it's an iterative looping kind of cycle, but more or less you march through the steps sometimes going back and then forward again. The first three in which you identify the problem, explore the problem and formalize the concepts, I think are the most relevant to developing the research questions. And if you are interested in knowing more about my eight step research cycle, you can sign up for my Becoming a Computational Social Scientist Workshop. These happen twice a year in the UK data service. You can find, and maybe Jules can share the link to the next one in the Zoom chat. So thank you very much for sitting patiently through our introductions and now on to the next interaction. We want to know what topics you are working on. Autism or urban regeneration, very cool. I did a project on that. Aging also cool. Gender, mental health, children, crowdfunding. Oh, we got some great, great options here. Some good stuff. Maths, anxiety, pensions, genetics. Wow, this is filling up fast. You guys are amazing. Intercultural education, design, nurses, health and well-being, migrants, waste plastic, fantastic. Well, we've got clearly very diverse topics that we're all working on. So hopefully with all of the different examples that we're going to show you about research questions and all of the different features of research questions, you will find things that apply to your particular topic because it can be a little bit hard to understand how abstract discussions about research questions might apply to your particular field of interest. But hopefully we'll find something that helps you out and also there's plenty of time at the end for Q&A. So next, I am going to pass it back to Nigel, who has some explanation of the research cycle and how messy it is. OK, so just thinking about the research cycle, I suppose I'm fairly much a traditionalist and that comes probably from teaching on the graduates for a few years. So I would always advocate understanding what we know already. And that might be looking at academic literature. It might also be looking at policy literature and policy papers. And from there, beginning to think about what we want to know and then thinking about the practicality of how to get to that. So what data and methods will we use to find that out? And that may take us back into iteration because of what isn't available or what is available. We might either expand our questions because there's more data than we think that we could get hold of or reduce the scope because there's less. As we begin to look at that data and pull together some analysis, we might identify new topics. So for example, I did my PhD on housing focusing on the private rental sector. And I didn't include race as a characteristic. Though there's lots of evidence to say it's quite important. So it emerged. So my research questions then developed to saying what are the characteristics of people who are living in the private rental sector and describing those and age and race became quite significant factors in that. As we begin to produce outputs, similarly we might find there's a hole in what we're saying and we need to return and go back. And I think one of the important things to say to most of you at least is you're doing this in a relationship already an existing relationship. If you're studying with a supervisor and they're at the center of these discussions. I mean, we are giving you a brief input which hopefully informs you and how you take on those discussions. But the key people are you and your supervisor and how you manage that relationship and how you work your way through this research cycle. I think it's probably fair to say that most people doing the search never really get to the end of what they want to achieve. Our ambition is always a bit higher than what we achieve. So there is a kind of pragmatic element to this research cycle about where we draw a line and say, right, we've got enough. That's enough for me to get the bit of paper I want or to produce the report I want or to get paid for this piece of research I've done. So I'm just gonna go through some practical examples. So there's a research center called the Center on the Dynamics of Ethnicity and they made a proposal and got funding to carry out a survey on evidence on ethnic inequalities and produced a free e-book which you can access if you just search for evens. We also hold the dataset, but it asks these three big questions. So what would a racially just society look like? How close is Britain to being a racially just society and has the COVID-19 pandemic taken Britain further away from racial justice and ethnic inequality, and that's a kind of quite broad topic area, set of topic areas within the analysis, they went into different aspects of living. So health, well-being, mental health, housing, education, et cetera, et cetera, employment skills. To take that into a kind of policy context, this is a statement by Manchester City Council and it's really about saying because we know more now because of the census data, we want to understand better what's going on in our communities and there's kind of still time in the social community cohesion, which is another debate altogether. But in effect, they pull together all of the evidence they have so the demographic evidence they can get from the census to understand their population and then the things they learn through the different types of engagements they go on. So if you were going to say this, this is quite an open set of questions to start with, saying, well, what is our population like? What things are they facing? But then it moves from there into actions and commissioning services and policies that link to addressing kind of what they find in terms of fundamental inequalities within their citizens. So we'll go on to another one. So this is a piece of work I did with Sue Luke from London and Nisa, who's now at St. Pandora's. And what we were looking at was housing disadvantage faced by migrants and ethnic minorities. I suppose the kind of rationale here was that housing studies is not great at looking at issues facing people because of migration and race. And migration and race are quite messy concepts in that my mother was a migrant for a while. I don't know if she came to a point in her life where she wasn't a migrant anymore. So therefore I'm the child of a migrant and I'm from an ethnic minority. But in the way I'm treated in certain settings, those definitions don't mean anything. I might be treated as a migrant by people because of the way I look. I might be treated as an ethnic minority because of the way I look. So there was a problem in there that we wanted to articulate the need for further focus and particularly a focus around this kind of messy interface between migration and race. And we also looked at the legal policy and market forces that shape those, how they developed over time and how they're manifested both nationally and locally. So in this example, there's not a specific research question, but I think I frame the questions we were looking at. The rationale for us doing it was we thought this area was under-researched and the kind of questions we wanted to look at. How has this come about through the legal policy and market forces? How that's changed over time and how that operates differently at different scales? So if you think of housing, it is basically a local policy function. So it operates quite differently in different places. And how people respond to them locally is a great workshop. I think the most interesting person for me there was probably the Bishop of Manchester who is a strong advocate of social housing. And he basically just talked about his mailbox and what mail he got which highlighted the issues that kind of framed a lot of what we learned from that element of it. So we looked at historical policy. We did some statistical analysis of census microdata and facilitated that workshop with academics, key kind of stakeholders in the housing field and voluntary sector organizations. So people like Joseph Browntree were there, et cetera. So if we move on, I'm just going to show you an output from that which was around the policy and legal challenges. So what this was trying to show was how things have changed over time. So in the middle of the graph is net migration and it's got immigration and emigration. So that frames that timescale which was from 1964 to about 2016. We then looked at the kind of legal changes that had happened over that time in terms of immigration. So the way that immigration controls have kind of operated. So the first kind of example we came across is the 1905 Aliens Act which was excluding Jews and then transferred to enemy aliens as well. That was repealed in 1919 and we begin to see them acts associated I suppose with elements of racism about feeling the country has been swamped. So from a period in which external borders were created to the current period in which we have internal borders and for those of us in universities we can see the internal border operating. So in my building I walk in and there's a sign saying visa control over here. So our internal border operates in my building. It has been in other universities I've worked in as well. I kind of struggle with it but we have our different points of view. The second thing that came along that we looked at was citizenship. And this is a period where people like my mother were citizens of the British Empire. They were born into a British colony and inherited that status. When she came to Britain she got naturalised so she then became British. But us as children of those generations of migrants inherited British citizenship by being born in that period. But from 1984 onwards those rights are not automatic and we've seen more recently how those rights can be taken away from people who were born in Britain in the case of the Bengali girl who went to Syria. And then the final part of the legal framework was the way that housing has changed and particularly the way we've moved from a process where housing kind of legislation really focused on housing as a right moving to a focus where it became much more about the market and the asset of housing. So in particular the 1988 act I think took away security of Kenya and rent regulation. So we're now in this scenario where excuse me where many of you are possibly renting and facing significant challenges in sustaining your livelihoods in the face of the way that rents are behaving. And then the final bit of that is to put in some things around different acts and different events about which groups came in, which organisations were working. So the 1976 Commission for Race Equality which framed a lot of the way that housing was shaped and housing became part of the remit of that through to the Equality and Human Rights Act and then patterns of migration. So that's the kind of piece of work that we did and yeah go on move on. Another piece of work I did was looking at thinking about policy. So I've worked for quite a long time with an organisation called the Race Equality Foundation in London on housing stock and they work with the housing learning improvement network and what they wanted to do was to see how aging is going to affect different ethnic minority groups. So the first thing we did was to think about the demographic profile using census data to see what the growth was and how the migration groups tend to be younger when they arrive the kind of pattern of migration is around the kind of late teens through to early 30s. So you have a much younger population when you have a group of new migrants coming in. So that was particularly true of the Commonwealth migration from the Caribbean, from Africa and from India in the period of the 50s to the 60s, 40s and more recently that's been true of other groups coming to Britain so patterns of EU migration etc. So looking at how those will contribute to a growing older population looking at housing deprivation of those groups by age and thinking about why there are higher levels of housing deprivation experienced by minority groups that pattern is still fairly sustained to think about care and residential homes and to think about what those demographic changes mean for future demand and then to look at some of the geography of that concentration which is quite different to the geography of older people more generally so if you look at older people across Britain there is a tendency to retire out of cities to move out of cities and older populations tend to live in areas like coastal and rural areas whereas for a lot of minority groups that pattern isn't the same This one's me so I wanted to explain a different kind of research this is a project that I've been working on in fact that I'm hoping to submit for publication next week if I can write the conclusions The research question underpinning this research was what differences can we find in the way human geneticists use person first and identity first language so an example of person first language is person with autism or child with ASD or something like that and identity first language would be autistic person or diabetic children or something like that so we looked at all of the abstracts submitted to the European Conference on Human Genetics between 2001 and 2021 that ended up being almost 30,000 abstracts and we looked specifically in the context of autism because we wanted to narrow the focus down so not just person first language and identity first language but person first and identity first language around autism and we wanted to see how the use changed over time how the nouns used were different so whether it was person or child or cohort or something like that and we wanted to look at abstracts that used both kinds of language in a single abstract and we did this through text mining and natural language processing methods so this is a much more computationally intensive method as is what you might expect from my research focus and our conclusions were that roughly the same number of examples of person first and identity first language in relative to autism in the articles in the abstracts more or less as popular as sort of up going up and down more or less over time until 2019 in which they started one started being much more popular and then the next year reversed and then the year after that reversed again so they just instead of going together indicating general popularity of autism as a topic the language started really changing and that was interesting we also found there were fewer different nouns used in person first language most always people nouns so boy, child, sibling family, things like that whereas the identity first language used a greater variety of nouns and there were much more science nouns things like cohort or population or subject we also found that about 20% of the abstracts that had at least one example of person first or identity first language around autism 20% of those used both patterns which is interesting it shows that people probably are using them more or less interchangeably or that they were using them in some ways like they might use identity first language for the cohort and then person first language for an individual so this does answer our original research question but then it motivates other questions like what happened in 2019 when the pattern started changing or can we drill down into those patterns in the abstracts that use both to see which one is like how they're being used a bit more of the context and that might involve a lot more reading rather than computationally intensive approaches but at least we know which 20% of the abstracts to read manually instead of trying to find them out of the original 40,000 which does not sound fun okay so we've given several different examples here with a couple of different kinds of research method and we do want to point out that some types of research namely like empirical or empirical styles of research will have hypotheses and null hypotheses so for example my question about whether identity first or person first language is more common in this set of abstracts we could answer that with a hypothesis in a null hypothesis other things like causative kinds of research or longitudinal research are less likely to have these kind of strict hypotheses approach they will also have different kinds of methods potentially different kinds of data Nigel did you want to say any more about this I suppose we're presenting ideal types and kind of looking at what people have submitted we are doing people are doing quite complex things and many of you at the beginning of that journey so I would say I'm a kind of mixed methods researcher that I get paid for being a quantitative researcher yeah yeah it's lovely to idealize these kinds of research but they often come out much messier and squishier and I think for many of us being open to the kind of methods to find the data we want is a kind of useful attitude to adopt so I wouldn't be frightened by the complexity of some of these if they help answer your question and you find that other people have been doing them they may well be things that you would want to engage with more yeah I agree I find that the research question the methods to use and the data to use all kind of inform each other they all have to play nice together and you might end up changing your question if you are not able to get the data that you wanted and you get something else instead or you might change your method or both or it all keeps going around everything's changing every time you look at it try and avoid that if you can so we've got another interaction for you in this time it's about methods we'll have another word cloud so pop in the kind of methods you use or that you want to use or that you typically use or that you absolutely hate using I don't know I mean it's up to you talk to us about methods cognitive interviews web surveys alright we've got comparative case studies I have to say no one so far is using text binding and natural language processing I feel deeply hurt just kidding focus groups semi-structured interviews we've got quite a variety observations there as well quite a variety of methods here and each of these will lend themselves to creating different kinds of data or to using different kinds of data and that will relate to the research questions that you can answer or that are a good match for these kind of methods so you're not going to enter you're going to struggle to answer certain kinds of you know empirical questions with hypotheses and all hypotheses with semi-structured interviews unless the questions are really basic like do these people talk about these things which is not a very interesting research question okay got some great variety here and thanks everybody data scraping got a computational method in there alright moving on we are now going to talk about actually how you develop the research questions and this is a little bit some focus on features that your good research question should have and I do want to point out this is a good research question that is the end of the research question process so you start with a bad research question and you make it good by the end of the process by checking for example is it clear is it a question you straightforward grammar language appropriate for the audience is it a sensible question is it focused in that it matches the time resources and data to which the researcher has access and is it concise is it expressed in the fewest words needed not fewest words possible so let's go into these in detail clarity for example you might start with a question like how should networking sites address the harm they cause unfortunately this question doesn't specify what type of social networking sites or what kind of harm or who is harmed and it doesn't even necessarily support the the existence or the extent of that harm it's this question as it stands is not very clear it's too ambiguous it leaves too much room for interpretation a better way to to write this question with more clarity would be what actions should facebook take to prevent vulnerable users from exposure to extremist propaganda so in this case we clarify who is is likely to be harmed what kind of harm it is and who we're talking about so facebook in this case what kind of no set social networking site and the answer to this question would be a list of actions so it's sort of specifying what the shape of the answer should look like next is focused so you could have a very unfocused question like what effect have anti climate change innovations hand and that is absolutely huge what you want to do is narrow that down you want to specify what is to be measured in how so how will you measure this effect what are the space and time boundaries to consider do we want to look just at the UK do we want to look at Europe do we want to look at cities specifically that kind of thing also what is the time frame what is the level of observation are we looking at anti climate change innovations you know what effect have they had within a home within a city within a country within an industry so a better way to phrase this you know something that's much more focused would be what effect have UK government green grants had on heat pump installation since 2008 so the thing here to measure would be how many heat pumps have been installed and there's a time frame so 2008 you might compare that to before 2008 and it specifies a country as well and it also specifies an interaction so we wouldn't necessarily look at heat pump installations for which no green grant was linked so that's what focus means finally concise what measurable difference can be seen in pre and post COVID academic lockdown testing outcomes of human individuals between the ages of 13 and 19 when those individuals are domiciled within the country of Great Britain and Northern Ireland I made it all the way through on one breath however it was a challenge and that should let you know that maybe your question is a bit wordy it's a bit academic it's a bit hard to follow to put this in fewer words without using any specificity or clarity what differences can be measured between pre and post COVID student test scores for UK residents 13 to 19 so this is much easier to say on one breath but if you compare them it means the same thing so this is just a little warning not to try and sound academic often we try and sound academic because we see lots of examples of academic people being quite clever and we maybe are a bit nervous about sounding not as clever but being overly academic in your language makes it harder to understand so remember concise is valuable which of these do you find the most difficult shall we say which do you struggle with clarity with focus or with conciseness I tend to focus pretty well because I always set myself really narrow goals and then I might expand if time allows concise I like I find clarity to be the hardest because I never understand what other people are going to interpret something as it seems really obvious to me and other people are like I have no idea what you're talking about so for me clarity and it looks like we've got a pretty good spread here everyone finds different things challenging and certainly this might be a skill that you pick up over time you might get better at something that you use to struggle with looks like a pretty good spread here which suggests that we're all talented in different ways this is why I'm working together with others works out well because we can each lend our different talents so those three clarity focus and conciseness I find are absolutely essential for a good research question however there's also things that are beneficial or useful or laudable in good research questions but maybe not entirely essential and those are things like novelty should your research question should address a question or a problem that is not yet fully or sufficiently addressed this I find is beneficial but not essential because exactly what counts as novelty is quite debatable you might use a well established method on a new population and some people might say that's not novel enough and other people say that's plenty novel it's a bit tricky not all research has to be groundbreaking new amazing stuff there's a lot of value to be gained from consistently doing good work chipping away little bits at unknowns arguable is another one your question should not be answerable with a simple yes or no or with a simple repetition of well-known facts objective is a good one you should not rely on good or bad or these kinds of judgment words maybe if the topic you're researching is fatalities or something like that you could get away with arguing that fatalities are always bad and therefore we want to reduce them but you should probably still in your research question talk about what you're measuring whether you want to reduce that number or something like that not just we want to reduce the bad things and appropriate not only should your question and answer should match your time and resources but it should also match each other so you don't want to ask a qualitative question and answer it with a bunch of like statistical numbers that doesn't quite match up that said if you have a mixed question you need more than one kind of answer and you can certainly give descriptive statistics as part of a qualitative answer but just be aware that you don't want to ask a question and then give an answer that people can't really relate to the question so given these desirable but not always essential features which of these do you consider to be essential for your work or maybe that you think everybody should maybe get involved in this these are pretty much essential for everyone according to your view on science okay we got a good distribution so far somebody's got to come in for objective no alright so yeah these will apply more or less to some fields of research or to some research methods more than others and it's clear that we need to be open minded about this not everyone is going to appreciate novelty or arguably if the work they're addressing is actually filling in tiny little gaps in a generally well known field but there's a few pockets in which this method has not been applied to a particular population or to a particular question