 Hi everyone, my name is Louise Kaepernar and today I'll be leading the Computational Social Science Introductory Workshop. Okay, so throughout this workshop we're going to be covering three main sections which include what the first one is what is up with computational social science so what does this even mean? Then we're going to discuss a bit about how you actually become a computational social scientist and finally we'll cover the eight steps of CSS which is where we'll have the most interactive part of the workshop and where you can really get stuck in sketching out your own CSS project and of course at the end there'll be time for final thoughts and any questions that you might have. What is computational social science? It's the use of computational and empirical methods to address social science questions so we can break this down a little bit more. So computational social science requires human thinking to identify important research questions and because we're going to be dealing with social science questions we need to understand how people behave and what they want and what they want to achieve so we need that social science brain in order to formulate those really interesting research questions. As you can see we also need a little sprinkle of computer thinking and that's so that we can turn these research questions into computational or empirical methods and after you've completed your research you're going to need that social science human thinking again to effectively communicate those results to other people. So another way to understand computational social science is to think about what its main components are so let's just go through a few points. So computational social science is not just using computers within a social science research project you know I'm sure you've all sent an email but that doesn't necessarily mean that you're doing computational social science. It's also not just using digital versions of purely traditional social science methods so it's not just asking for survey participants to complete an online form instead of handing one out so it's not just a digital version of something that you would normally do in a social science. And it's not just using digital but purely non-empirical methods. So I'll go ahead and I'm going to give you a few examples that will kind of untangle these ideas a bit more because I know they're not always entirely obvious. So here's some example that might help put this into context a bit. So one example of a CSS project would be collecting, processing and analysing millions of online news articles to show changing political attitudes. So you can see how we have that human thinking here to formulate our social science research question about political attitudes but we also have that computer thinking inherent in using a computational method so in this case it would be web scraping and that's what we use to gather a massive amount of news articles. We could also use real-time weather and traffic data to show how travelers react to events. So for example maybe you hear that there's a storm that has caused damage to a local town and you want to look at how people react and how that event is dealt with in real time. So you can see how this project isn't just a case of using a computer with a social science project or using a digital version of traditional social science methods instead we have these uniquely computational methods and we have some other examples which include combining data from novel wearables or apps to establish a correlation between social media activity and heart rate. So that could be aiming to answer a social science question about how people feel about certain images so you could look at how they react on a positive or negative feelings, are they irritated and so on. Finally we could also be interested in mapping family names over time by importing, processing and formatting centuries of parish records. That could allow you to explore the movement of certain families to different areas or you could maybe look out the changes in family size or whether families have moved away or not. So these are just a few kinds of broad but really CSS examples. Moving on there are key factors that make a CSS project computational and the first one of these is data volume, complexity, speed, difficulty or novelty and this is more important than the exact data source or type. So in our previous example where we had parish records the source of data is not entirely important it's about the volume and the complexity and you know that difficulty or the novelty that data. Additionally another key factor like I've said is that the data must pertain to people, actions, behaviors, choices and statements. That's that key social science ingredient that we'll need and just to hone in on that point the research question should be a social science research question which uses that atypical data to talk about how people make decisions or what influences their behavior and choices so the exact research question is not important but it must be a social science question and that's where we have this really obvious intersection between the computation and the social science and that's what we're going to focus on today. And here we have a really important quote so in essence CSS is an opportunity to do socially valuable research that would not be possible without computational methods and tools. So by this I mean that we couldn't for example manually scan years of police recorded statistics to try and understand how crime rates have changed over the last 10 years this physically wouldn't be possible to do manually but with the use of computers we can apply advanced statistics and models in order to understand that change in crime rates. We could look at how we could count many types of crimes that have taken place in certain areas and then we could aggregate these crimes to map them spatially. We could look at exploring the long-term trend seasonality or the noise components of these different crime types and this type of research just wouldn't be possible without the use of those computational methods. Another example could be as I mentioned before web scraping millions of online articles and you could be doing that to try and understand how political opinions have changed over 20 years. In order to get a sense of political opinions we'd want to also examine you know the words and the articles and how many words belong to different categories and what sort of themes appear and we could look at the proportions of these words and how that changes over time. So that's something that we wouldn't be able to do without computational methods like web scraping and natural language processing. So in order to sort of see if any of that made sense we're going to have a little bit more interaction now and I'm going to give you guys the chance to vote on whether you think a given project is a computational social science project or not. So if you want to head back over to Mentimeter we can get this set up and then you can start voting. So I'm just going to switch to Mentimeter now. So if you guys want to head over there maybe Emma or Nadia could pop the code back in the chat for anyone who's joined late if you head over to menti.com. All right so yeah it seems like already got people voting but is this CSS or not? So the example is scanning historical recipes and using AI algorithms to recognise text to identify that should be ingredients sorry and measures used over time. That should say to identify ingredients then measure used over time. So we've got a lot of votes coming in. Seems like the majority have said that this is definitely CSS and we have some saying there's not enough social science very small and aren't saying there's not enough computation. So in my opinion I'd say that this project does sound quite computational because we're talking about the use of AI algorithms to scan recipes. So we definitely have those computational methods being used here. But I suppose the 27% that saying there's not enough social science I could see how you come to that conclusion because you know this one sentence here doesn't seem to indicate that we're asking any questions about people's behaviours and choices. So I mean personally I'd probably say that this is CSS but I do agree with people that saying it's not because we don't really have enough information and it's not clear exactly what the social science question is here. But you know maybe we could be talking about how imported foreign foods integrate into domestic scenes and how people then take up. Maybe it could relate to how wealthy people then start adopting you know fancy foreign foods into their diet. It could be exploring how that trickles down and you know sort of like foreign foods and ingredients then become more normalised. So you could see how that would be a social science question but it also could just be looking to talk about the proportions of natural foods that people are eating over time and maybe that would be more of a biology question. So yeah it's kind of one of those annoying ones where there's not really a right or wrong answer but this is just to help us you know explore different methods in CSS but me personally I'd probably say it does look CSS to me. So let's move on to the next example. So is this CSS or not using gamified smart home displays to understand how people interact with energy saving technologies? What do we reckon? All right I'll just let some votes flow in. We've got some people saying they need more information to decide absolutely fair enough. Again it looks like definitely CSS has it in this one so we have some people saying there's not enough computation. Personally I'd say that there probably is you've got this gamified smart home display and that's going to capture a lot of digital data and it's looking at how people interact with it so it might be tracking you know what time of day it is that they interact with it maybe who in the family is interacting with it. So it's possibly got some sort of tracking data and that would in my opinion make it fall under CSS. Maybe the purpose of it as well is going to look at changing people's behavior to make them engage more with energy saving technologies. That would definitely pose some sort of social science question. So yeah in this one I'd probably agree it looks like it's CSS we've got that looking at how people interact with them. I can also see how people have come to other conclusions with that one. So let's move on. Is this CSS or not? We're looking at advertising for survey participation on social media and once we get those responses we're going to store them in a database. Is this CSS or not? So you could think as well back to that slide where we talked about what CSS is not. So it's not just and then I had a few points of you know what makes something CSS or not. So let's see what here's only to actually open the voting on this. That's my bad. Okay. So it seems we've got the majority saying that there's not enough computation here. So yeah I would also agree with that. So like I said if you remember on the previous side we talked about how CSS it's not just using digitized versions of traditional survey methods. There is no clear line between you know how much computation is enough computation but you have to ask yourself if you can do your research without digital tools then it's probably not CSS. If you draw the conversation back to surveys in itself typically the larger the survey sample the more necessity there will be for computational tools because the analysis is going to become much harder but the use of online survey doesn't automatically make it a CSS project because you could probably collect roughly the same amount of responses through paper surveys if you would online might take you a bit longer but you know you'd be able to do that right and yeah we've got some people saying that they need more information to decide and that's why because like I said these are just one sentence right one sentence we don't really know where this project is going to go from here so perhaps it'll branch out into something more computational but yeah let's move on to the next one. So is this CSS or not? Reading in real-time weather and air pollution data to create complex models of hyper-local air quality what do we think? It looks like the majority have agreed that this is not CSS due to a lack of a social science focus so we'd have to think about you know is there a link between hyper-local air quality and human behavior from this statement it doesn't look like there's any social science sort of question going on here and we'd have to know more about the project are we looking for a political impact or is there going to be maybe a overlap or branching out into different sort of fields and schools so maybe this is going to branch into geoscience or physics so it's not clear from that statement that there's any you know intent to go down a social science route so I think with this I would have to agree the majority that there's not enough social science in it and finally we've got our last poll which is we're training a neural network on social media data to create a believable chatbot that counteracts online radicalization but straight out of the gate we've got a few people saying definitely CSS some people are saying there's not enough social science we need more information to decide I'll just let that go for another few seconds so personally I'd have to agree with the majority here and it seems to include all of the components of the CSS project and there seems to be an implicit social science research question as we're focusing on people's actions and attitudes and behaviors and we have that computational method which is going to help us explore this so we've got a neural network which has been trained on social media data but those of you that don't know what a neural network is these are kind of artificial intelligence that teaches computers to process data in a way that's inspired by the human brain so it's a type of machine learning process also called deep learning that uses interconnected nodes or neurons in a layered structure that resembles the human brain so basically what we can do is predict the user's behavior based on their posts and their chats from their social media accounts so there's also you know going to be a bit of web scraping involved all around very computational stuff the chatbot's probably going to have some natural language processing in it too so in my opinion I'd say definitely CSS so that was our last example and thanks everyone for taking part it's really interesting seeing what you guys think as well I'm going to just go ahead and switch the screen back now actually no I'm not we're going to have a little word cloud as well so you can send in you know three terms that tells what you've learned about CSS so maybe something that you found interesting something that surprised you or you know anything you want to send in big data sets yep so you think about scraping like you know millions of online articles you know the massive data set right and we talked about and you know the fact that we're working with like big data high dimensional data social movement yep so we've got that social science component socially valuable yep social scientists have got to think about you know often have to justify their research on the grounds it's going to you know be valuable for society human behavior expectations transdisciplinary yeah for sure working with different fields exciting yeah I'd say so new data analyzing behavior but yeah it seems like we've got a lot of people saying big data complexity coding yep I'll give this a few more seconds see if it changes that much can be difficult to define yeah for sure yeah neural network valuable knowledge theory matters for sure programming social science genre yeah this is all really interesting guys vague space to research yeah we're gonna try and break it down a bit more um after we've got a little break all right I'm gonna stop that now and go back to the presentation thanks for that guys hopefully everyone can see and when they're back to the PowerPoint um I'm gonna give you guys a little five minute break um you know grab a brew and go for a walk not too long walk but get a drink of water whatever you need to do and then we'll meet back here in five minutes so let's see what time we're on okay so we're going to talk a bit about how to become okay computational social science it's now and first we're going to start by covering what a social scientist is so the first thing that we can say is that social scientists think like people and what I mean by this is that they use a lot of those human type thinking skills like abstraction inference they're going to really understand fuzzy sort of concepts and background knowledge and social scientists in general don't shy away from gray areas or overlapping categories these are just part and parcel of being a social scientist and it's no surprise that they're going to need those human type thinking skills because they're still being people interactions and behaviors and that's going to require a certain skill set but it is also important to know that social scientists build up a lot of data skills in the course of doing their research so if you think about things like response categorization encoding quality evaluation pattern detections and statistics these are all things that social scientists regularly do but whilst they will be using computers and statistical programs this often doesn't involve writing computer code you know you might be doing some analysis on sbss or stata but yeah like I said maybe not so much any programming or code and then we have computer scientists and we're making some big generalizations here but in contrast to that you know sort of human type thinking skills that social scientists have we can say that computer scientists think more like computers so what I mean by that is that the thinking skills that they have are more along the lines of concrete definitions and absolutes they think in terms of strict hierarchies and categories and clearly defined and stoked variables and rules and in terms of data skills computer scientists collect analyze and manipulate data through programming scripts computational methods and technological tools but unlike social scientists they might not be as familiar with identifying or motivating research projects on the basis of societal impact or value so those of you watching that are social scientists you might have had to justify your research on the ground that will make the world a better place so you know this could be well I'm researching radicalization in order to produce insights that could lead to countermeasures whereas a computer scientist they might be more focused on a sort of logical justification for a particular project and that could be I want this algorithm to be more efficient so that it uses less memory so we can kind of see the differences here and obviously this is very broad so in order to undertake the computational social science project you're going to need a blend of those social science and computer science skills you'll need that sort of human thinking that I've mentioned you know the ability to understand gray areas and quality concepts abstraction inference you're going to need that computer thinking so you're going to want to be able to translate that research question into the computational method so you might have to have that more you know programming sort of mindset more of a logical justification for why you're doing things you're going to need open-mindedness as well we'll talk a bit more about that and you'll need a mixed problem to tackle so let's have a look at each of these four things in a bit more detail so yeah you'll need those skills that we briefly touched on before and those are things like being able to identify important problems or knowledge gaps you're going to want to be able to consider possible solutions and you'll need to be able to collect connect problems to relevant theories or perspectives and you want to be able to collect relevant information and research to bring your approach and these are all things that social scientists really excel at the ability to understand context and nuanced perspectives and how to communicate abstract ideas and how to attack a research question and that's not to say that computer scientists can't do this or that the robots I'm just talking in broad generalizations there about the differences between the two fields whereas computer scientists might find this a bit more difficult they might be more used to those concrete definitions and absolutes rather than you know these sort of gray areas or murky social science concepts so that human thinking is going to be the first thing that you need and now we're going to cover why you also need these computer thinking skills so what do you need you'll need the ability to access organized process and handle vast or complex data you're going to want to learn how to write collaborative code and also how to properly document your workflow which is often a step that a lot of people do neglect these are going to be skills that computer scientists might find quite easy as well as data scientists who've been trained in computational methods but for social scientists it can be much harder to transition towards these computer thinking skills but like I said in a few slides ago social scientists do have the data skills that they can build upon and these are things that I've mentioned before like coding responses pattern detection and statistics format in your surveys and this is a really important ingredient I can tell you it is someone that has moved away from a social science field to doing more CSS type so it can seem pretty intimidating at first you might just have that perception of oh I don't have a code in brain you know and that's why it's good to remember that no one's going to start out with all of the skills that that they need more are you going to know all of the skills that you might need to acquire and that happens quite a lot you might start off by saying I want to scrape tweets for information on the 2016 US election and you're going to expect all right well I'll need some coding ability right but you might not know that that's going to entail learning about an API or different file formats or ways that you're going to visualize the data but if you approach CSS with an open mind and a willingness to learn you're going to be able to gain those skills along the way and you'll also start to understand that some skills you'll also start to understand that some skills will have a steeper learning curve than others so it might be fairly simple to learn how to do a little bit of web scraping maybe but learning to build your own neural network and then train it on a large amount of data that's going to be you know much more complex and another really important thing to know is that you can't do it all by yourself you're going to need to be prepared to collaborate so you can look at collaborating with people from other fields and that's where you start to get this bridge between the social science and computing world a social scientists learn more about computing and vice versa we begin to see more conversation and collaboration between these two fields and that produces really exciting and interesting research and what you'll find is that you don't need to know everything about computer science what you want to know is enough to have those productive collaborative conversations with others in the field finally what you're going to need is a problem that requires these CSS skills and that's a mixed problem so one that requires that blend of human thinking and computer thinking and some of you might be here because you've already encountered one of those problems and that won't surprise me because as resources become more digitized these problems are going to become more relevant we've got the fact that large volumes of data are being made available and updated faster than ever so if we think about maybe a classic social science problem maybe we're interested in how men and women move through cities differently or maybe we want to narrow down that question to look at how people with disabilities navigate seas traditionally for this kind of social science problem you might station some interviewers in different places to spot people as they go past right or you might want to count how many people go by that are maybe using nobility aids or maybe you're going to send out some service to some people's houses but now there's much more opportunity for us to gather that large amount of data with these novel computational methods so you could collect data from public transport networks about how many people bought tickets or how many people swipe their card at the tram stop you can't even get sensors which can track how many cars go past a given point you could use AI algorithms and CCTV cameras to identify how many people are moving through space and at what speed so as you can see there are all these new ways of approaching traditional social science questions that there's also no reason why you have to abandon those traditional methods completely right so part of good research is going to be evaluating different methods and then comparing outcomes and it might be interesting to see whether by using those different methods you can then get different answers and then if you do get different answers then you can ask well why is that which then may prompt further questions and so on so there is a massive amount of benefit with building upon you know building up some computational skills and then strengthening those social science skills that you might already have so what I want to do here is quickly introduce an eight step process for how to undertake your CSS project so these steps are going to include first identifying the problem exploring the problems formalizing your concept collecting data implement in the software and then verifying those concepts and then using those concepts to experiment or analyze the data discussing your findings present in a conclusion then communicating publishing and presenting as well as sharing your findings and documenting and validating your findings to make this process more useful to you I want you to start thinking about either a project that you want to tackle or a research idea that you've been thinking about or it can't even be a project that you've done in the past you can jot this idea down or maybe even put it in a chat and just have it in mind as we go through these steps so once you've identified the problem you're going to want to be as clear and specific as possible about the patterned problem or the lack of insight that you're going to be covering right you're going to want to identify who is involved where it is etc and that's going to help you to define your recent issue so maybe you have a goal in mind let's say we want to get more people traveling actively through the city centers we want less cars on the road and we want more people riding their bikes or you know scooters or whatever or just being able to get from A to B in their wheelchair so the research question might be what are the barriers to active travel in city sentence and what you need to do then is identify who is involved so you can start to list and who is involved whether that be companies people or different demographics that might be of interest to you so maybe you'd want to look at city councils, bus companies, different businesses and vehicles and it's better to just go all out with these lists as well and that's going to give you different avenues to explore and you can always just you know go back and cross out any that you've done a bit more investigation into and started that they're not actually relevant so that's going to be how you can really hone in on this first step and the next step is going to be it's going to involve exploring the problem and this is where you'll gather information and perspectives in multiple ways so you might do some preliminary sort of surveys observations secondary data analysis, web scraping, access and APIs so this might also you know it might involve conducting a few interviews with people of interest you could interview if we go with our example from before maybe we'll interview the manager for the city's transport network or some local council workers but you would also probably need a survey or observations or secondary data analysis to capture how many people are actually moving through the city centre so it's about using those different methods and tools to further enhance your understanding of the problem or research topic you'll also want to spell out sub-problems, processes, relationships, simplifications, assumptions etc so after settling on your main research question you're going to need to get more specific in order to make that question relevant and measurable so if what are the barriers to active travel in the city centre is your main question you might specifically be focusing on what are the barriers to active travel through this specific city centre at this specific town of day given the way that these specific roads are laid out right so that's where this is the step where you really nail down the particulars of your research question moving on to step three you'll need to formalize your concepts and what I mean by this is you'll want to make all the concepts and processes explicit formal and both computer and human understandable so oftentimes we use something known as pseudo code but you don't need to know how to write code and what you do need to know is just start understanding how to formalize things when it comes to transitioning from that research question and then using those computational methods so maybe you have a research question which focuses on trust trust is a very social sciences sort of concept and if your goal is to get a computer to be able to measure it or model it or represent it in you know a simulation you're going to have to define it in a way that a computer would understand so maybe you will define trust as a variable between zero and a hundred then you're going to need to make rules about how that variable will change in certain situations so maybe if you're looking at the way two parties interact maybe if they interact positively that trust increases and then given a negative interaction where one of the parties is judged to be deceitful maybe that level of trust then declines or resets to zero so you have to start thinking about how to formalize concepts in your research question so that a computer would be able to understand it and then what on to step four so actually what I'm going to do now um because I should have done it three steps earlier is just get some idea of what you guys are thinking about and what you're jotting down so what I'm going to do now is if you could head over to Mentimeter um sorry I shouldn't film this at these slides earlier I'm just going to get a bit of idea about what are you first in your second step so what kind of ideas are you thinking about so I'll just stop sharing for now and we'll head back over to Mentimeter so hopefully you guys can see this yeah I do apologize should have done this a bit earlier but just maybe tell me a bit about your step four and two so what research questions are you thinking about how would you think about exploring that problem hopefully you guys will be able to type in anything that you want to tell us a bit about what your ideas are what you'd research so we've got a few answers coming in all right sweet so we've got seven answers I will find out how I can actually access them someone's also just said can we have access to the recording of the session because they've joined late yep um this will all be up on youtube after so no worries try to complete view results um there's my banner maybe hiding it at the top maybe it's because I'm sharing my screen I can't see that unfortunately I can't share it in full screen mode I'm really sorry about this I guess it just goes to show that I'm not computational enough to be using Mentimeter but let's have a look at what some of you guys have thought hopefully you guys can't see this I will just read it out if you can't we've got identify the problem and who it is that's involved in it yep nice one um nothing specific at present but your step methodology is really useful oh nice one glad that you think so defining terms so they are mutually exclusive assigning weights or numbering the terms yeah yeah definitely important you want to translate that social science into something that your computer can be able to understand we've got someone that's looking at perception difference between different refugee and immigrant groups that sounds really interesting be a good one for um maybe some potential web scraping as well phd study offer gaps beginning to think it's not something that would work as a css project um I don't know maybe maybe talk to me about at the end I'd like to hear more about that um looking at the change in attitudes over time about non-english vegan immigrants again that would be a really good one to explore with some web scraping you could scrape some tweets although you know it's an interesting one right because when you scrape it you've got to think well you're not getting that demographic is largely male it's going to be quite young as well so there are pros and cons to using these different methods uh analyze attitudes on immigration from social media yeah sweet so we've got like quite a bit on that um how gender is presented in magazines word scraping to find themes and most common words yeah that'd be a really cool one you can use a lot of their natural language processing for that that'll be super super interesting um we've got one that's slightly cut off implementing intervention to increase sense of belonging of unfortunately for me it's a little bit cut off um really screwed up with these um with this slide um someone said these 8 steps are excellent I'm a data librarian in Italy so assisting PhDs with their research design and data strategies thank you oh thanks that sounds yeah super interesting and no worries at all uh glad you found it useful so we've got some really really interesting research avenues here and it seems like you guys are really getting to grips with um kind of like how you formulate um a CSS project I'm gonna see hopefully um when I go back to this I can see the full um responses I am sorry about this guys um I have to change those questions um to a different format last minute but um I'll keep this in mind for the next time this is not the best way to do you so I'm gonna switch back now to the presentation and we'll continue through those 8 steps so we're now on to step 4 and what this involves is collecting data implementing software and verifying your process so you'll need to select and implement one or more methods many of you might have thought about these methods in your step 2 so um some of you decided you want to do a social media analysis you want to do some web scraping um so yeah you could pick any sort of method maybe you're looking at agent-based modelling um or sentiment analysis so this is a step where you implement these methods and make sure that they work in the way that you anticipated so maybe the data comes in a different format than you might have expected or maybe you're just getting a bunch of error messages in your code so we're looking at does it work in the way that we expect it to maybe if it doesn't you could use a toy data set so that's a smaller data set with less variables rather than your full data set while you fine change the method that's something I often end up doing I have a massive data set and I think I'm going to make it smaller because it's just so much easier to work with and see if your sort of computation and your any math that you're doing is actually um is actually correct and of course the choice of methods going to be highly dependent on the research topic and what you're going to need to do as well is thoroughly check that the selected method has been implemented correctly and that's what we mean when we talk about verifying your process or method it's about answering the question did we do the right thing oh sorry did we do the thing right important distinction that um and at step five you're going to then want to experiment and analyze the data so you will of course run the experiments build any models that you need analyze the data or otherwise use the method selected in the previous step and you'll need to identify and explain the results within the context of the experiments in the model or the method that you're using then once you've run your experiments and you've analyzed the data you can then start to interrogate the results and form some conclusions so this means going beyond the experiment in the model or the method that you've used to draw some conclusions about what the results mean so what sort of picture they're forming and does it support any policy recommendations so is the research indicating that there might be policy recommendation who or what do these results affect and why does it matter and what's the change what should change so and who benefits from that proposed change so if we use that example of looking at how people move through cities perhaps you've found that people with disabilities relate to mobility have more difficulty navigating through particular areas maybe there's uneven surfaces or particular roads that are too narrow in which case you can then recommend some changes wider footpaths in XYZ area right so we're nearing the end of the research process now where we're going to focus on communicating and sharing that research and in terms of communicating the research it's important to understand that you know all of the previous steps must be communicated to multiple audiences in multiple ways and what's really important is that you think about short-term and long-term engagement for a lot of us our mind will instantly go to well I want to get this published in an academic journal which obviously for sure definitely important but it's also good to think about other forms of communication so are you going to present your work at a particular conference or submit a piece about it to a blog and you can think about whether there's any workshops or classes that you could present on it to get your research shared more widely and with that you'll want to think about how you adapt your style and your tone of communication to suit those different audiences right and presenting a conference for example is going to be a much different style and tone than submitting a piece for a blog so you're going to really want to think about how you communicate that that research to different audiences and finally you're going to want to share document and validate your findings and what we mean when we talk about validating your research is making sure that the right thing was done by allowing your work to be studied reproduced and or modified as needed and to do that you want to allow as many people as possible to be able to access your methodology and any code or data you're going to want your research to be as transparent well documented and open as possible and of course they're going to be caveats to this so you might be working with adding in data that's restricted so you could look at work rounds maybe you could create a dummy data set so that people can still run your code and work for your methodology and yeah and I just I know on that that reproducibility is a really important area of research and it's often neglected which is why it's good to think about how you'll document your work before your research gets underway and there's just a few things to know as we head towards the end of the presentation and that's the these steps that I've outlined these eight steps they're not linear there's many points from each step that you'll need to return to or apply throughout the research research process for instance documenting your research is something that you'll want to apply from the beginning and when it comes to computational social science projects most or all of them are going to require many iterations and that means you're going to need to revisit certain steps so maybe you'll come up with a research question in step one but after you've actually explored that problem further in step two you might want to jump back to step one to then reform you know sort of reformulate that research question in light of something that you've read or maybe you're on step four and implemented your method but you need to then go back to step three because you've not outlined the concepts and the processes enough so this is iterative and yeah that's why um documentation is really important so that you can capture all of those nuances and changes as your research evolves and I've often had it where I'll make some really important changes to my code but I don't end up documenting it and then when I come to writing up my methodology I'm struggling to explain why I opted for this type of algorithm or this particular Python package and yeah it's a shame because those are the really interesting insights right it helps you figure out how someone took a path and why they took that particular path so it is really important to just document each of your steps and it just makes the writing process so much easier right um you have everything there that you need so before we head to the end I'm just going to go back to Mentimeter because we've got a few more polls and stuff um but just to note this is pretty much the end of the presentation we are going to have some Q&A but I'll just head to Mentimeter because we're going to do a few last interactions so I just want to know um some main takeaway points that you have um so anything that you've learned that's interesting today what maybe just um a few points on um what you think of CSR and yeah anything you feel like nice fascinating useful steps less intimidating now that's that's exactly what we want to hear because you know it just because you're a social scientist does not mean that you can't use computational methods um for sure useful accessible layout nice I'm glad yeah dive in refine already have some skills yeah nice one you're all going to have those if you're social scientists you have like you know you know how to sort of um you've had to recode questions you've had to do statistical analysis right like you have that good data foundation reproducibility nice simpler than I thought love to hear that diverse skills needed definitely and um if anyone is interested we've got a bunch of um really good um past webinars on css stuff like hex mining and machine learning and maybe Nadia could also link the github as well so we've got these um things on the github called binders and they allow you to code without setting up your computer environment so it is really good if you want to get stuck in with some text mining or um I'm trying to think what we're going to stuck we have on there we have a bunch of stuff but just it's not coming to mind right now um you know you want to look at synthetic data whatever you can do that through our binder notebooks without having to have any previous coding knowledge so um Nadia will link that in the chat if you want to access that yeah let's have a look at some more of these clarity on overlap fair enough um promising more effective human technical balance yeah for sure I mean I'm sure a lot of you have heard of chat gpt and just like the strides that we've made in AI in like the pattern month so um it's a it's a cool field to be um getting interested in you know using some of the techniques nice I'm also going to ask you um okay um just a few references as well for anyone that's interested um just to say as well we are going to be making these presentation slides available so you can always go back and look at these in more detail um it's all going to be uploaded online as well so you're going to have the recording and if you need to go back check um you can do um so I'll also leave up as well um I'll leave my contact details as well so I've got a twitter account and you've got my email there if you want to ask me anything personal uh about the um not anything personal anything about the workshop anything css um get in touch let me know what you're interested in researching let me know if you're undertaking a project and like I said we've got those um css drop-ins as well and they're really useful if you just want to like connect with other people in different fields that are doing css stuff um they're really useful