 So thank you very much for joining us today for this webinar on the Global Flourishing Study and Introduction to Wave 1 Sample Data. My name is Leslie Markham, the Program Manager for this study at the Centre for Open Science and I'm joined today by fellow presenters Brian Nosek, Executive Director of the Centre for Open Science and Theresa Stankov, the Data Manager for this project. Mark Call, Product Owner for the Global Flourishing Registry will be providing his expertise with the Q&A and we have Amanda Stoller from our Communications team ready to help with any technical difficulties and also to help answer questions around getting the word out about this study to the community. Please add any questions you may have to the Q&A rather than the chat and we'll try and answer some of them directly in the response box and we'll have plenty of time at the end of our presentations to answer more questions and we'll also follow up with any questions we don't get to after the webinar. Please note that this webinar is being recorded, the recording will be sent to all people who register for this webinar and it'll also be used as a recorded section for the two upcoming moderated webinars for people joining in different time zones around the world. So it's my pleasure to introduce Brian Nosek, Executive Director of the Centre for Open Science to provide an overview of the Global Flourishing Study. Over to you Brian. Thank you Leslie. Good morning, good afternoon, good evening wherever you are in the world tuning in. We're delighted to have you here for introducing the Wave 1 sample data from the Global Flourishing Study. I am going to give you a brief introduction just about the context of the study and then Leslie and Teresa and others will provide some very practical insights for how it is you're going to get access to the data and make the best use of it you can for your scholarly questions. Leslie, you can move ahead to the next slide. This is a unique study. It is unique in a variety of ways. The place where it's not unique is on investigating questions related to flourishing, life satisfaction, health, mental and physical, pursuing understanding of meeting and purpose, character and virtue. These are topics that have a variety of different research applications across the social sciences and many researchers are pursuing questions advancing theory related to these topics. But what the Global Flourishing Study adds is a couple of unique dimensions to expand and improve the investigation of these areas of research. One is that the sample is very large. It would be 200,000 or more participants in total allowing for precise estimation of the effects that get investigated using this data set. A second feature is that it's a very substantial survey with a variety of different measures that provide a great degree of flexibility for what questions to ask and how best to answer and address them with the data set. A third feature is that it provides the opportunity to examine both the variability and generalizability of the phenomena across a variety of different countries. A total of 22 countries are included in the overall data set and each of them will provide a context for being able to make some inferences about whether these things appear to occur globally as you investigate them or are situated in the cultures and contexts in which the data collections have occurred and the varieties that appear in that. The last key feature is that it is a longitudinal study that the same survey will be conducted over a five-year time span allowing investigations of change over time and also allowing modeling that can help to reduce the number of plausible explanations for the relationships that are observed to get closer and closer to making confident causal inferences about these kinds of questions that are very difficult to investigate causally in normal data collection circumstances. So that unique combination all in a single data set will really, we believe, help to advance these areas of research substantially compared to what's been possible before. You can move to the next slide. It is important for us to call out here that this is a community effort, the role of the Center for Open Sciences on data management, making the data accessible and providing the guidelines and guidance for how to use that data as effectively and rigorously as possible. Gallup is responsible for the data collection parts of the project, seeking out representative samples for each of these and then managing the data collection and engagement with those samples in each nation. And then the substantive expertise on these research questions and the survey design is provided by researchers at Harvard and Baylor universities who are the leads of the overall conceptualization of the project. And you can see here on the slide that it is generously funded by a team of funding organizations and foundations that contributed a substantial investment to make this data collection possible. The next slide. One of the key things that we bring to this project is to conceive of it as an opportunity to be an exemplar for how research can be more rigorous and more transparent to both make sure that the use of the global flourishing data is done as effectively as possible, but also to provide inspiration and aspiration for others in these areas of research to emulate the types of rigor and transparency that you as researchers will use in applying this data. So there are a number of different features of this project that ensure that it will be as open and usable as an accessible as possible for the research community. Teresa and others on the team are working very hard to make sure that the data is highly usable, easy to understand, easy to apply. We have created and will manage a registry so that researchers can do their planning as clearly and effectively as possible for how they're going to engage with the data prior to having access to the actual data so that the inferences that one draws from that research can be very clearly distinguishing between what was planned and advanced and the hypotheses that were tested and confronted versus what was discovered after the fact and unplanned research or often referred to as the distinction between confirmatory versus exploratory analysis. So that registry will be a key part of the workflow that you'll experience in going and getting access to these data and then making use of it. Another feature is that we are trying to encourage early data, these rigorous practices, by providing early data access to those researchers that decide to submit their research designs as registered reports to participating journals or just pre-register their research so that they can provide that clarity between what was planned in advance and what was discovered after the fact. If you're not familiar with registered reports as a publishing model, don't worry, you're going to get an introduction to that. It is an exciting model that's been adopted by more than 300 journals that relieves the pressure to have to find particular things and instead focuses the review process on asking important questions and designing good methods to test those questions and then you get guaranteed that your results will be published regardless of the outcome rather than feeling like you have to get certain kinds of outcomes, positive results, novel results in order to get the publication. And finally, our team has been working hard to provide a variety of different materials. If any of these practices are new to you, pre-registration, register reports, data sharing, and otherwise, to make it easy so that we can lower the barrier to entry and improve this as both a rigorous research expedition but also an education and learning opportunity for how it is these practices can occur, can be done well, and how they might be translated to other research applications that you might pursue on your own time in other contexts. So I'm going to hand it now back to Leslie to introduce the ways to access the data and how you can engage with it. So thanks Leslie and thanks everybody. Thank you very much, Brian. That was a great introduction. So ways to access the data. I just want to start with a few practical comments. At the moment, we have Wave 1 sample data, which is the Sweden data set that's available now. And you'll hear this frequently said throughout this webinar. This data set cannot be published because it is perturbed and randomized. So please use it to develop your studies but don't publish it. And then the Wave 1 global data will be available in the coming months. Gallup is still in the field trying to wrap up some final surveys. And so we're hoping that we will be able to release that data in the winter and do make sure that you sign up to receive emails so that we can make sure that we stay in touch and let you know when that's available. So Brian has already said, this is a longitudinal data set. And so to be able to do rigorous research designs with this Wave 1 where it's not longitudinal yet, we are requiring you to either construct a synthetic longitudinal study by linking this global flourishing data to other data sets or by using the retrospective childhood assessments that are within the Wave 1 data. You can find more information about that on our website, but you must make sure at least one of the questions or hypotheses is tested with this synthetic longitudinal design. So there are three ways to access the data. You can prepare and submit a registered report with pre-registration for early access. You can prepare a pre-registration independent of a journal registered report workflow for early access again, or you can wait for the data to be publicly released, which will be about one year after its initial release date. So once you've added your pre-registration to our registry, you'll receive a view-only link and it'll be emailed to the person who actually submitted the pre-registration. That researcher is free to share that link with your colleagues that you have added to the pre-registration. Please don't share the link with other researchers. We want to make sure that we capture all of the studies in the registry and so if you have colleagues who want to also get the data, please ask them to go to our website, complete the pre-registration template and we'll be happy to provide the link directly to them. So let's just talk a little bit about registered reports. So this is a novel publication format and it offers the opportunity to receive peer reviews before analyzing the data and it can guarantee the publication of well-designed studies regardless of whether the results are positive or negative. So registered reports aim to change the reward system for earning publication. So in the standard model, researchers design their study, collect and analyze their data, write a paper and send it to a journal for peer review. The authors decide where and when to send their papers and editors and reviewers decide on the value of the research based on the results. Publication bias produces published literature that looks more beautiful than reality, particularly with an inflation of false discoveries and an exaggeration of the strength of evidence for true discoveries. The first step of a registered report submission is similar to any research project. Authors submit a stage one registered report in which they provide the motivation for the research question, preliminary evidence and the proposed methodology. They write down this study proposal as a preregistration. With registered reports, the initial phase of peer review moves to before the results are known. The preregistration is submitted to a journal that accepts registered reports and we do have a list on our website of journals that we think are appropriate for this study. And this is when that journal receives your preregistration, it's called a stage one review. The date has not been analyzed at this point and so the peer review judges your proposal based on the quality and design and the importance of the research question. And the editor and reviewers evaluate the submission with two considerations in mind. Firstly, is this an important question? And secondly, is the proposed methodology an effective investigation or that question? And the journal can decide to reject the submission, ask for more information or provide an in principle acceptance. And this acceptance means that if researchers follow the plan, their study will be published regardless of whether the results are statistically positive or negative. After the in principle acceptance, researchers can analyze the data. Please don't look at the data until you have in principle acceptance if you're going for a registered report. So then you're going to be analyzing your data and then you're going to resubmit as a stage two registered report. Deviations are fine, but you should clearly document them. And if the deviations from the planner are substantial, then you should bring the journal editor into the conversation just to make sure that you don't end up with a rejection at stage two. So the last step, the researchers submit the final manuscript peer review. And this is called stage two review. Now the manuscript includes the data analysis and the conclusions and reviewers check the conclusion. It's not possible that the data is rejected after a negative result is observed, except under rare circumstances where quality checks indicate that the methodological problems led to an uninformative study. The editor and reviewers only assessed whether authors followed the planned methodology and interpreted the results responsibly. The results themselves are not a basis for deciding whether to publish the paper or not. So you may just stop for pre-registration and when you pre-register your research, you're simply specifying your research plan in advance of your study and submitting it to a registry. Pre-registration separates planned confirmatory hypothesis generating and unplanned exploratory hypothesis generating research. Both are important, but being able to distinguish between the two makes your research much more rigorous. A pre-registration is therefore a time-stamped description of your design and analysis plan written before the data has been analyzed. So in doing this, pre-registration helps to prevent confirmation bias, hindsight bias and selected reporting. And it allows for transparent reporting and an increased confidence in the claims that you're making. So that was a very brief introduction to registered reports and pre-registration. So we do have another sort of part two to this webinar coming up in two weeks' time. And so please do visit our website and sign up for that webinar as well if you'd like to have a more in-depth view of pre-registration and registered reports. And just also just highlighting at this point and I will add this to the end as well and also share my slides. If you need any help, please do contact us at globalflourishing.co.io. We're there to help with one-in-one walkthrough entering your pre-registration, office hours or to answer any questions. We're particularly interested in helping this be a community effort so that we want to hear from you as well as us helping you. And we would love you to help us make our FAQ section more rigorous by adding comments to us about things that you think other researchers would find useful. And also we're very interested in making sure that the registry and the pre-registration template is working well for you. So we would really be delighted to set up some one-in-one calls with people to walk through their pre-registration with them. And Mark Call, who's on this webinar today, is a human factors specialist and would really like some feedback to make sure that the registries were working well for all researchers. So do consider that. Okay, so now we're going to do a quick introduction to the website. So the website is a sort of microsite within the Centre for Open Science website. And it's basically the access to the data and tools for researchers. There is an additional Global Flourishing Study website which is for the more sort of public facing aspects of the project. If you Google it and end up on that site, you can, there's a tab that says for researchers, click that and you'll end up on our site. You'll see across the top there that there are several tabs. So I'll just walk you through what you're going to find on the site. So under the overview tab, that's all about the study and the commitment to open science and some information about the data. Data access sort of speaks for itself, how to access the data. There's a page about sample data. Theresa will go into more detail about that later, but that's where you're going to find the code book and the links to access the sample data. And we have a lot of resources on the resources tab. Most of them are specific to Global Flourishing Study, but there's also further reading on preregistration and registered reports. And there's also some training materials which I'll mention later. And then the FAQs. On that page, in addition to the FAQs, you'll also find the list of data request agreements. When you submit your preregistration at the end of the form, you'll be asked to confirm various agreements. And if you want to read those agreements ahead of time, you can find those on our website. So these are the real key documents that I really encourage you to read. We have a guide to preregistration and it's got lots of screenshots of the registry and it will take you right the way through how to do a preregistration, how to update your preregistration, etc. There's also a preregistration template, which is a Google Doc that you can make a copy of and use that to develop your research plan. I do encourage you to use that because you can share it with your team and work on it collectively, get something final, and then go to the registry and cut and paste into the boxes on the registry. That said, if you would prefer to work directly in the registry, you can do that. And you can save your preregistration as a draft so you don't lose anything if you need to go back to it. You'll also find the code book on the website. So obviously, take a look at that. And again, if you have some questions, you may well find that they have some answers on that FOQs page. So the other thing you'll find on the website are these training resources. So it's basically six self-guided modules around the why of preregistration and registered reports. They're slides, but they also have a text narrative below, providing you with more details about the content of the slides and obviously all the references. And they also contain tips and links specifically for the Global Flourishing Study and very much focus on the kind of the how to for preregistration and registered reports. So I hope you'll take a look at those. So now let's look at the registry. So what is a registry? It's basically a collection of preregistrations that have been created to publicly share the existence of all performed studies. It increases the availability of study results and reduces publication bias. So to start using the Global Flourishing Study Registry, you'll need to create an open science framework account. And we highly recommend that you link your account to your Orchid ID if you have one. The Open Science Framework, or OSF for short, is a free open source project management and collaboration tool that supports you as the researchers throughout the entire research life cycle from planning to conducting to reporting and discovering research. And the OSF is a product of the Center for Open Science and it's maintained and administered by our staff. So if you have any questions about the OSF you can direct them through that Global Flourishing email as well and we will make sure that they get to the right people to help. So this is a view of the Global Flourishing Registry. This is the sort of the home page and this is where what will house all of the preregistrations submitted for access to the study data. In addition to accessing the guide and the template and the code book on the website, you can also access them here. And again, I highly encourage you to take a look at those as you're developing your pre-registration. So just actually, I've already said this, but this is where using those templates and guides are going to be really helpful. You can see there's a little pink arrow there saying add new. That's just kind of like the first step to get you going. That's where you're going to be pushing add new to start your pre-registration. Now I'm talking a little bit about updating a preregistration on the registry, which is a little bit early to talk about this, but I think it's an important thing to know right from the start. When you submit a preregistration, you're going to be requesting the sample data. You're then going to come back to that same preregistration and update it to request the Wave 1 data. So when you go into the template, you'll see that you can click on requesting sample data or I'm requesting Wave 1 data. Please just make sure that you only click the sample data. You will come back and update to request the Wave 1 data. And the nice thing about this is it means that you can do your preregistration, get your sample data, then come back to that preregistration and update it based on having refined your research questions, your methodology, and then make your requestable Wave 1 data. And you will find all this detail in the preregistration guide. So when you submit a preregistration, the OSF will automatically create a private project for you and your team. Now projects are a flexible tool for your researchers to plan, collaborate, and organize a research study. And because of their flexibility, projects are an excellent place for a team to show their work and refine their ideas. And obviously it is a private space. You can make it public if you want to, but if you want to use it as that sort of collaborative project management space, then you are obviously free to do that. You'll find more details about projects on the Global Flourishing website, and they are linked to the OSF as well. All right, let's talk a little bit about preparing your preregistration. So I think this is a great way of thinking about your preregistration, and thank you to Brian Nosek for coming up with this. So use your preregistration to draft your methods and results section of a journal article. And this helps you think through exactly what you're going to be doing and how you're going to report it. So these are some features of a good preregistration. So you want to have specific steps to answer your research questions, provide details of the statistical tests you're going to use, and what p-value. Think about your methodology and how hypotheses will be tested, what exclusion rules might be applied, and how will you combine variables if you decide to combine variables, and what outcomes will be reported. And can you think of unplanned work that you might do? Add that there. And also try and anticipate any deviations from the plan, and include that. So to be as thorough as you can. And just one little example, which I think is, you know, a good one, but very simple, is instead of saying something like, we're going to report the average, say we're going to report the mean, median, or mode. So do it at that specific level. So just once again, please note that it's only the sample data that's currently available, and also that there is no non-sensitive data currently available. So when you're in the preregistration, you'll see that there's a box ticking, you know, are you requesting sensitive data? And if so, do you have an IRB, et cetera? That will not be applicable for this sample data. So you can just ignore that. Okay, I'm going to hand it over now to Theresa to talk us through the sample data access. All right. So yeah, I'll give you guys all an introduction to what this sample data entails. So first of all, the purpose of the sample data set is to allow you to start to plan your analysis before the real wave one data is released. So this sample data consists of just the wave one data from Sweden that we have made various transformations to such that it is informative as to what the data structure is, and what a real data set might look like. But it doesn't contain the real values for the wave one Sweden data. So it is very important to understand that this is not the real wave one data for Sweden, and is therefore unsuitable analysis, it is there to help you plan for your actual analysis once that data is released. So we actually have two sample data sets available, one for researchers who submit a preregistration for their study that contains the data that has been perturbed from its original values, and one available on the GFS website that is both perturbed and randomized. So next slide. The perturbed data has a small amount of random noise added to the data in order to obscure the underlying values. So again, due to this process, the sample data is not suitable for publication. Next slide. So for ratio and ordinal scale data, such as the many questions that have respondents rate their feelings about something on a scale, the perturbation process involved selecting a random number from a normal distribution, rounded, of course, to be a whole number, and adding it to the original value to produce the perturbed value. So the end result is a data set that somewhat resembles the original, but isn't usable in final analysis. And just as a disclaimer here, this is all simulated data that you're seeing, but this is what a real question from the data would be. Next question. To account for the questions that have categorical answers, where perturbing the response value to the next numerical value that it's coded to isn't necessarily meaningful, like it is for the scale data, we chose to randomize a certain percent of the responses into a different category, such that the perturbed data set, again, roughly resembles the original data set in terms of the available answers, but it doesn't reveal the true values of the underlying original data set. Next slide, please. Now, the data set that is available on the GFS website without a preregistration is also randomized in addition to being perturbed. So basically for each of the variables, the values are shuffled into different rows, such that any correlations that might still be discernible between the different variables, even with the perturbation process, will be obscured. Next slide. We also have a code book available on the GFS website to help you interpret the data and see answer values that might not be available in the data from Sweden, such as political parties that might be suitable options in other countries but aren't suitable options for the Sweden data. So this document is a spreadsheet that describes what each column label means, the wording of each question, in English, what type of assessment each question was a part of. So was it a part of the intake survey versus the annual survey, what the different answer values mean, among other things. So this information found in the code book is for all of the Wave 1 data, not just Sweden. So again, you will see those questions that weren't asked in Sweden because they weren't applicable. So there's questions like in countries where Buddhism is the primary religion. It's only asked in those questions or in those countries but not necessarily in Sweden. And again, those questions that have answers that are country specific due to things like political party or income, those are also found in the code book. So the sample data is provided in CSV format, and this is actually a question I was in the process of answering, but I'll answer it out loud instead. The data is available in CSV format, but we have also made the sample data available as .rds files such that the question and value labels are attached to the data for your convenience. So you can use that or you can use the CSV. And the current plan for the final data set is that we will release it as .csv and also as an SPSS file because that is the format in which we receive it from Gallup. So that is the current plan for the file types that we will be releasing the sample data as. And yeah, thank you very much. Thank you very much, Theresa. That's really great and very helpful. I would just add one additional thing as well that the Gallup questionnaire development report is also available on our website and it's under FAQs if you want to have a deeper look into how the questionnaires were developed. Thank you, Theresa. So now I just want to talk a little bit about whether you would like to participate in some research opportunities with us. So the Centre for Urban Sciences is expanding the knowledge base on the efforts of registered reports and pre-registration with a randomized control trial and a survey in the Global Flourishing Study. And this work is being supported by a National Science Foundation grant. So the study measures the effects of the impact of registered reports on timelines from conception to publication, publication outcomes, so which studies are ultimately published and researchers' beliefs and experiences with registered reports and pre-registration. So this is just a diagram of the participant workflow. So there are four arms, so participants can agree to be randomly assigned a Global Flourishing Data Access pathway of pre-registration or registered reports and be surveyed on your experience or opt in to be surveyed on your experience with either mechanism after you've completed your registered report or pre-registration submission. Anyone can enroll to be randomly assigned a pathway prior to uploading a submission to the Global Flourishing Registry and the main outcomes will be comparing the two randomized participant arms while the self-selected participants will help us better understand selection processes. So we believe that this work could really be potentially transformative to how research is conducted and we encourage you to enroll if you're interested, especially if you want that early access to the Global Flourishing Data Sets but are undecided on whether you want to do a pre-registration or a registered report or if you want to try out a registered report but aren't sure where to start and if you want to be part of a growing evidence collection on pre-commitment devices like registered reports and pre-registration. There's a link on the COS website and as I said before, I will share these slides with you so don't feel like you need to write down that relatively long URL there. So once again, just reiterating that we are definitely here to help so do feel free to email us, arrange a call to walk through your pre-registration and definitely join our email list to receive updates.