 So I want to introduce the next speaker. This is Sakshi Guy, and she's going to be speaking about the lack of within-region sample diversity in high-team science. Thanks, Sakshi. I just want to make sure the clicker works, and it does. Uh-oh. This is definitely not going in my favor. Hi, everyone. So today, I want to talk about the lack of within-region diversity in big-team science studies, and specifically talk about how it hinders on the issue of generalizability. Before I dive in, I want to give a big shout out to my brilliant collaborators here, Patrick Forcher and Chuan Peng, without whom this work wouldn't be possible. So since the cat is already out of the bag, I thought I would start with two visuals that really articulate the problem of diversity or the lack of diversity in science. So the first one, in this cartogram, I've tried to sort of plot the world's adolescent population by the size of not the land mass, but the population distribution. And you can see the darker blue regions represent the more densely populated countries, like India and China. And the lighter regions represent the sort of less densely populated countries, if you will, like parts of the US and parts of Europe. Naturally, now, this suggests that most of the world's population sits outside the Western pockets. However, a majority of behavioral science research and specifically psychological research sits within the Western pockets and particularly is more so skewed towards the US. And this is work from about five years ago by Muhammad Rad and colleagues who show in one of the most premium journals in psychological science that, of course, more than 50, almost 70, 80% of it comes from the UK and Europe. But what's really interesting is we also have an interesting 23% here where we don't really know where the samples come from. Now to switch gears a little bit. I want to show you this map, which doesn't look as bleak as the map before. And naturally, big team science studies are a new revolutionary methodological approach where a large sort of distributed researcher network around the world comes together to collaborate, to collect data, and has really meaningfully increased the diversity of research samples from representation right from Latin America to Asia to South Africa. And this is not just in psychology. I think what's become really interesting in this is sort of spanning across disciplines from cancer biology to primary research to animal research. So it's really in all our issues of what we've been dealing with psychological science so far when it comes to diversity, whether it's issue of replication, whether it's issue of power, whether it's issue of representing diverse cultures, big team science studies have really emerged or are heralded almost as a solution to the issue of generalizability. However, in this talk, in the next seven, eight minutes that I have, I want to argue that actually it creates an illusion of generalizability and hopefully show you some data that can bring this analogy to life. So to start with, for establishing any claim of generalizability, naturally the bar is really high at least in the social sciences. We need nationally representative samples. We need probability sampling. We also need to be really thoughtful about not just between and within region diversity, but also where our theories come from and how we can redefine some of the theoretical concepts to be adapted for global population. More practically, how can we rethink measurement and really introduce variability in research methods? And most importantly, how do we really include local researchers and collaborators from around the world to be able to pick up on generalizability? So in the talk, in the next few slides, I'm going to be sort of putting together, unpacking these three pieces of the puzzle, if you will, which is sample methodological and researcher using a case study. So this is a big team science study that was published about a year ago which claims the globalizability of temporal discounting, which is a classic decision-making bias where we prefer present over delayed gains. Researchers in this study claim, have strong claims of generalizability and we sort of reanalyze parts of their data to pay attention to different aspects of whether it's sample and methodological. So before I sort of show you the data, just to very quickly in one slide, summarize some of these claims and give you a little flavor of how far we have come with these big team science studies and how revolutionary some of these approaches are to increase the sort of scale of science that we are trying to pick up from around the world. So study had over 13,000 participants across 40 languages, but the authors claimed it represented 76% of the world's population. Clearly this is not working. There were 171 researchers from around 109 institutions and the authors claim that temporal discounting is globally generalizable, robust and highly consistent. And I think this strong claim of generalizability in particular is what I want to unpack and before I do that again, I wanna take one step back and really think about how we sample populations in research and naturally we take a slice of the world's population to meaningfully make inferences about human behavior. However, if that slice is skewed from the start to begin with then that will yield not very generalizable effects. To truly capture sample diversity we not only need between region heterogeneity which is global north-south populations also need to start to pay attention to within region diversity which is particularly more salient in rural versus urban areas in the global south. So now I'm gonna show you some data and we re-analyze. So Rogeria Toll's primary original paper had 61 countries. We took three countries across the three income classifications which is low, middle and high income. So in Africa we looked at Nigeria and compared each of these country data to the geographical sort of population distribution using census data and really tried to compare how meaningfully it differed or were similar from each of these regions. From Asia we looked at China and then finally from high income country we looked at the US. So I'm gonna show you the results sort of in sequence but I just wanna point out the red line which is naturally not visible on this because this represents only gender which was more or less split. In the next sort of represents the Rogeria Toll data that we've tried to overlay on top of the primary census data. So when we look at gender we see more or less equal distribution but when we really start to see the skewness in sample is when we look at age. So if you see right from Nigeria where naturally most of the population is younger but it's also very much skewed towards younger populations. When we look at China we also see a little bit more skewed towards younger population very similarly in the US. And then finally when we look at education levels across these countries most salient is in Nigeria where really the participants are very skewed towards overly or highly educated populations in these regions. More similar in China a little bit more representative in the US. And then finally we also wanted to compare each of these countries with the geographical distribution and really to make meaningful inferences about generalizability. We need each of the districts in each of the regions to have an equal chance of being represented but also not just that we also need to see which regions are over represented and actually we see at least in Nigeria and in China certain regions are more represented over represented than the others. And where I wanna lead you with this is sort of to make the case that why it's really important and critical that we start to sort of really rethink within region diversity in particular when we are advancing samples on underrepresented populations is really to look at countries in Africa and countries in Asia where majority of the population actually lives in rural countries. So if our big team science studies are sampling 100 to 100 participants from around the world and if those participants are not representative of these populations then it tells us very little about that phenomena and how much it replicates or how much it generalizes to each of these contexts. And finally to add another layer of complexity what even counts as representative and what even counts as generalizable vary significantly from region to region and actually what we know about diversity and how we understand diversity really comes from a Western perspective. But growing up in India I mean they were very salient and very different marginalized identities that I grew up with like caste and religion and equally other parts of the world might have more slightly more different identities moving away from the salient racial categories. And if I have a few minutes I also wanna really quickly touch on the methodological side of the story because I think it's not just that we need diverse samples we also need to a little bit do a rehaul of how we study and culturally adapt these samples to fit diverse contexts. So classically the way you measure temporal discounting is by offering participants sort of a hypothetical scenario where you ask them would you prefer $100 today or $500 tomorrow. And actually while the authors did meaningfully localize it to the currency of each country however what is the subjective value of a currency to a rural population and how can we really thoughtful about accounting for cultural variation and here I wanna point to one study which is by Dan Harushka and colleagues who really adapted a very similar social discounting protocol to the rural populations and saw their main effect disappeared when they tried to adapt it to rural populations because naturally they found that that wasn't very resonant and also more importantly they used rice and different cages of rice instead of currency to make it more salient but also to not have unintended consequences within that sort of rural village. So all to say how can we think about social and ecological and cultural variation when thinking about collecting diverse samples and then finally while they did sort of go very far or they did make the journey to translate their survey into like 40 languages which is absolutely amazing however to truly pick up on the regional diversity it is also important to understand who we're excluding by not using languages outside of official countries and then finally even though author diversity is much more led and it can open up a whole new can of worms of how imperfect our current measures are on research diversity we did use a common proxy way of determining author diversity which is through institution affiliation and we find that 87% of the authors had an institution affiliation based in the global north and why this is important and concerning is because the absence of local researchers from these underrepresented regions have important implications for what counts as equitable theme science but also sort of they are the ones who can shape some of these recruitment strategies and it creates this problem of unknown and unknown which further sort of amplifies the diversity. So with that I just wanna end by closing and thinking about how we can mitigate the risks of over-generalizability and really think more deeply about not localizing not generalizing but instead localizing in call for my epistemic humility in research and it looks like I went over time a little bit. So. Thank you. Thank you. Are there any questions that we can take at either of the microphones? Interesting, thank you. So it seems clear that we shouldn't just be studying 20 year olds in Chicago and there are nearly infinite number of ways that one sample to focus on samples can differ from another. Some are maybe more plausible to more plausibly moderators of psychological phenomena than others. How do we know when to stop? So if we incorporate urban rural differences how do we know that it's not important to incorporate short sleeve versus long sleeve preference or any other number of the many ways samples can differ? Yeah, I feel like you've thrown a very challenging question because I'm sure when it comes to diversity categories and which of it matters actually we didn't correlate this with the results to see whether which of it really matters but you can almost imagine that education would matter for income differences and through those sort of intuitive ways you can make meaningful differences but equally I think when it comes to sort of mapping which of these diversity identities are salient for the research topic that we're interested in it becomes a really messy story because in a room full of 100 people we can come up with more diversity categories than I can imagine but I think the point here is more less about sort of how well can we do better and we absolutely should on diversity but how more importantly can we sort of calibrate and make more narrow conclusions when we are advancing research particularly on underrepresented samples like from the global south and like in big team science studies. Thanks Sachi.