 Hello everyone and welcome back to the second panel discussion of the day. My name is Ashley smart. I'm a black male with short hair and a beard and wearing a blue sweater. And I am the associate director of the night science journalism program at MIT and an editor with undark magazine. And as a science journalist, I found that the stories that really interest me the most are the ones that have really interesting science questions, but also have a real potential to impact people's daily lives. And also where the, the experts even don't always agree on a lot of the questions surrounding the science and what it all means. And I think this is definitely one of those topics. And we are fortunate to have three experts in this area who are all bringing to the discussion, different perspectives and different specialty areas of expertise. And so I'm looking forward to this discussion 30 minutes is not a long time to talk about it so I'll go ahead and introduce the panelists and we can dive in. Dan Benjamin is a professor in the behavioral decision making group at UCLA is Anderson School of Management and in the human genetics department at UCLA at UCLA School of Medicine. He is a co founder and co director of the social science genetic association consortium, which aims to promote the responsible use of and responsible communication about genetic data and social science research. Daphne Marchenko is an assistant professor at the Stanford Center for biomedical ethics. She holds a master's degree in politics development and democratic education and a PhD in education, both from the University of Cambridge. Dr Marchenko's work advocates for and facilitates research efforts that promote socially responsible research on communication of and community engagement with social and behavioral genomics. Welcome Daphne. And last but not least, Aaron Panofsky is a professor of public and public policy and sociology and director of the Institute for society and genetics and sociology at UCLA. Very California centric panel today. The research focus is on race and genetics from various various perspectives, the ambiguities of the race concept and genetics, how behavior geneticists manage race researchers in the midst, and how race racist understand genetics research and appropriated for their purposes. Thank you all for being here. And today we're talking about the state of the science on social genomics. We're the science stands and where it's going. And I'll go ahead and dive in and start with with you dad you've done a lot of work in this area if you've worked with groups that have done genome wide association studies on a wide range of behavioral traits, notably educational treatment, but also traits like neuroticism and depression and risk tolerance. Can, can you give us a big kind of birds eye view picture on where the science currently stands what the science can and cannot tell us about links between genes and these kinds of complex behavioral traits. Sure. There's been. So prior to social and behavioral genomics there had been 100 years of research with twin adoption studies that had shown some influence of genetic factors on nearly all human phenotypes including social and behavioral phenotypes. But if we go back just 10 years, we knew almost nothing at that point about the links between specific genetic variants and social behavioral phenotypes in humans, and much of what was believed at that time has turned out not to be true. For example, 10 years ago, it was widely expected that there would be just a few genetic variants that would drive traits like a person's willingness to take risks or their willingness to delay gratification. And that each of these genetic variants would explain a lot of the variability across people in these traits. In the last 10 years, there's been extremely rapid progress due to a lot more genetic data and to a research tool called a genome wide association study, a GWAS term that's come up a few times but just to briefly explain what it is. You take data on millions of genetic variants from throughout the genome, rather than just studying a particular one that you have a reason to think is going to be associated with a phenotype. You measure, you get genome wide data, and then you let the data tell you which genetic variants are linked to the phenotype that you're studying. When we applied GWAS to social and behavioral phenotypes, what we've learned is that many of the earlier, so one thing we've learned is that many of the earlier published associations with social and behavioral phenotypes turned out to be false positives. A lot of the earlier work had been publicized as finding the gene for something like the warrior gene or the gay gene. And one thing we've learned is that those genes just don't exist. Instead, for essentially all social and behavioral phenotypes and also many diseases, there are many thousands of genetic variants that are associated with the phenotype and each of them has a tiny effect. Another thing we've learned is that far from being a genetic variant for a particular phenotype, there's a lot of overlap between the genetic variants that matter across different phenotypes. And in particular, many of the genetic variants that matter for social and behavioral phenotypes also matter for many medical and disease phenotypes. One of the most important things that we've learned for the purpose of future research is that it's possible to aggregate the effects of many genetic variants into a single predictive variable. And that's called a polygenic score where the term I prefer is polygenic index, which doesn't have the same connotation of scoring higher or lower. But the idea of a polygenic index is that it's essentially adding up the effects or associations of all of the or many or all of the genetic variants that you've measured in the GWAS study into a single variable. Now, how powerfully a polygenic index can predict the phenotype depends partly on the heritability of the phenotype, which is to say just how much genetic factors in general matter. But also how big the sample size is that you use for running the GWAS, which lets you get the, you know, learn what the associations are between the genetic variants and the phenotypes. And it also matters what ancestry group you're you're studying and virtually all of the genetic research has been done in people of European genetic ancestries. As Melinda mentioned in the last panel, what that means is that for the polygenic indexes that we've created, they by far have the most predictive power for individuals of European genetic ancestries. One of the most predictive polygenic indexes for social behavioral traits at present is for educational attainment, which is the number of years of school that a person goes to. So, to be precise, we can explain in individuals of European ancestry of European genetic ancestry, we can explain about 15% of the variation across people in their educational attainment. But that's about the same amount of predictive power as family socioeconomic status has for years of educational attainment, and not as much, not quite as much as you can predict if you know a parents, both parents, educational attainment. And key point about these polygenic indexes that I want to emphasize is that that amount of predictive power is large for research purposes and that's why it's so useful, but it's very small. If you're trying to predict any individual level of educational attainment so it's really not a good tool for trying to predict life outcomes for individuals but potentially extremely useful for research. And I'd like to come back. I think we'll eventually circle back to talking more about polygenic scores. Then you kind of alluded to this explosion of research with GWAS or genome wide assessment association studies, and which have produced massive amounts of data and information about links between genes and traits and Aaron I'd like to bring you in here. Because this explosion of data raises the question of, okay, who should be able to use this data and what constitutes a legitimate use of this data, and we're already seeing it used in a variety of ways some that seem more legitimate than others. Do you think the field has coalesced around what constitutes a legitimate use of social genomics data do you think it needs to. Thanks Ashley for the question. First I'll just say that I'm a white man in my late 40s I have short brown hair, no beard and I'm wearing a checkered shirt today. So, so I think actually the thing that's interesting. One of the things that's interesting in the field is that I think that there's wide dissensus about exactly. First of all what the boundaries of the field are so are we just talking about social and behavioral phenotypes social behavioral science. But it's actually very connected to the fields of medical genetics in the fields of population genetics and other neighboring fields of the boundaries between the social and other domains medical population research and also clinical and commercial applications. Those boundaries are fuzzy. So, and within that field I think that there's a fair amount of dissensus about within that broader space I think there's a fair amount of dissensus about what really constitutes the most promising and also most legitimate uses of the science so. So, I think that there are disputes about whether we should really, and I mean we is in this sort of global we we should be really focusing on traits like the social behavioral traits that some of the ones that Dan and Melinda in the previous panel described or should we really kind of be sticking more to more less seemingly behavioral less socially fraught traits like cardiovascular disease or certain cancers or things like that. So that's one divide in the field so which of these domains is is more legitimate and promising and part of that is because of the difficulties in defining these traits right so what constitutes intelligence is or what constitutes even something like addiction. So we have definitions, clinical definitions or we have operationalized definitions, I think people sometimes dispute whether those are as clear definitions as something like cardiovascular disease hypertension. So it's partly about the definition of those traits and the kind of biological reality of those traits, but also about their sort of political dimensions their social and political import, and the way that they could be used to sort of stigmatize those help people things like that. Another dimension of disagreement, I think is the wall all jump to this one to the point of the research right is the point of the research to intervene in some kind of either policy or a clinical context right to sort of change people's lives and some fairly immediate way, or is it more of a research tool right we're really interested in longer term understanding or in the case I think of many of Dan more than many of their colleagues as an instrument to pry open other kinds of social and behavioral processes, where we're actually not that interested in the genes themselves or in the biology itself, but to control other sort of factors or to using it as a tool in other kinds of research where we don't really aren't really so interested in using the genes themselves in a clinical context. And then just let me talk quickly about two other divides. One is exactly how to operationalize and think about racial and ethnic diversity or ancestry diversity right is this something we want to study directly or is this a problem of stigmatization, and so forth. So how do we how do we contend with that and actually even how do we operationalize those concepts in our research, especially when scientists realize that it is a socially constructed set of categories that yet has some biological and genetic correlates with it so it's very hard to sort of think about it. Then we might talk later about the portability problem and the way that it's difficult to connect or or translate genetic results from one ancestry to another and sometimes from even within ancestry groups. And then finally I think that this there's I think disagreements in the field about definitions of academic freedom and academic responsibility. I think no one would deny the value of what Dan, his group other groups and what Eric parents was talking about in the last group of the FAQs and so forth. But some might deny that the scientists have a responsibility or an obligation to do that right some folks would say hey we should another version of this scientific freedom and responsibility. Some might say we should really restrict the availability of these data from the kind of people that I've often I've studied who might seek to download and do their own science, but in a racist mode right so maybe we should restrict the free ability of that free availability of that data, and others would say hey we can't put, you know, academic freedom the principle of scientific freedom, even if we dislike this research means we can't put boundaries on it. So, let me pause there but so I think even though there's a consensus that racism is bad and no one should be using this research for racist purposes. You know there's a consensus around that. I think there's a dissensus about how to deal with it and the way it connects to these other broader social questions and how we should move forward as a field. So there are a lot of threads there that I hope we can come back to a lot of good ones. I want to pick up on on one of the last things you mentioned, you brought up academic freedom and academic responsibility. I know that's something that that you think about a lot in an essay you call off it for scientific American in the wake of the buffalo shooting last year. You and your colleagues wrote that ethical scientific research requires a delicate weighing of risk and benefits, and that in the current model of genomics research risk to the individual are factored into the equation but broader risk to society. Do you think social genomics as a field that requires a new way of thinking about ethics of thinking about academic responsibility. Thanks Ashley and hi everyone I'm Daphne I'm an African American woman wearing a bright green shirt and bright green earrings. I think as has already been made abundantly clear in the conversations that we've had and in the points that Aaron just made in particular social and behavioral genomics is the subject of a polarizing and active academic debate. But I want to emphasize the academic aspect of that to describe the debate, because discussions about whether to conduct social and behavioral genomic research, how to conduct this research what its risks are what its benefits are. What has mostly occurred in academic environments between academics all of us here in this zoom webinar right now are academics or identify as academics. And of course this is not by random design we rely right on researchers on the regulatory bodies that they're accountable to like institutional review boards to identify what constitutes a harm and what constitutes a benefit. As mentioned already Ashley, and there is a tendency to focus on the direct harms or direct prospect of risk to individuals when doing those kinds of ethical analyses. I think there are a couple of things to to mention here one of which is that, although we rely on researchers and these regulatory bodies to help identify what constitutes a harm and what constitutes a benefit. Many other stakeholders who are affected by this research who affect this research. And when Eric Green kicked off today's event, talking about the misuse and weaponization of genomic research or perhaps more specifically the misuse or weaponization of claims regarding genetic differences in human behavior. There are non researcher communities who have been subjects of this misuse and weaponization, and who I believe ought to be included in decision making processes about harms and benefits. And so in addition to the lack of breath in terms of who we allow to define harms and benefits. There are also a dearth of policies and incentives to consider the downstream social and policy implications of scientific research you know I already explained how IRBs are focused on the prospect of direct risk to individuals. They seldom think about the broader downstream social or policy implications of research. And so the existing mechanisms that we have in place specifically here in the US because I'm coming from this, this US context. The mechanisms that we have in place for regulating the ethical conduct of research, they are limited currently and their ability to appraise the downstream implications, and in particular the potential social harms. And of course this this isn't to say and I want to make it clear this isn't to say that there aren't efforts that are being made by researchers in social and behavioral genomics to try and mitigate against potential downstream uses we've we've already had a bit of a concern about these FAQ documents, but generally there's a lack of incentive to to think about these downstream social policy implications. So to kind of summarize that, you know, on the one hand, we have a relatively narrow set of lived experiences and expertise that we include in conversations about the risks and potential benefits of research like social and behavioral genomics, but we also have little pressure to seriously consider the downstream social implications. And so in order for us to more thoroughly account for the downstream implications of social and behavioral genomics. I think also in order for us to successfully identify who holds social responsibilities for the ethical conduct and translation of this research, and that including not just researchers but other stakeholder groups. In order for us to effectively develop strategies for mitigating the harms of this research and promoting the benefits of it. I think we really need to recognize and to rectify the currently sparse and ad hoc consideration of the social harms and benefits of social and behavioral genomics, and also address the existing constraints that we put in place in terms of who gets to define those harms, and who gets to define those benefits. And I want to kind of keep this conversation about potential risk and potential benefits going and also point back to something that to a point that Aaron raised is that it feels like the field is in some ways still trying to answer the question what is the point of the research. So Dan, I'll put that question to you. As someone who's doing a lot of G was, what's what's the point for you and what are some of the broader goals you think are being prioritized in the field or somewhere promising than others, are you dubious of some. Sure. So I realized, I forgot to describe myself. I'm a white male wearing a light blue shirt and a darker blue jacket. So, why do this research. Well, I think researchers in this field have a lot of different reasons and aren't in full agreement on what are the most interesting applications. I've already mentioned that that the polygenic index for educational attainment is not a good predictor for any individual level of educational attainment. You know, some researchers are excited about the prospect of using polygenic indexes to track students and schools into the kinds of classes that that they should be taking. But, you know, that's an example of an application that I'm dubious about because of the, the, well, for a number of reasons, one of which is the low predictive power of the polygenic index. My own motivation for doing this research is that it can allow us to do better social science. So for example, this is something that that Aaron alluded to. A lot of social science is focused on identifying interventions that can make people better off. And often genetic variables are a nuisance when we're trying to identify what works. So, you know, that's clearly the case when you have non randomized study designs where genetic factors could be confounds that you would want to do research for. But even when you have randomized experiments, it's useful to control for genetic factors. Imagine, for example, you're interested in the effect of providing free preschool to young kids. That's a very expensive experiment to conduct. And we want to, you know, we might randomize we there have been experiments that have randomized whether people are given access to free preschool or not, and then look at the effects. And when you do that kind of experiment, because it's expensive, the sample sizes are small, you want to control for as much else as you possibly can to increase the precision for estimating the effects of preschool and genetic factors are a major contributor, one of many major contributors to educational outcomes and so controlling for them controlling for a polygenic index can be helpful in in identifying the effect of preschool. Another example is we often, you know, a lot of social science is interested in parenting behavior and the effects of rearing environments on on children. And it's very, very difficult to separate the effects of those environmental factors from genetic factors that are inherited from parents. One recent idea that's come up in in social and behavioral genomics research is that you can look at the association between the alleles of parents, the genetic variants of parents that are not transmitted to their children and look at how that affects or predicts children's outcomes. By virtue of the fact that you're looking at the alleles that are not transmitted, you're ruling out genetic factors as explaining why the parents non transmitted alleles are associated with the children's outcomes. So it's a way of studying how parents characteristics that are affected by those alleles end up affecting the children so it gives you a clean experimental design for studying parenting behavior and rearing environment separate from genes. Another example, which illustrates why one of the reasons why the National Institute on Aging which is part of NIH is interested in in this field. It could be helpful for identifying drug targets, for example, for late onset Alzheimer's disease. If you can put aside the genetic variants that are associated with educational attainment and Alzheimer's disease, we know that educational attainment and other environmental social determinants of health play a role in the development of things like Alzheimer's disease. And that's one reason why many genetic variants are associated with Alzheimer's. But if we're interested in identifying drug targets, we want to find the genetic variants that have more direct biological effects on the development of the disease that don't operate through those social factors. So being able to identify the genetic variants that that are associated with educational attainment lets you put those aside and and better focus on the ones that you care about. And there's many other examples like that that that are relevant to the broader interest of the National Institutes of Health in this kind of research area. There's also many examples of studying gene environment interactions and interplay. But I'll stop here. And I would encourage all of the panelists to feel free to weigh in on any question. Aaron. Yeah. This is Aaron again. So I just wanted to jump in on this and say that I think everything that Dan described is really important and interesting applications. But I think one of the difficulties or challenges of this research is that these relatively sober and you know, rigorous applications are not the only ones out there. So, and also because the tools are so readily available, right? I mean, you can, I could download my 23andme profile, which I did many, many years ago, and upload it into various portals and locations and find out my polygenic score for educational attainment, some intelligence scores, other things like that. And so one thing is that even though I think, you know, the researchers like Dan and many others would say, hey, look, this isn't going to tell you what's interesting about yourself. It's not useful and predictive at the individual level I could have a high polygenic risk score and then a low, a low actual phenotype score for whatever trade or vice versa, or accurate. And it's just not going to tell you that much but that doesn't prevent me from doing so on people from interpreting those research. Recently a couple of years ago, also these tools are so so one is that the tools are quite available. And they're, you know, they can be applied to individuals, even if they're not very legitimate a couple years ago, Spotify the music streaming app had a thing where you could download your ancestry. You know, you can get your data and uploaded to Spotify and they would curate you a a list. Now, who knows what they're doing right so I haven't really researched it far enough to realize that they're doing anything for real, or if they were just giving people or if they were somehow mapping on to your your putative genetic ancestry and then somehow giving you a ancestral playlist. Who knows they could have been doing any number of things. But the point is that that these that the data are becoming more increasingly freely available. We have an appified and we're in the sort of domain of platform capitalism where people want to combine data for all sorts of things. And there's no reason to think in in principle that say car insurance companies or life insurance companies, or other corporate entities, or private entities could start start using these data for different things. And we know that some what do you call it some there's, there have been some companies that have been trying to develop the capacity to give you polygenic risk scores for pre implantation genetic diagnosis for IVF right so the point is that I think Dan describes these. I think very important and potentially very fruitful uses of these in social science research, but we're in a kind of Pandora's box moment where the, where we don't know what people are going to try to do with them. And even if they don't have that much legitimacy there are major implications for for privacy and for the public. So interpreting these things to mean something that they that they don't necessarily mean, and for self fulfilled fulfilling prophecies to emerge. I want to try to squeeze in one more question where we open it up to a more general q amp a, but Aaron I think your response speaks to that it's potentially what you might describe as a gap between kind of the way the scientists understand and interpret these studies in the way the public understands and interprets them. And, and to that point, you know, I think some of the technical caveats the scientists are well aware of may not always kind of filtered through to the public understanding Aaron you mentioned this portability issue earlier that you know studies done for very specific ancestry groups often don't lose their predictive power not as predictive or explanatory when applied to other groups. Dan you mentioned the, the educational attainment studies is one of the kind of success stories where polygenic scores can explain roughly about 15% of the variance that we see. But for many of the other traits, polygenic scores explain much less right sometimes one or 2% and I feel like these are caveats that sometimes get lost on the public and kind of contribute to this gap. So this is a long why not to ask you a question, that's me, because you've, you've been working with developing this repository of FAQs that are in some ways designed to help bridge the gap I think between the scientists understanding public understanding. And from your experience with with pulling those together and working with so many FAQs. Do you feel like, I guess I'm interested to hear where you think the gap between public and public understanding of social genomics is widest and and if there are things additional things you think scientists should be doing to help close that gap. And one thing that I want to first mention is, you know, Dan Benjamin and the SSGAC Michelle Meyer, who were involved in the development of that first FAQ which came out, I believe in 2013, alongside the first genome my association study that the group did on educational attainment. And that, you know, a team motivation as I understand it for developing that initial FAQ was to prevent misinterpretation by media journalists as they were reporting on, or considering this study. And so we've seen over time an increase in the number of other research groups that have adopted this approach to responsible research communication and one of the impetus or underlying motivations for the creation of the repository was to be able to collate these materials which, you know, we're existing on consortia websites or researchers individual blogs but but not in this single location have them accessible to everybody. And it's hosted by the Hastings Center who put together a workshop, specifically for journalists around reporting on genomic studies on human behavior. And so we hope that the repository will be of use in particular to members of the media we've we've tried also given that these FAQ documents can be quite lengthy to provide brief brief summaries of them so that they're perhaps hopefully even more accessible. But relatedly, I think there are a couple things that I want to mention, one of which is when Aaron was talking about the increasing accessibility and availability of this data and how it's making its way into the hands of of average citizens average people who are not, you know, trained in genomic research who have that context to recognize the limitations and the nuances of the data that they're receiving. I guess Matthews who's a researcher at Columbia has done some work demonstrating some of the psychosocial implications of people potentially receiving genetic test results for things like educational attainment and the impact that that could have on self confidence. So I think that's one of the key perceptions of one's academic ability. And I will say, you know, we have a paper under review and I can put into the chat the original study that that Lucas published. But we have a follow up study trying to understand ways to potentially mitigate. Interpretations of these genetic polygenic scores for things like educational attainment. And, you know, one of the things that we found that I think really is an important take home message but if you describe appropriately the limitations of polygenic scores. You can can potentially mitigate the more determinist interpretations of these scores and what they mean so that's just all to say that responsible research communication really really does matter and I know that we've had a bit of a debate over these FAQ documents and you know what their potential impacts might be, but but at least from from the survey study that we did, you know, we did find that when people were presented with some of the text from these FAQ documents, describing the limitations of polygenic scores, it did temper their interpretation of the predictiveness of the score so that's that's one thing that I want to mention. The other thing that I think is related as we've talked kind of been dancing around but we haven't directly talked about the subject of genomic literacy and I think that there's some really interesting work that's being done by scholars such as Brian Donovan and Robbie Widow who are looking to transform the high school biology curriculum to move us away from these more mono genetic focuses to really expanding and studying the complexity of human genetic diversity and genetic information and I think that that is also something that that we need to be paying a lot of attention to and which obviously comes a lot earlier than when these FAQ studies come out because it's focused on on going earlier on in a person's educational trajectory and trying to prevent these determinist conceptions from taking root. Good points. And with that, let's go ahead and open it up to questions from the audience. I start with a question here from Larry Brody for Daniel. Given the lack of concordance between populations, the small effect sizes and large variances that come along with polygenic indices. How useful would they really be as an adjustment factor in a social science study. It's a good point that for given the discrepancy right now between the predictive power in European genetic ancestry populations and other genetic ancestry groups. It's as a control variable, the polygenic indexes are currently by far the most useful in studies of predominantly European ancestry individuals. And they have been used that way there's a study a couple of years ago by Neil Davies and some colleagues using data from the UK. They're not controlled for a polygenic index and gains the physical power that way, but like with many other uses of genetic data in both the medical sciences and the social sciences. We can, they'll be far more equitable, equitable benefits and widespread benefits of the research. We can generate polygenic indexes that have predictive power for other groups and that's going to require primarily more data, genetic data on individuals of other ancestry groups that that's a priority of NIH right now. My group is prioritizing and other groups of prioritizing methods development to allow us to try to use the data we do have from European genetic ancestry individuals to predict as well as we can and other ancestry groups but you know that that's a big problem for the field. At large, even broader than social, social and behavioral genomics. And the question for Dr. Marchenko and the whole panel. You point out that the consideration of the downstream consequences of this work is ad hoc and frequently left to the researchers. What ways can you imagine systematizing this to prioritize responsible inclusive research practices, beginning with whether the research whether the research should be done in the first place. Thanks to happy to start with that. And that's an excellent question and one of the projects that I have been involved in and which Eric alluded to in the first session the wrestling with social and behavioral genomics working group, which is led by him and Michelle Meyer and which Dan is also a But one component of that project that I helped to oversee was the formation of a community sounding board, comprised of members of the public from across the United States. And I think one of the unusual things about the formation of that community sounding board is that we were bringing together members of the public to think through ethical analysis. Specifically, when we think of community engaged efforts, it's focused on more empirical research questions, or single research studies, you know, trying to get community engagement on the formation of a bio bank and people's perspectives on how that project should be built and what are important considerations as that as it happens. And here we were really interested in including the public and discussions about what do you see as the harms and benefits of this research what are things that concern you about it and what are things that make you excited about it. And also what questions or recommendations do you have for researchers as they navigate, inducting this research, disseminating this research. And so I think that sounding board you know it's one kind of small study pilot example of an effort to try and bring the public into these conversations more directly but I think that there's still a lot that needs to be done and I see that as kind of one small step in that direction. I also would add that I think work that is being done by indigenous tribal communities is an excellent example of ways to have the community be involved in decision making processes about research use and to really really center the perspectives of community members as it relates to privacy and confidentiality, and, and again research uses so I think that those are examples of initiatives that are underway to really include and center communities non researchers groups that have historically been exploited by scientific research into these conversations about the downstream implications and not only include them but particularly in the case of indigenous tribal communities have them be involved in decision making processes around this research. And do you want to weigh in. I'll just briefly quickly say just that I'm heartened by efforts like this one. And the NIH has continued efforts to to to fossil these kinds of conversations by the, you know, American Society for Human Genetics recent statement that was referenced early earlier in the, the previous panel. I agree that some of these things that the ad hoc character of, of some of these efforts sometimes seems like a weakness but I think it can also seem it can also be a feature not not a bug because what it suggests is that there's a lot of experimentation and a lot of groups that are working on things in different ways that many institutions are starting to take this seriously and are concerned about this and that social change takes time. And so I think, you know, it might be a different thing if there was a slam dunk. Absolutely no questions solution to the problem, the kinds of problems we've been talking about and then we could figure out how to sort of institutionalize it but I think given the fact that it is a, you know, multi valent multi factorial complicated and contradictory problem, the, the actually kinds of social, the kinds of social experimentation and the diversity of institutional responses makes sense and and maybe what will start to coalesce coalesce in the next few years as a set of best practices, and so forth. And then the final thing that I'll say that I think what is really important is just cultural changes happening, right, and that I think that they're, you know, compared to so I wrote a book about behavioral genetics a historical book about this field, and compared to, you know, a generation ago, the, the communities of researchers are much less antagonistic between the you know so called critics like myself and Daphne, and the so called practitioners like Dan and others right that there's much closer to the relationship between these groups and much more willingness to mutually listen and hear each other, rather than just this kind of like sniping across a wall relationship and I think all of that is really productive, and, and also speaks to the social change throughout the sciences and the social sciences about how best to address this. And I think that there's starting to was the word kind of snowball effect of more and more people are getting concerned with these issues. So it's not just a couple of you know, hippie social scientists like myself, you know yelling about the geneticist and the geneticist saying like why are they criticizing us. It's a broader concern that people are working on collaboratively and collectively and so I think all of that is really positive, despite what I said about the difference disagreements in the field earlier I think that there is a lot of like productive discussion about how to how to deal with these things. A question from Gabriella Castillo. Why is there a lack of non European data. How much of it is distrust and how much of it is access to and from the communities that are not European. I'm sure there's no simple answer to that question but but maybe Dan maybe you can weigh in since you've been working with a lot of this data directly. Yeah, I think it's mainly a matter of where the resources were in the 2000s and, you know, in the last 20 years when genome my data have been collected the resources for paying for the data collection recruiting the, you know, having infrastructures where you had the participants in the in the research, where the researchers were located who were conducting the genetic research they were predominantly in in Europe and in the US and Australia, New Zealand, and the result was a huge imbalance in the data that ended up getting getting collected. And that's despite many efforts over the years from the very beginning to have more diverse data. It's just, it's, and I think what the community of researchers and the NIH have recognized is that it's going to take the having as a very specific goal, collecting diversion like data and and dedicated resources to that goal in order to reduce the gap. And I mentioned, I think it's some of both as the as the reader. I'm sorry, the question asker stated that it's a combination of as Dan said that the just where the data are collected from the rich American and European countries that have gathered these data and especially in Europe the relative homogeneity of those countries, the connection of health records to genomic data which has been especially valuable and country like in Scandinavian countries in the UK, and that those companies countries happen to be predominantly, not entirely but predominantly Euro European ancestry. And then, but, but, you know, also I think in the in the United States, for example, the, the legacy of data collection in some groups where there was great skepticism about how the data would be used has has meant, for example that many Native American groups in in the North American context have put moratoria on the collection of data from their groups. And that's been the skepticism and collective skepticism not sort of individualized mistrust but collective skepticism about where these data are, and how they'll be used. And so it's I think a combination of both availability. You know, where the data are drawn distrust and also some of these sort of market marketplace type type issues. Do you see it similarly, definitely. Yeah, I mean I think I would, I would echo what what both Aaron and Dan have mentioned and you know also say that I think sometimes we can as members of the research community let ourselves off the hook when we say it's it's just that people are also as has already been mentioned in which I want to emphasize there are more structural limitations in terms of where, where these research centers are located that people can get to to participate in scientific research so I think that's also something worth reemphasizing. So as I predicted the time flew by. There are lots of questions lots of really good questions that we weren't able to get to. But I think this is a conversation that will continue to go on, and I'm glad to have the three of you here today to have it with us. And at this point I think we'll take another 10 minute break, and then we'll come back reconvene at 310 for the next session. Thank you again, Dan and Daphne. Thank you. Thank you.