 I think we're gonna get started now. It's awesome to see everybody here this morning and just wanna say good morning and welcome to NHGRI's Social and Behavioral Research Branch's seminar series. Our goal is twice a year to bring in scholars who are involved in innovative research at the intersection of social and behavioral sciences and health and so this is our spring rendition of our seminar series and it's really my great honor to introduce today our distinguished speaker, Dr. Joanna Mountain. I wanna give you a little bit of background about Joanna. One of the things that I learned about her in terms of her biography is that following completion of her bachelor's degree, Joanna spent two years in Kenya as a Peace Corps volunteer and as I looked over her career trajectory since that time, it seems to me like that experience was really pivotal in terms of shaping her scientific interests as well as her career path. Upon her return to the States, she completed her doctorate in genetics at Stanford University and then went off to a little bit fair distance to UC Berkeley to complete a postdoctoral fellowship at an integrative biology. Upon completion of her postdoc, she joined the faculty at Stanford University in Anthropological Sciences and Genetics and in 2007, she joined the leadership team at 23andMe as the Senior Research Director or Senior Director of Research. There we go. Her efforts in this role are really focused on overseeing the ongoing research enterprise at 23andMe while ensuring protections of research participants who are engaged in that work. Recently, she's done some work with the PGIN team and really the focus there is understanding how receipt of genetic test results impacts people's lives. Dr. Mountain has co-authored and authored over 60 publications in the field of human genetics. And I think that she has really brought to this work her interests and experiences from the beginning in culture, linguistics, genetic diversity in Africa. And I found this wonderful quote in her own words where that really describes I think her perspective in terms of her science that each tiny segment of the genome has a history. So with this, she's played a key role in the discovery and discoveries related to genetic variation in ancestry. And I'd like to ask you all to give a warm welcome to Dr. Mountain as she's going to share with us today some of the research priorities at 23andMe in customer powered research. Thank you so much for that introduction. Yeah, I was actually gonna start out by saying a little bit about my personal history anyway, because I didn't really become a scientist till I was in my 20s. Before that I was really interested in international studies. So it all kind of came together, but the scientific thinking didn't come up until not that recently, but relatively recently. But I've become pretty much a scientist and like many other scientists I work with today, sometimes we can be bigger thinkers, sometimes we're very focused. And I'm fortunate to be complimented at 23andMe by our CEO and founder, Anne Wojcicki, who thinks bigger, she thinks big at all times. And the title here, Customer Powered Research kind of gets to her thinking big, where you don't just sort of say, I'm going to do a study of one condition and gather data for 1,000 people with that condition and 1,000 people without. I'm going to do something where those 1,000 people and many thousands more get something back. And we're going to create an engine that's powers research, but it also benefits people much more broadly. So I've had the benefit of working with someone who thinks big like that and being part of this infrastructure that I think is very powerful and I'm excited to talk about today. So I'll give a little bit of background on our mission and the service we provide and then jump into the customer powered research. So our stated mission, and we've stuck with this for years, which is one reason I've stuck with 23andMe is to give people access to their genome, personal genetic data, enable them to understand, help them understand the genome and enable them to benefit from the genome. So they're all pretty lofty goals and even providing access to personal genetic information was a huge challenge. And for me at first I was like, that's the big challenge. Enabling people to understand that information and benefit from it was sort of too much for me to think about, but we've actually made some progress, I believe in the last 10 years since 23andMe launched. So we have a bunch of sort of core beliefs at the company and I'm proud to share those beliefs with hundreds of my fellow 23andMe employees were growing rapidly and it makes a huge difference when you have this sort of shared set of beliefs about the right to access personal genetic information, the power of personalized medicine, at least potentially, and supporting people as they make decisions around health and working to accelerate research and thereby benefit people indirectly. So we have our process for getting genetic information to people, you know, starts out with a little box and people order that box, they get a tube. I think they're freaked out by it sort of seems kind of sciency because you have to spit in a tube and then mix the buffer in, like it's sort of overwhelming to some people like how can I do this sort of sciency thing? I've never done this before. So it's an introduction to how to follow the instructions and we try to make those clear. But people manage, I mean, five million people have managed to do it with our kit and they send that box back in with the saliva sample and then at some point they're invited to open up their account and learn things about themselves. So the things they learn about are based on our reports we provide and it's an ongoing, we continue to add new reports, seven reports around genetic health risk. Oh, wrong button. We actually, our first case where the FDA allowed us to market was for carrier status reports. We have over 40 of those now. Ancestry reports have been something I'm very focused on and we are really making efforts in the areas of wellness and traits where we can actually use data from our customers to put out new reports. So this is something, so for some examples, it turns out that information about Alzheimer's was something that so many people really kind of asked for, requested. Again and again, we want to know do we have any higher than average risk for Alzheimer's? And so this is a report based on the APOE variants associated with late onset Alzheimer's. We don't have any reports on the early onset genes. For example, trait report, eye color. I see these reports often, hair color, eye color, freckling as an opportunity for people to learn a little bit about genetics. They often will see that it doesn't match what they, I get less likely to freckle, less likely than average to freckle. I'm like, what? You didn't see me when I was 12. Full, just fully covered with freckles. So what's going on here? So it gives people a little bit of skepticism, which I think is really important to be a little bit skeptical about what you're seeing here. And that is an important part of the education that we provide. And people get to dig in as deeply as they want to the details. Around ancestry, we have, this is sort of our core feature, where people just crave to, they particularly want to know their proportions of ancestry from different parts of the world. And they want lots of detail as far as we can tell. And so this is one of my colleagues who has ancestry from many parts of the world, certainly some from Europe, which is the blue stripes, and the yellow indicates from Asia or the Americas, and the red indicates some ancestry from Africa. So this individual identifies as Latina, and that is sort of consistent with this geography that we see in her genome. So this is sort of just one example of what people get back when they sign up for the service. Now, at the same time, and I actually just got this invitation yesterday because I have about 10 different 23 me accounts because I'm always being a tester, spitting and spitting and getting, testing the new versions. So yesterday I got yet one more invitation to answer a survey. And so our research platform is something that people are made familiar with pretty soon after they sign up. And so the idea is that we take our customers, we have the genetic information that they benefit from, and then they provide a survey responses and we combine those to make discoveries. Now, this is kind of key. We think a great deal about how to make this, not just palatable to people, but give them confidence and trust in our research platform. So it's always opt-in. People not only consent to be part of our research program, at it's own overall level, but each question or survey is a choice. Do I answer this or do I skip it? And so that's the opt-in piece. We do have a very broad IRB protocol and the consent is broad, which was something a little bit challenging at first, but I think we've iterated and it seems to work very well. We've done a few small studies to get an understanding of our customer's understanding of the consent form, which is challenging when our IRB encourages us to do that. And from what we hear in telephone interviews, our customers tend to trust us and they think we're doing more sharing of their data than we actually do, so we're actually more conservative than people actually expect of us. They can choose not to cease participating, to change their consent status at any time. And once we typically aggregate data from many, many customers, thousands or tens of thousands or hundreds of thousands, put that together and publish, the aggregated the summary statistics rather than, and certainly, none of the individual level data, though we do have in some cases asked people, can may we share your individual level data with a collaborator? So that is not unprecedented and we, but we have to do an extra level of consent for that. So from the beginning, we have believed that online research with the access to the internet, the participation is fairly easy. Whereas my phone, it's not in my pocket for some reason. It's right there, but meaning, if I'm at the train station, I can open up the mobile 23andMe app, start answering questions while I'm waiting for the train. Instead of looking at Facebook, I'm contributing to research. So that participation, we're always trying to make it easier and easier. And the mobile app is one big push we're making on that front. Geography is not a barrier. People can participate from anywhere in the world. If I travel, wherever I travel, as long as I have internet access, I can participate. And the final point here is that, I have been told I've participated in at least 60 studies, 23andMe studies myself. And everyone is typically participating in multiple studies and actually contributing to multiple published papers. My son the other day, because he signed up again, he's like, I've done more surveys than 37% of the customers. He was so excited and which is, I would of course love to see that. He doesn't really express enthusiasm from my career very often. So that was good, but seeing that, that's the reaction we hope for. So at this point, we have over five million customers and over those five million, roughly 80% of consented to be part of our research program. That doesn't mean they've all answered a survey or a question, though many of them have, at least 75% of those have answered at least one survey, which means we have over a billion questions answered, which is kind of mind-boggling. But we're in Silicon Valley and numbers are what we care about. So demographics has been something we've tracked over time, been relatively consistent, and especially in terms of male, female, usually very, very close to 50% male, 50% female. The proportion of customers who would have mostly European accessories, over 75%, and that's been true also over time, though there was a time in 2010 when the fraction of customers who identified as African-American was closer to 1%. We launched a program to, it's called Roots into the Future that invited African-Americans to sign up. And for that program, we sent out 10,000 kits and that really seeded the growth of that portion of the cohort. And we're looking for ways to change that and do the same for the rest of the world, a big part of what I think about. So I've just brought this up. Pretty early on in the history of 23andMe, we launched our first major research initiative, focused initiative. And the goal was to enroll 10,000 individuals with Parkinson's disease in 23andMe. Have them take surveys. So we launched in March of 2009 pretty early on. And as of now, we have well over 11,000 participants who have a diagnosis of Parkinson's disease. And that's very, very powerful because we can do surveys, follow people's progression over time and really do some, a lot of research and get that published. Parkinson's disease was that first focus initiative, but we have a number of others launched since then, several, much more recently and rolling 5,000, 10,000 and we're over 25,000 people with either major depressive disorder or bipolar disorder. And that was launched actually quite recently. So that's a lot of what I do. We also have, as of a couple of years, three years ago, we have a therapeutics division led by Richard Scheller who's really focused on the power of genetics. And we actually did some of our own research to show that if you look at drugs that have been successful, the ones that have a human genetics, sort of evidence that they might be successful are much more likely to succeed. So we've been able to sort of back do that analysis and show the power of genetics in developing therapeutics. Wellness research is kind of a thing we've been working on at least throughout, but certainly the last couple of years. And so this is, I don't know of the, I'm trying to remember if I got invited to this. A subset of customers, it may have been based on at BMI. And that's why I think I should have been, because it turns out the BMI of 25 is, I've never been that uncomfortable with it, but it is sort of a cut off where people start giving it labels. And we've invited people who might be interested in weight loss to do a longitudinal participation and they are tracked into various cohorts and invited to do an exercise program or a dietary program for weight loss. And so the goal here, and this is led by my colleague, Liana Del Gobo, who says that she's very, very interested in finding out if there's any genetic architecture underlying the people's ability to lose weight and what factors are relevant there. And we may be able to do it if we get hundreds of thousands of people to participate in a study like this. So that's one track on our research, wellness research. So when we get this engagement, draw people into these studies, either the focused ones like Parkinson's or the weight loss, or just generally questions about freckling or handedness or motion sickness. We make a lot of discoveries. So each person participates in an average of 200 studies when we go down into the details. And so this is the official slide, 90 plus papers published to date, but 416 says two days ago, I got an email saying we have reached our 100th publication, which I'm super proud of. And this is just a little, we have a nice page that went up recently that shows the list of publications and the different colored boxes there show the areas, including psychiatry, behavior, methodology, traits, immunity, and reproduction. So all these, and there's an LC paper in here too. So we have papers coming out in all kinds of areas and these are each powered by 23 meat customers. Many of these are collaborations because we don't have the domain expertise in-house for everything. And so we need to reach or we don't, we benefit from people reaching out to us and us reaching out to others to team up on studies, for instance, on autism schizophrenia and anorexia and erosa. That is done in collaboration because others have much more expertise than we do in that domain. So just to go into a paper that I was particularly fond of as someone with myopia and with a simple question, we had multiple questions in the end, but it started out with the core question being, did you wear glasses before the age of 10? And if I ask in this room, how many people wore glasses before the age of 10 besides me? There are people tend to, I was excited to see, people tend to remember, I know it was like second or third grade. So you kind of remember people or you can ask your mother someone remembers when you started wearing glasses. And that simple question enabled us to sort of say the cases where the people who wore glasses early and the controls were people who did not. And that simple question led us to discover like over 20 genetic variants associated with myopia. And the cool thing was these were not just scattered randomly throughout the genome. They tended to be in genes that were linked to these, the visual cycle and seemed to be biologically relevant for myopia. And we thought that was very validating. And this is just a Manhattan plot of those genes across the genome. So motion sickness, again, I was interested in this one. And again, you see the biology once you get the genetic variants out, you see that it's relevant that the genes that are discovered are around neurological processes and interior development, which makes sense given the subject matter. And this paper, pretty much mostly 23 and me scientists, which is two of a number of our papers, but certainly not all of them. The ancestry research is ongoing. This is a paper published, led by my colleague, Kasha Brits, who dug into our database because we asked people in our customer base and we did not used to do this. We only started asking about your sort of racial or ethnic identity maybe six or eight years ago because NIH kept saying, you need to tell us how many people you have in these bins. And the IRB was also pushing us. So we had these sort of reasons, non-research reasons to ask these questions. And some people were really kind of unhappy with us. Why are you asking us this? The wonderful skeptical 23 and me customer wants to know why. But we, and so we could reply and say, we need this to do our annual reports to various entities. And so we did get information about people who identifies as African American, Latino, and European American. And so this is really customers from the US driving this research. And we could, so this is just part of a blog post. So one thing we do with almost all of our papers is put out a blog post, which is a little bit more easy to digest for anybody, customer or otherwise. And here we are in this piece, we're highlighting the part of the study that looked at Native American ancestry in Latinos and how that differed across the US. And nobody is that surprised to see that in the states bordering Mexico, you see much more native, what's called an indigenous American ancestry or Native American ancestry. You know, if you look at African Americans and look at on the upper left, you see the average African ancestry of African Americans across the states, upper right, the Native American ancestry in African Americans, lower left is the European ancestry in African Americans and how that varies. So people got to sort of see themselves and you sort of look at your own state and go, okay, does this make sense to me from what I know? And that, so what was fascinating was that, and we also went a little bit more detail looking at the country level, how much Italian ancestry do people appear to have in different states in Scandinavian? So, and this article was fascinating to me because when you look at the Twitter impact, of everyone who tweeted about this paper, members of the public who were not identified as scientists 64%, so a lot of the impact was for the public or customers, so people were going and looking at the paper. Now, we heard from the very beginning that if the paper is not open access, nobody's gonna pay that $30 just to look at a scientific paper, but if it's open access, it seems like thousands of customers are actually going to take a look at the paper. Now, they may not go and look at every detail, but they may, if they want to go more deeply than we do in our service, they can do so because of our policy of publishing and open access papers, open access journals. And so this is sort of the media impact of that paper. Many people picked it up, which means that the story goes more wide than just our customers. And they may not go into the paper and see this particular figure, which is fascinating and you can stare at it, we stared at it for hours. And each circle is a pie chart and you look at a particular proportion of African ancestry and Native American ancestry and you ask, how do those people identify? So this here, roughly 40% African ancestry and say 16% Native American ancestry, you have maybe a quarter of the people identifying as European American and three quarters identifying as Latinos. So this story here is that the proportion of your ancestry as estimated by genetics is not at all tightly correlated with how you self-identify. And that's a message that I think is fascinating and some of our customers actually focused on. All right, so that gets to something that's core to what I think about and this is how we broaden out the data that we have either for customers or other approaches. And we have two reasons to expand the diversity of our data sets. One is that we can do research. I mean, as many of you may have seen, there was a paper by Pope Joy and Fullerton that highlighted the fact that of genetics research papers and publications, participants 81% of participants seem to have European ancestry and that is something that has been found not just recently but over time. And the 81% is actually much better than the previous summary that suggested it was about 94% European participants. So we want to do something to kind of move that needle. So that in general in terms of research, but also we want to improve the service we provide to customers by having more diverse reference data for let's say our ancestry features. And we've done some analysis to sort of get at how many samples we need from each population in order to be able to tell someone you're more likely to have ancestry from population A or B and it's roughly 500 individuals, a thousand haplotypes, which is where you end up kind of getting high probability of accuracy in your inference. So recently in February, we launched a sort of an expansion of our previous project we had, we called the African Genetics Project. This is, we went global and we started inviting people from 60 countries from around the world to sign up for 23andMe. The idea was if you live in the US and you are from this country or even your parents were from a particular one of these countries, you can sign up, get access to 23andMe's service and support our research. So we're actually expecting to enroll thousands of individuals over the next year or two. And we've already reached over a couple of thousand and that's been exciting in a relatively short period of time. And so this is one way, you know, because we have this sort of give and take model where people can, you know, get something. And we, you know, sometimes you ask, well, if someone knows they're from, let's say Ghana, why would they sign up to find out? There are, and I've even seen this written about people who are from Africa, actually interested in helping other people know where they're from. So sometimes it's a generous act. Yeah, so, and we, this is even more recent initiative we launched where we are supporting researchers who are going out in the field as I did, you know, 20 years ago and, you know, working with populations around the world. We're working with a couple of PIs in the Americas, one or two in Africa, and we'll support the genotyping component of their research and maybe provide some additional support. And that's something that, so the idea is, you know, 23andMe sends the collection kits to the research who, you know, goes to the population, you know, and, you know, so this is the steps. This is sort of information for a prospective collaborator. And then, so we have done some of these collaborations, you know, we've done over the years, but they've been one-off efforts and they've been very successful. We'd like to scale that up so we can really reach out and support more researchers from around the world because they're the ones, again, like our health domain experts, these are population experts we want to team up with. We look forward to doing that over the next few years and this. Actually, this is an expansion of a previous program called our academic collaborations program where we receive submissions every year from dozens of researchers saying, I would like to do a study with you on, let's say, motion sickness or anorexia or psychiatry and may we replicate our results or attempt to in your database or may we team up and do a meta-analysis. So we've done that, you know, dozens of time we supported dozens of projects that way and this is an expansion to focus not just on conditions, health conditions and traits but on population diversity. So that's, you know, one thing I'm very proud of and this is our timeline over the next year for this populations collaboration program and our global genetics project. So we kind of have this, working for a company is quite fascinating because you are constantly iterating on your goals and like, okay, these were my goals yesterday. Are they still my goals? It's really kind of tipping on top of things which I have come to appreciate over the years. I didn't appreciate it at first. I was like, what, you want me to commit to something? You're right, you know, outside of a grant. Yeah, so we believe that this research participation benefits not just the world through publications or our service but that people get direct rewards. And so we make discoveries, share those with our customer and thereby we hope to develop a cohort of, you know, educated and engaged consumers over time. And one way is that we, you know, make the publications, we publish in open access journals or we, you know, publish, you know, pay that extra to make a paper open access. And our customers are actually quite interested, you know, with, you know, for instance, you know, with the prevalence of depression being very high, we have, you know, customers who might want to go and look at this paper in nature genetics. So that's sort of for the expert customer who wants to dig deep. We may, and you know, we have, but even, so certainly researchers, you know, finding out that this is a paper indicating that variants associated with prion disease are not nearly as pathogenic as previously had been thought. And, you know, the prognoses for having particular variants were not nearly as severe as had been considered. And we also, this paper also gives therapeutic clues. So this is a very indirect way to benefit consumers where the paper may, you know, change prognoses and the, and therapeutics research. A little bit more directly. We often publish a blog post in order to get the word out about a paper, you know, a headline about motion, morning sickness. You know, many, it's sort of in the news all the time now because of, I can't remember whose, which person in People Magazine has been. Okay, right, so yeah, right. So it's in the news, so we publish a paper. It gets the attention and people learn a little bit more about genetics by reading the blog post and so you look on the left here and that's the more serious paper for anyone who wants the details. Published very recently, just last month. On the right side, you have a friendlier, easier to access or easier to read summary of the story written by one of our writers. We've been fortunate to have writers coming in from journalism where, you know, journalists are having a harder time sticking with newspapers these days. You know, we fortunately have benefited from acquiring, you know, bringing in writers who can write in a way that's accessible to the public. So something more direct. There's a little button you can push. If you click far enough, you get to a page that tells you how many publications your data were contributed to. Mine is up to 62 for one of my accounts and so then I find out which papers were there. I am a control for the morning sickness story, I know. And I, oh, so the genetics of empathy was around a reading the mind in the eyes survey, which is a, I think a validated survey that we used and boy, that is a challenging survey. But I just, I don't know whether I was a case or control in that, where I fit in the spectrum on the genetics of empathy or the empathy spectrum. But anyway, it's fascinating, it was fascinating for me the first time I looked at this like, wow, how did I fit into each of these studies? And I could scroll down and look at all 62 of them. So that's a more direct reward of research participation. We also provide a lot of sort of mini reports that, you know, this is about morning, whether I'm a morning person or not. And it's very clear that it's because of my age, I mean, not just my genetics, but genetics combined with age contribute to this sort of suggestion that I'm more like the DBM a morning person. And I think I may be, I mean, it's still pretty early for me. It's only 730 for me. So feeling kind of like a morning person right now. Yeah. All right, so I'm going to wrap up there and take questions, but I have to thank all these, you know, millions basically of people who are participating in our research and the, my colleagues across the company, not just in our research team, but people all over the company have to contribute to building this infrastructure to conduct this research. And it's an amazing team. And I have to acknowledge all that. So happy to take questions, but I understand that we need to use the microphone over there. Hey Joanna, thanks for coming. That's a great talk. I'm curious what the sort of push from the consumers are to get results back. And I mean, from their research, it's a pretty obvious question. You guys are good at giving results back. And how do you deal with that or resist it or aggress it? So there's sort of, I think there are multiple questions in there. There's, you know, what are the, you know, if we're doing this all this research, what is happening to all this data that I'm giving you? What is, I think there's a sense amongst our customers like we want to know that you're being responsible about, you know, and not just leaving it sitting there. And so we do our best. So, you know, you know, six or seven years ago, we felt like we can't actually do all the research. We can't make, take advantage of this amazing database fully enough, which is when we launched the academic collaborations program, we thought, well, why don't we team up with outside academics to conduct the research and extract more from the database. And it does end up going into publications. And once those are out there, we can either do this sort of, you know, mini research summary in, or we can do a blog post. So though it's changed over time, you know, with the FDA sort of looking, scrutinizing everything we do, our ability to provide reports is, has changed a little bit. But we find ways to at least give people the sense that we are, you know, actually conducting the research and making use of the, the data they're providing. So. Hi, I came in a little late, so I apologize if you already mentioned this, but one of the criticisms of the health reports in particular is that they focus only on single genetic variants. And I'm wondering whether, whether you folks are looking into more complex models, polygenic risk scores, models that might include other non-genetic predictors in kind of more accurate predictions of health outcomes. Yeah, so hopefully that question was clear about going beyond, you know, more simple genetic models to polygenic models. And that is very much a part of our, you know, genetic health risk. We, it's been a challenge to sort of convince the world that we're doing that. We have to convince the world that, that if you go to a more complex model that consumers will understand that. And, but from everything I've seen, you know, when the, with the personal genomics PGEN project, people get the fact that genetics isn't everything. And that is one of our big messages that there are other factors, just like I showed you, age and genetics, as it gives, yes, gives an indication of my likelihood of being a morning person. And so we, it's a theme that comes out throughout our reports. And we certainly are doing a lot around polygenic risk scores. We have a health research and development team that is very, you know, focused on that and using machine learning approaches to come up with new models. And we're starting a lot with traits, because those seem to be, we've often used traits as a way to kind of get going and presenting customers with results because they're less, you know, worrisome and so we have a number of traits where we're using the polygenic risk scores and moving towards bringing in non-genetic factors as well, which we think is absolutely essential. I've got two questions. One is what IRB do you use? So the IRB we use and have used since beginning is E&I, ethical and independent review services. They have two locations, one in California and one in the Midwest. And so they, we first thought about using an IRB. When I started, there had been some discussions between our founders and IRB and it wasn't going well. So I reached out to a local IRB, the California one, and explained our situation. So we need help. And the person I spoke to contact said, well, the first thing you have to do is separate your consent form from your terms of service. You can't package them all together. Those have to be two independent decisions. And so we did that right away. And at that point, the IRB started getting comfortable with advising us as we proceeded. And it's been, you know, very much a collaborative effort. And we have, I don't know, many protocols. We do modifications on a regular basis as we make changes. You know, for instance, you introduce a mobile app and all of a sudden we need to get them to review how we do things on mobile. And so as we get more and more complex in our project here, the IRB has to keep up. So it's my role is to review all our submissions and try to make sure that the IRB team can even understand what the heck we're doing because it's so complex. And, but we've been fortunate that that pair of IRBs that has been willing to take the time to understand what we're doing and watch it as it's going in complexity. My other question is, how valid is self-reported disease data? And is it more valid in a population that chooses to do this kind of a test? I don't know if that was clear. The question is how valid is the self-reported disease information? And we've done a number of studies to try to check that validity. The first one was to look at how well could we replicate GWAS findings that had been done previously and we found that we had a lot of success replicating. So if we could sort of, the quality of the self-reported data was good enough for GWAS we found because we were able to replicate in most areas so in around psychiatry it was more challenging. We didn't replicate as, but nobody was having much success with GWAS anyway until we got to models that have, hundreds or thousands of genes invariant. So, so that was your first question. And the second one was. It's more valid in the public. Right. So, we have a non-representative fragment of the US population. Are they better able in a better position for some reason to report accurately their health status for any given condition? And that may be the case. And that may mean that the results we provide are most valid in a similar population. But we can look at that. We can look at, we can bring in to take into account other non-genetic factors, socioeconomic factors and so on and kind of examine that and check to see if it's consistent across those cohorts. Hi, I also have two questions. My first is with your, the global diversity outreach. How are you, or is there validation for customers that say they're from a certain country or their parents came from a certain country? Yes, so that's an important question now. I have come to trust people more over time probably because when we first did, we enrolled 10,000 African-Americans by basically asking, if you identify as African-American, here's a kit and sign up and take the survey and so on. And we were able to look and at, we would also ask people again, same question later on. And if people are consistent over time in how they self-report their identity or ancestry, that gives us confidence. If we look at the genetics, we also have a sort of a sense from averages over, or even smaller data sets and published data, what we expect for people, a cohort who identifies as Latino or African-American. We kind of have a sense, we can look at the genetic data and say, well, our expectation is that the average for this cohort will be about, say 80% African, 20% European and so on. So we can kind of evaluate at least at an aggregate level. And because of those have really met our expectations in all ways. And when we recruited people from Africa, we looked at their genomes and really not everyone looked like they had 100% of their genome from Africa. That's not surprising given the history of Africa that you're gonna find some people who have all four grandparents from Africa. But the fact that they have a great-grandparent or a great-great-grandparent who's from some part of Europe is not that unusual. So we have ways of, and then we can, again, ask for more detail about that ancestry later on. So we basically do it by looking at the genetic data, but also asking the same questions a couple of times and seeing if we get consistent answers. Thank you. The second question is with your academic collaborations. Is there any bridge towards the public health kind of sector and that population level medicine instead of just the individualized stuff? Right, so I did emphasize the academic piece. Public health-wise, I think, again, we tend to work with academics. You know, like working with Robert Green on the PGN project to look at. But that's not really, it's more social science, but not public health. And we have had conversations with people focused on public health and with the CDC, for instance. But we haven't actually teamed up that directly. And so that seems to be an opportunity for us. There are so many opportunities out there and we haven't really gone in every single direction yet. So I don't trust us not to go in every single direction. It's maybe in our future. You had a slide talking about global partnerships for population genetics. And it sounds like the data that are submitted by those partners go straight back to their researchers. So those accounts are handled differently than other accounts. I guess is there any thought that those participants would actually gain access to their data the same way that a... Yeah, so we, the previous projects we've done that our population focused have been where the researcher gets the data back and that participants often, we hope, get something else back. Either say support for a student to get education or some kind of results back. It's really tricky thing to figure out. It's sort of you have to do that on a case by case basis. We have another model that we've used in the US where a research is conducted. And this was done in Nevada where the researcher signs up 100 or thousands of people and each of those individuals gets an account. So we have a model and we are in a position to do that kind of model with a researcher, say working in a country where people have internet access and where the researchers or local public health officials think that there would be some benefit or value to the participants of getting that access, which so that again is on a case by case basis. And so we've had discussions with researchers who think yes, these participants are absolutely, they're educated, they're in this research program because they're interested in their health. We think they could benefit from access. So we, and we have a pipeline to make that work. So yes, definitely in the model. I was wondering about your genotyping technology that you've used, it's evolved over the time that you've been using it, how have you managed the different technologies assuming you've used more than one? Okay, so genotyping technology. We've actually been very consistent in always using an Illumina array. And that array itself has evolved over time, but so that's, as it changes, we have to update everything we do pretty much. We have to check and make sure that all the algorithms that link person's genome genomic data to a report or function properly. And we have a whole team of people who are dedicated to that. Yeah. Okay, so I was just wondering, will you ever evolve to moving to whole genome sequencing? That is a topic of conversation quite regularly. So we keep an eye on a couple of things. Basically the cost of the whole genome or targeted or whatever it is sequencing, and then the accuracy. The genotype array technology we started with because it was so robust. And to go from there to something that we really trust and our customers trust, something that is not as solid would be, that would hurt me. But so we haven't made that leap yet, but so we're keeping an eye on where things stand in terms of cost, because cost is a huge factor for consumers and accuracy. So we're watching that closely. Thank you Joanna for that presentation. So you mentioned that 23andMe is increasing its efforts to diversify its data. And usually or ideally when we talk about data diversification, it goes hand in hand with increasing engagements with communities that have been historically disadvantaged exploited by like scientific research. And so my question is specifically for tribal communities and indigenous populations and how 23andMe is engaging with them when trying to be more inclusive versus just drawing inferences from existing consumer databases. So that is a challenging question that's, I mean it's a topic of discussion amongst a number of people at 23andMe. We recently had a speaker come in, I don't know if you know Kalu Fox, he's in San Diego and his whole point is that if you're going to study these communities, you need to have a representative from that community actually helping conduct the research, not just conversations, but actually engaged in the conduct of the research. And that stimulated me to sort of go beyond where we had even been going before, which is how do we actually have direct communication with members of a community, but how do we identify an individual within the community to be kind of an advisor? So it's really, I mean, and by teaming up with researchers already have that you know, sort of communication and engagement with the community, we think we can get started in that because we may not have, I have links to some communities in South Africa and Tanzania, but not the rest of the world. But so we team up with others who do have those, you know, relationships with members of communities and that way, but I think we can go further with this, but it's not going to work for every community. Not every community has somebody who has the education to be a research partner and in which case, maybe we can support the educational efforts and we have plans to do that with one of the collaborations where we support a student going from a particular country to the UK to get a master's degree, return to their home country and be an expert in not necessarily genetics, but in a position to be an educator. And so it's super challenging if you have the stringent requirement that you have to have a member of the community as part of your research team because how do I identify that person and it's not something we've totally figured out but it's certainly on our minds and if you want, I can introduce you to Kailu Fox as this is his whole focus in life. Yeah. All right, so there's a question here and then in the front. Is the data that you provide back to your customer considered part of their medical health record? And so I guess maybe for life insurance companies. So if I apply, they often require that you release information and if I didn't do that, would that invalidate my policy? So the data we provide is personal and everyone gets their individual access. It certainly doesn't go directly into a medical record without us requesting your permission. In terms of insurance, the GINA Genetic Information Non-Discrimination Act policy provides protection in the areas of employment and health insurance, but not all areas of insurance. So we've been watching that, the negotiations around GINA to see if that's going to evolve to broaden out to life insurance and so on. So I'm not an expert in that area. It's something that is I'm sure debated in some corners, but it's something that we have to kind of keep, not me, people in 23 me have to keep an eye on. So we certainly support measures like GINA that do protect consumers as they gain access to this information. Mm-hmm. So what information do I get when I put my salag and the test tube and send it to you? So, so. And how long does it take? All right, so the information you get, I mean, it's going to be around your risk for particular conditions and it's dozens of conditions and then some information about what your genome says about your ancestry. And maybe some of your traits, eye color and what are the dozens of traits we've looked at and hair type and so on. So many reports and what we, the hope is that there will be something there that is novel for you that you'll learn something from the reports we provide and maybe not from every report because some of them won't be as relevant to you, but something in there will be novel and interesting and engaging and you'll learn about yourself and about some genetics. And so there's a lot to learn and everyone finds, some people actually discover new relatives that they didn't know before. So it goes from discovering the genetics of eye color to discovering a half sibling you didn't know about or a half uncle you didn't know about which is happening quite often. I keep hearing more and more stories and certainly even the employees of 23 yard making discoveries and that's, so we have the whole company is made up people who are learning about themselves through genetics. So we are very sensitive to the things people might learn. But you didn't answer the question, how long does it take? Oh, so how long does it take? So you send in your sample, well, you sign up, it takes a few days for the kit to get to your door and then when you send the sample back, it's a matter of weeks, anything. I don't know the time now, it's probably between two and four weeks. It depends on the time of year. So best to submit your sample in the summer. Right, Christmas time gets really crazy at the lab. Yeah. So what is the company? Is it the comprehensive genome database you provide from the GW studies? Is it related to ancestry? Is it related to the public? I mean, all my health and disease? Yes, it's everything. You get everything in, but on your account. So you go into, you log into an account, you get all kinds of reports and you get to choose which reports you look at, right? And so, but it's not the whole genome that we study. We study around half a million positions in a genome that are known to be informative. Right, thank you. Welcome. And thank you for your questions. Thank you.