 genetic counselors in population screening, the potential role of telehealth and AI. That might be an interesting discussion. Importance of keeping approaches and guidelines simple and understandable. The risks of and dealing with false positives, something we all think about regularly, and research needed to make health care system ready to handle population screening. How close are we? Or are we long way off on that one? There are a few things that we're not going to focus on. We're not going to address newborn screening. It has its own standing committee and expert bodies. And other than something as a model and using lessons learned from a newborn screening, we're really not going to talk too much about newborn screening. We're not addressing the screening of children or parent guardian consent. We really want to focus on adults in the meeting in the next two days. We're not debating appropriateness of nor evidence behind potential interventions once a screening for condition is found. So we're going to assume that there are professional guidelines established by others that will address those. And we're not proposing funding opportunities or mechanisms. Let's focus on where the gaps in knowledge are and how we might go about filling them through research studies. So a brief note on the structure of the meeting and its aftermath. So we've built in ample time for questions and discussions. And many of the sessions have panels. We want to allow for at least one or two clarifying questions from each of our presenters. But we want to save most of the discussion for a discussion period that's been built in. The moderators will redirect from topics. The meeting will redirect from topics to the meeting that we'll not address here. We've asked the moderators to save a few minutes at the end of each session to list a few key points that have been made during that session. And in-person attendees should raise hands to speak. Online attendees can put questions in the Q&A, but we can't guarantee we'll see them or address them during the meeting. But we will try to follow up afterwards if we don't make it to them in the meeting. A meeting summary aimed at the goal is to post a meeting summary in four to six weeks. That's traditional after genomic medicine meetings, and generally adhered to. And if a warranted, presenters and moderators will jointly prepare a white paper for journal submission. And some of those papers have been fairly influential in the past as well. So with that, if there's no burning questions, I will turn it back to Terry. You do a wonderful Gale impression. So thank you very much. So I really appreciate your stepping in. So I just wanted to recognize and maybe give a little bit of background on these meetings. Rex alluded to they're really critical in terms of the Division of Genomic Medicine and NHGRI planning, future studies in this field. We have an advisory group, the Genomic Medicine Working Group, which is a subgroup of our advisory council. These are the members, and they are almost all here unless they are filled by illness. So and then several of us at NHGRI who helped them out. The charge to the group is to assist in advising us on research needed to evaluate, move genomics into routine medical practice, reviewing current progress. Much of the goals of this meeting, as you said, now focused on genomic screening, identifying publicized key advances. That's our notable accomplishments website, which we can tell you about at some point. We don't need to talk about it here. Planning genomic medicine meetings on timely themes. And we identified population screening as a timely theme, not necessarily that it's something that would should begin tomorrow, but the research needed to determine what should be done and when probably is needed tomorrow. And then we also facilitate collaborations, coordination, et cetera. This is sort of a collage of the first 12 that we held, it's all that would fit on this slide and you can see that they're in a variety of topics. Again, as things come up and are nominated to the group and you feel free to nominate future topics for us. We had actually, we've had two others in addition to the 12 that were shown there. The most recent one was August of this past year. So they're roughly annually. And then this one is on population screening and genomics. So just to give you a feel for some of the products that have come out of these meetings, from the first we had after an initial sort of summit, we had a separate meeting that we called ClinAction from which the clinical genome resource grew. So that was specifically designed to address variant curation and how that could be done in a more group sourced way. From that has come the GenCC Consortium, which is an international effort as is ClinGen and they've both been quite successful. We added pharmacogenomics to emerge shortly after that first meeting. The Ignite Consortium, our network came out of that looking at implementation of genomic medicine, which has grown into another phase, which I'll show in a sec. I'll skip over some of these because we can't fit them all on one slide. The fourth meeting was on training and it led to the Inter-Society Coordinating Committee for Practitioner Health and Genomics, which is a very active group. The sixth led to the Global Genomic Medicine Consortium and from that the International 100,000 Cohorts Consortium, both of which are active groups and engaging groups all around the world. I've gone with sort of the major even numbers from the eighth meeting, which was sort of an overview of where our programs currently stood. We recognized we needed to develop some training efforts, particularly modules to help to train clinicians in this work. From the ninth, we had a program announcement on variant functions and disease, which also brought together the basic science and clinical communities. The 10th on pharmacogenetics led to ADOPT-PGX, which is part of the second phase of the Ignite Consortium, testing pharmacogenetic interventions for three treatment scenarios. In our 11th on implementation, we followed that up with a group of employers and exploring the role of employers in the possibility of genomic screening of their participants or their employees. The 12th meeting on risk prediction led to the next iteration of the Emerge Network, which is focusing on genomic risk assessment, as well as the population risk methods in diverse populations, sorry, polygenic risk in diverse populations, that consortium, the primed consortium, which is actively pursuing that kind of research and methods. Our informatics meeting led to a solicitation on patient-centered informatics, which we've received a few applications, not a lot, so keep that one in mind. Our 14th meeting led to a solicitation that was actually the applications were due yesterday. Yes, for genomic learning health system, so sorry if you missed that deadline. Yes, I know, I know, what can I say? And then we brought a concept, which is sort of the first stage in bringing forward a program. We don't know that we're necessarily bringing forward a program, but it has been approved by our council to at least be considered for moving forward, and that's on electronic consults or sort of distant consults provider to provider effort. So a lot coming out of these meetings, and we expect we'll have to squeeze in the 15th meeting after we have had this one. So with that, I will stop, and I'm gonna pull up the next slide and turn it over to the moderator. So I think can moderate from here to advance your slides. And Eric and Mark, take it away. I am. Well, thank you, Terry. Good morning, everyone. Thank you for coming here today and tomorrow. For those of you, I mean, looking around the table, I've seen a lot of you recently, like downtown DC last week. Thank you for enduring two weeks in a row of DC meetings. You can blame us for this one. Obviously, ASHG chose their location. On the other hand, seeing the double header nature of the ASHG meeting and this genomic medicine 15 meeting, I did arrange for really good weather both weeks. So, and if you believe I have any influence on the weather around here, I'm really impressed. In any case, I'm delighted to see all of you and I am glad there's good weather because the falls around here can be quite nice, as you've seen. Thanks to the organizers to put this together as a tremendous amount of work. But the feedback from these meetings are always extremely positive. The productivity, as you saw with Terry, is speaks for itself. Things really do come out of these genomic medicine meetings and we always learn a lot and it's really very valuable to NHGRI. I also want to put in a special thanks to my office communications crew who are over there with headsets on and doing various things. The other aspect of these meetings from the beginning, I think, was I guess we used to video capture it and put it up. I mean, now with Zoom, we do it live and we video capture it. The outreach of these meetings in part because of my terrific communications crew that captures this in a very high quality way and we'll get it up. And so it lives on. Besides live, we will have really good video of this that a lot of people around the world really do watch. We are all very fortunate to be here in this room, but not everybody's as fortunate. First of all, we can't have huge meetings, but more importantly, there's some people just practically can't travel here. So I really appreciate and when we really take very seriously the ability to get the information out of these meetings through our communications group. So I thank all of them for their hard work meeting up to the meeting now and post-production. So Mark and I are picking this off as the moderators, we have two speakers. We're not gonna give any bio because the bios are in the books. And so if you wanna read more, first of all, everybody in the room probably knows Les and Mike, but if you wanna read more, I know Les's bio is on page nine of the electronic PDF. I would just say in introducing Les a couple of things. First of all, he and I have grown up together professionally in some ways. We both came into the intramural program at NHGRI essentially from the beginning. He arrived 30 years ago because we're celebrating our 30th anniversary of NHGRI's intramural research program. I arrived one year later, even though I committed to come the year before and we were tenure track investigators together and then tenured and then have taken on various leadership roles. And Les is currently the director and the founding director of our intramural center for precision health research. And the other thing that I could say about Les, which says something about the lies, ideas of the organizing committee is that if you wanna get discussion going, you get someone like Les could give an early talk because he doesn't pull punches. He's gonna tell us what he really thinks and that will always stimulate conversation and rigorous discussion. So with Les, with that, let's take it over. Is that the bowl in the China shop introduction? In an incredibly loving way, yeah. Oh, thank you, thank you. Thanks, okay. All right, great to be here. I've attended a couple of these meetings and they were awesome. So I'm eager to participate with you here today. So I wanna talk a little bit about the Reverend here. And I think that we have to think about the Reverend because what we have are genomes. We're genome people mostly. And what we need to do is convert a genome into health. That's all we have to do. And the challenge here is that little blue arrow includes, pun intended, a myriad of considerations and nuances and complexities. But that's what we have to do. And we have to make that arrow work efficiently and effectively to improve health because that is our product. Now I'm gonna revive a bit of my rant from my ASHG presidential address and tell you there is nothing new about this. Genomics, genetics is just a technology. It's just a test and it obeys, it has to obey all the laws that apply to every other test we use in the medical center. And so it has sensitivity. It has specificity. It has PPV and it has NPV. And that's the way it works. And it is no different than a hemoglobin or a sodium in that respect. And so the PPV of the test, the positive predictability, which is of course that thing that we are going to operate on is a patient with a risk of a disease. We want the genome to tell us who those people are. And that result depends primarily on the testing scenario which the topic of the seminar is about changing, the topic of this meeting, sorry. So as geneticists, I have to say we're a little bit feeble in this regard because we have been spoiled by decades of practice where people with typically an enormously high prior probability of disease. These are loaded families, people who actually in many cases, patients who understand the disease as well or better than we do because they've lived with it for generations and decades. And they have this context and they have an enormous high prior probability of disease. And the irony is when we do genomic testing or genetic testing in that context, we actually don't change the probability of them having that disease very much by a positive result. So we're not moving them very much from a prior probability of disease to a posterior probability of disease. But if we then all of a sudden change and shift the game to a population-based game, then the prior probability plummets and it's a completely different business. So this is old, this is old, this is probably a woodcut of the Reverend Thomas Bates. It's probably a woodcut because the historians can't agree if it was a post following his death, made up a fantasy of what he looked like, but they think it's him. And this in a more modern context is what the Reverend said. And it's an equation, I'm very sorry. It's math, lots of people don't like math. And in fact, I think in college, there's a bit of a bifurcation. People who like math and science go into physics and chemistry and those kinds of engineering. And people who like science but don't like math went into biology. But here we are, we're biologists, we're geneticists and it's completely mathematical. So this equation governs not only how we interpret and classify and think about genetic diagnosis, but of course, again, it's all diagnoses. So what's the Reverend's equation saying? There's two forms of terms in there. This notation P parentheses A is the probability of some event A. Then the second notation, the probability of A bar B is the probability that A is true if B is true. That's a conditional probability. So what the Reverend is saying here is that something A, you wanna know if it's true. The probability that it's true, given some piece of evidence is equal to the probability of the evidence if A is true, if the thing is true. Times the probability of A, that is how likely that thing is in the first place divided by the probability B, because the more common the evidence is, the less likely it is that your assertion is correct. And so some people refer to this Bayes rule as the law of inverse probability because of this term, because it depends enormously on the probability of B being true if A is true. And so how do we then think about screening? Screening has to obey Bayes rule. It's probabilistic. The analytic validity of the genomic test result is a probability. The clinical validity is a probability. This thing that we now call the pathogenicity of variant. That's a probabilistic measure of the validity of the relationship between the variant and some condition. This term we coined called the clinical molecular diagnosis, which is if you take the variant and the patient's presentation into consideration, how likely is it that they have that underlying condition, which is different from how likely it is that they have a manifestation of that condition. I'm gonna talk a little bit more about that. That is penetrance. And that is a separate probability from the likelihood, the probability of the clinical molecular diagnosis. And then of course, expressivity is a probability. And that's a qualitative distribution of the nature, not the quantity, but the nature of the phenotype. And so what we need is a probabilistic model genetic diagnosis in a screening context that takes all of those things into considerations. And we just have to admit that most people don't like probability, right? I always often say when I travel, there's only two kinds of people who sit next to me. The people who are sure they're gonna get to their destination safe and sound. And then the white knuckles, the people who are sure they're going to die. People don't like intermediate probabilities. And there's a great story on how many of you have read The Signal in the Noise by Nate Cohen, a wonderful book. I'm sorry, Nate Silver. And he talks about the National Weather Service and they used to issue precipitation warning, precipitation probabilities. And they found that the customers of the website were the least satisfied when the probability of precipitation was 50%. Because they didn't know what to do. Should they go to the scene of the movie or should they go on a picnic? And so what was their solution? They would randomly change it to 40 or 60%. And people were happier. I'm not advocating for that, but I think it's a very telling comment about people don't like probability. Okay, so what I wanna frame here is a notion that all genetic disease is a susceptibility to an abnormal phenotype. We tend to think of only things like cancer susceptibility syndromes as being disease, susceptibility, genetic diseases, and let's say birth defects as deterministic. But I think we have to consider all of them probabilistic and realize that the penetrance function just goes very, very high, essentially 1.0 for some variants. And a key distinction here is that we have to recognize that having the disease but no manifestations of the disease is non-penetrance and that not having the disease is not the same as having the disease and being non-penetrant. It's a critical distinction. And I think a lot of people miss that. And of course, we know in variant classification nearly all variants are not certain to cause disease. They have a probability of pathogenicity that is less than 100%. So if they are not certain to be causative, then you have a Bayesian problem, which is that it depends, the diagnosis of the patient depends on their prior probability of disease because there's a possibility that the variant is in fact not pathogenic. There's some chance of that. So harboring the variant means, I'm sorry, for a few variants, they essentially do have a probability of pathogenicity of 100%. There's a couple variants, probably Delta F508, a few of the HBOC variants where the evidence is so overwhelming for all practical purposes, we can say the pathogenicity of that variant is 100%. In that exceptional case, then Bayes law does not apply. If you have the genotype, you have the disease, period. Whether you have a manifestation, remember is a second consideration, that is a probabilistic function, but they have that susceptibility. And that's why we have to think of diseases as susceptibilities. So I think what we need to start doing is thinking about a stepwise approach, which is to first, what we're doing in ClinGen is assessing the probability of the pathogenicity of the variant. And what the reverend would say we're doing is measuring the probability of pathogenicity, given the evidence about a variant. And that's the predictors about the variant, historical data on the variant, et cetera, ideally not the phenotype of the patient. And then we do a laboratory and a clinical interpretation of the person who harbors that variant, i.e. the reverend would say the probability that they have the disorder, given the phenotype they have, because they have the variant. And that's what we're calling a clinical molecular diagnosis. And again, harboring a variant, the pathogenicity is less than 100% is not the same as having the disorder. It's probabilistic. And for those who have the disorder, that is they have a clinical molecular diagnosis, but they do not have an apparent phenotype, then you have to assess penetrance. And the penetrance function is the probability of a phenotype, given that they have the disorder. It makes no sense to talk about penetrance if you don't even know that the person has the disorder. So graphically, again, people don't like equations. Here's sort of what the math looks like. And I published this paper a couple of years back in genome medicine. And I use a hypothetical example of diagnostic testing versus population screening from our fans syndrome with the genes on the ACMG list. And when it's in a diagnostic setting, there's a high prior probability of disease. I estimate a clinician will often order a test for Marfan syndrome if they think there's like a 75% chance. Yeah, I think this guy has Marfan syndrome. So that would be this orange and beige colored circle. And when you have that high prior probability of disease, the math says the false positive rate is this little sliver over here. Because the false positive rate applies just to the people who don't have the disease. And then the true positives apply to the people who have the disease. So you get this nice big, fat positive predictor value. Over here is when you're doing screening. That's when the prior probability of disease plummets. Then the false positive rate applies to the people who don't have the disease and it starts getting really big and you're positive. And then the true positive rate is small because the number of people with the disease is small. And so your positive predictive value collapses. And that's the challenge that we face in genomic screening. So we're applying this principle to population screening in secondary findings. And this is a real pedigree of a recent family we've ascertained. And it was a secondary finding because secondary findings, the math of the screening is the same as population screening. Now there's a world of differences between population screening and opportunistic screening secondary findings. I'm not discussing, I'm just talking about the math. The math is the same. So this man was sequenced for a neurologic disorder, this 37 year old. And it turns out his mother has a likely pathogenic BRCA2 variant. And she does not have disease. My little annotations here are the age and years followed by whether or not they have the variant plus, minus or question mark. And then a letter, no cancer or cancer. So you can do the math here. And using this model, the Bayes model, if you use just the test result on the mom, ignore the phenotype of her or her family. She has a 65% hereditary breast and ovarian cancer. If given that she is phenotype negative, i.e. she's 69 years old and she does not have cancer, that affects her probability of disease. And in fact, it drops from 65 to 47% if you consider that. Then you include the fact that she has a 65 year old sister who is affected and does have the variant. That pushes it way back up. And then if you include all the relatives, people with and without the phenotype and people who either are unknown to have the variant or are known to have the variant, then it goes back up to 83%. So you can see how these probabilities change. Now you might argue there's not a huge difference between 65 and 83%. But if you look across 58 cases that we've done, here's the distribution of the probabilities of disease in 48 pedigrees. And it's really interesting. It ranges from, I have this little red line here because we determined that the three common variants in the Ashkenazi Jewish population for HBOC are of 100% pathogenicity. So by definition, anyone who harbors that we say has HBOC. So they're to the right with a probability of 100. But here's the ones who we aren't certain of. And the posterior probabilities range from 22% to 99.98%. And that's an interesting range. And I would argue that the people on the left side of this curve need to be managed differently than the people on the right side of this curve. That's not the same clinical scenario. So I think we need to adopt this probabilistic model. And we need to, of course, do a number of things to make this happen. We need robust classifications of variants to determine what ClinGen is doing, the probability of the pathogenesis given the evidence and the actual numerical value of those probabilities are important in these calculations. Then we need practical methods. And I underline emphasize exclamation mark practical methods to determine the probability of the diagnosis given the phenotype. And I think that's going to have to be patient decision making support coupled with clinical decision support and defined care pathways in the EHR that allow clinicians broadly to be able to do this in a way that supports this probabilistic model and get rid of this foolish and naive concept of determinism, which is this simplistic thinking. If you got the variant, you got the disease, you're going to get the cancer. That's just wrong. So a couple of closing thoughts. When I teach this example, when I teach this stuff to the genetic counseling training program students, I teach them three rules. I tell them, you are absolutely obligated, no excuses if you don't, you have to assess patient risk precisely. Tell them number two rule is that you actually can't calculate it accurately. We never have all the information. Our information's always incomplete and some of the information we have is incorrect. It doesn't in any way mitigate your obligation to do it precisely based on the information you do have. And the perfect is the enemy of the good. And so we'll settle for precise and understand that it's not perfectly accurate. And then we have to think about what it means. And what it means for the patient is my third rule, which is that that number that we derive is probably not the primary determinant of care management decisions downstream. That's a much more interesting and nuanced question as to risk perception by patients and that will drive how they manage their care. And I think so the larger challenge is we have to change our mindset from one of non-directiveness to management. In screening, we are not presenting patients with necessarily the same kinds of questions that we presented them when we did our old fashioned diagnostic testing. We are not going to be primarily consoling and counseling and helping people adapt to diagnosis. What we're gonna have to do is start to motivate people who don't have manifestations of disease because we're gonna find them by genomic screening. They're not coming to us with these loaded families. We're gonna have to motivate them to engage in these health behaviors in order to mitigate their risk of morbidity and mortality. And we have to do that in a way that doesn't require hours and hours of geneticists and genetic counseling care. It has to be built into the healthcare system more generally for so all practitioners can do it. I'll stop there. I think you don't wanna take Q and A now, right? Correct. We're gonna wait until Mike talks and then we'll take questions together. Thank you, Les. Thanks. I'm Mark Williams. I'm a professor at Geisinger. And I'm pleased to introduce my colleague, Mike Murray, who is an internist and medical geneticist. He's currently the clinical director of the Institute for Genomic Health at the ICANN School of Medicine in Mount Sinai. But more relevant to this particular talk, he is co-chairing the American College of Medical Genomics and Genomics working group on population screening. As you heard from Les, the ACMG has had publications for many years on secondary findings, which we say we should not be using this for making decisions about population screening, which of course led in the absence to any guidance on population screening from the college. And so, Mike is one of the co-leaders to take that on and we'll be talking a bit about that. So, Mike. Thanks, Mark. So knowing that I went after Les, I tried really hard to come up with a reverend or a religious leader to put in my title, but I failed. So I'll start with a disclosure and that is regarding genomic screening. So I've published on where I think we're headed with all this and this was last year in the publication site at there. But I think eventually we're getting to the point where every individual has comprehensive genetic data set that's generated for their health and meant to be used throughout their lives. So it's going to be linked to an electronic health record and there'll be two types of evidence-based indications to access it. One is for reiterative screening based on age and other triggers and the other is for diagnostic assessment. So that's where my head has been for the last 10 years since we started doing genomic screening at Geisinger. Who's ready to do this? Nobody's ready, but there is a group and I'll mention it at the end that I think we'll be ready soon to enter into this. How long will it take to get there? I don't think anybody knows that answer either. So as Mark mentioned, the American College of Medical Genetics and Genomics has a population screening work group that I have the privilege of co-leading with Sonia Erasmussen from Hopkins and it includes several people in the audience here. And we're taking on the idea of primary findings. So if you create a dataset for genomic screening, what screening results should you look for and generate? And so that stands in contrast to secondary findings as Mark mentioned, that is something that the ACMG has been thinking about and publishing on for over 10 years, starting in 2013 with work that Les led together with Robert Green to think about secondary findings from clinical datasets. So the observation was made 10 years plus ago that as we started generating these datasets for diagnostic testing, that you could go back into that data and find things that were potentially useful for that patient and their close relatives. If only they know about it and perhaps they had no other chance to get that information. So looking into those diagnostic datasets and using it for the secondary use of screening. That was extended to research datasets, including the work that we started at Geissinger in 2015. So using a large research dataset that was created for research, but consenting patients to potentially get back that data and use it in the healthcare setting. So this idea of primary findings, what do you do if you create a dataset just for this purpose is something that the ACMG work group is tasked with? We've been asked to create two work products. The first is to come up with a rubric or a way of deciding what goes on the screening list. And the second task is to come up with a version 1.0 of primary findings. So a list just like is generated for secondary findings. And we anticipate that that will be versioned over time just as the secondary finding list out of ACMG is versioned. So whenever you're talking about a programmatic screening for disease, everybody refers back to Wilson and Younger. This publication is now 55 years old. This is the original cover. This can be found on the internet. It's definitely worth a look. One of the things that they point out in their principles is that they are screening for disease. And so I want to make a point with the next slide about screening for disease versus screening for risk. And it's a little bit of a different take on some of the stuff that Les just talked about. So pictured here on the left is this bright shiny truck in the 1950s that was a mobile unit put out by the California Department of Health. And so you go in the back door there. You give some health information as an intake. You get a chest x-ray. You get a result. And they were trying to diagnose tuberculosis. We were only about 10 years since the discovery and availability of good antibiotic treatments to cure tuberculosis. So the effort here was to diagnose disease. When we're talking about detecting disease risk, we're talking about something different. And this is of course where DNA and genetics sits, but this is not unique. We do the same thing for blood pressure and cholesterol. So we don't look for high blood pressure or high cholesterol because of the specific need to lower those numbers. We do it because it's risk for major disease, heart attack, stroke, dementia, kidney disease. So when we're detecting disease, the goal is treatment. When we're detecting risk, the goal is prevention and early diagnosis. So in 2021, two years ago, we had a ACMG group that came up with some points to consider around DNA-based screening. That's in your booklet that you received. On the left, we put the 10 principles from Wilson and Younger, 55 years old, but still durable and important. And we targeted them for the genomic screening use. And I'm not going to go through all of them or even read out these four that I've highlighted for you, but these four will be mentioned in the slides ahead. So the first thing that I'd like to mention is this idea that people need to be reevaluated in some instances. On the left panel at the top there, this is the diagnostic pathway. And for all three of these panels, I'd tell you to watch the yellow ball as it moves across or where it is in space with regards to other things. So in the diagnostic scenario, somebody comes to the health system with a complaint. They have signs, symptoms, family history, something else that drives an evaluation. The evaluation includes genetic testing. When there's a genotype-phenotype correlation, they make a genomic diagnosis or a genetic diagnosis and treatment goes on from there. When you're talking about screening, the initiating event is the genetic finding. So it's at the start of the line. And then the evaluation goes forth after that. In some instances, that initial evaluation will lead to a diagnosis. There'll be a genotype-phenotype correlation. In others, that patient or that individual should be recommended for reevaluation. And how often and how to reevaluate them will be different for each disease or condition or genetic finding. This is a different take on some of the stuff Les said, too. I think about genotype-phenotype as this Venn diagram. So on the left, you have genotype only. On the right, you have phenotype only. And in the middle, you have this overlap between the two. This is where the genotype and phenotype meet. And this is where, for this context, we're talking about disease or condition that's being screened for. So there will always be, in every situation, some that have genotype only without disease, including the sickle cell variant and F508. Cystic fibrosis, they're smaller in some instances, larger in others, but we have to take this into account when we're thinking about doing programmatic screening of populations. And so this Venn diagram will change, but I think it will always be incompletely overlapping, no matter what the gene disease variant or population. And so some people have heard this in talks that I've given over the last 10 years and said, gosh, if you're going to tell people about risk and then they're never going to get to disease, maybe you shouldn't be doing this work. And so I'd remind people, just like Les said, there's nothing new in genomics that isn't elsewhere in medicine. So non-penetrant risk is not limited to DNA. Here's Winnie Langley on her 100th birthday lighting her cigarette off her candle. So she was undoubtedly told by many healthcare professions, probably most of which she outlived, that that was going to kill her, that she was going to get heart disease or cancer, but she proved them wrong. I'd also point out that non-penetrant risk does not necessarily run in families. You all kind of know this intuitively. So identifying a risk in one individual and then identifying it in their mother, father, brother, sister, children, there will be differences in who has disease and who doesn't. And that gives me a chance to mention that cascade testing and the evidence development around that is really important for screening projects at this stage of the game. One of the principles that we talk about is DNA-based screening as a replacement for something that already exists. And I'm going to suggest that it has to be an improvement over the standing way of identifying risk and managing risk. So the U.S. Preventive Service Task Force came forward with recommendations for screening women for BRCA one and two risk. Here are the three publications over this period of time from 2005 to 2019. And it comes down to this table from the 2019 document. Sorry if you can't read that small print, but it says that the U.S. Preventive Service Task Force recommends that primary care clinicians discuss with women their personal and family history related to the cancers associated with BRCA one and two and their ancestry. And to decide essentially to divide all women into two groups, those on the top row there with enough risk to go forward with discussions and potentially testing and those on the bottom row where those should not be pursued. Now, when this came out in 2005, I remember having lots of conversations about a lot of people were excited that we had a strategy that didn't cost anything because it cost nothing to get this family history. But if you talk to primary care physicians, they might tell you something different. 10 to 20 minutes of somebody's time to sit down and talk to somebody about this history to record it and then to act on it is something that just isn't operational in any primary care practice that I've ever heard of. I raised this in a number of meetings. If anyone knows an exception to that, I'd love to hear about it because I think we need to know how they're pulling that off. So if people aren't being screened in this way, perhaps DNA is the alternative. We did work at Geisinger published now five years ago in this manuscript where we took the first 50,000 plus people from the Geisinger cohort, and we looked for BRCA one and two within that cohort. We found 267 cases, which comes out to be about one in 190 people with these variants and this risk. When we went and we said, well, how many of these people or what's going on with these people, we were able to find out that about 20% to be exact, it was 18%, already knew about that risk or we're already having it managed within their healthcare. What about the other 80%? So that broke up pretty evenly into two other groups. In green there, you see those individuals who had never been tested, but also don't meet those criteria, the criteria from the US Preventive Service Task Force and other groups that were actually designed to increase the pre-test probability that if you had a BRCA one or two tests done, that it would be positive, but these individuals, that strategy is not sensitive enough to pick up. In red there are the individuals who did meet the criteria or would have met the criteria, but no one had ever interviewed them about it or offered them the testing. So when you ask the question, when you take this strategy to a population, how many people with these variants are unaware but for genomic screening? In this instance, it was eight out of 10. So 80%. When I first started giving talk about this data, I remember somebody raised their hand and they said, well, you're missing 80%, maybe you're just not doing a good job, which I thought was probably wrong, but I didn't have any evidence till the Healthy Nevada Project out of Reno, Nevada, published their work in 2020, which looked at about 25,000 people. They looked not only at BRCA one and two, but they also looked at Lynch syndrome and familial hypercholesteroemia risk. And they found that about one in 75 people have a risk for one of these three genomic conditions. And in their data set, about 10% knew about this risk, 90% didn't. So I think we're probably based on these two studies looking at 80 to 90% of people finding out about these common genetic risks for the first time through a screening project. The question is how are we going to manage that over time to bring real benefit to those people? So these three conditions are called the CDC tier one, commonly and in many publications. And that's how I refer to them. So here, if you extrapolate from the data in 2023, if we did a screen for the nine genes that are associated with these three genetic syndromes, you'd identify north of four million people that have a risk for these, that have these genetic risks. And in the table there across the bottom are those genetic syndromes, but other than the people in this room, not many people care about syndromic names for these things. What they care about is risk for heart disease and stroke, risk for breast, ovarian prostate and pancreatic cancer, risk for colon and uterine cancer. So most people that are thinking about strategies will include these nine genes in their strategies. And then what to add after that is certainly a puzzle that lots of us are trying to figure out. I would mention that one of the Wilson and Younger refined criteria that we put out was that clinical implementation strategies should be in place and available to anyone that has the risk identified so that the healthcare system can manage those risks. And when you think about heart disease, breast cancer and colon cancer in particular, we have workflows and systems in place across the U.S. healthcare system to really manage these risks so much so that I think of you identified those 4.3 million people in the next to them without missing a beat. So we need to think about other risks that we add and how they'll be managed within the current healthcare system. I won't blow the punchlines on David Vintres and other people that are in the audience's work about their March publication from this year, but cost effectiveness and economic analysis is essential to do good work coming out of that. When you think about how should something get on the list, what does it really come down to? I think what it really comes down to is clinical utility. If you put something on the list and you give back a test result, a screening result, that should prompt an intervention that results in improved health outcome. Anything short of that when we start doing this programmatically on large scale or population-wide, I think has to be questioned. Now, over the last 10 years or so, we've got caught up in this term of medical actionability. It's a good term, at least for the initial thinking about what to put on the list. That's the idea that something would have a potential next step if they got a result back. But that's necessary but not sufficient, I think, when you start talking about really operationalizing this. And these two concepts, obviously, are linked together. To paraphrase, gross and quarry, a screening test alone does not have utility. The clinical utility comes in when there's effective access to appropriate interventions. One of the things that I think we should all think about is that we're in this era of genomic screening projects and pilots. And it is really the time for people that are running these projects to think about the engaged population that they have and the engaged expertise that they have and how that can drive a different list for your project or program than what other people are using. So I use the example of the pathogenic founder variance in the APOL1 gene and the TTR genes that arose in Africa and are very important for populations regarding kidney disease risk and cardiac amyloidosis if there are members of that population that have African ancestry. So not only people that identify themselves, obviously, as African-Americans, but also those who are Afro-Caribbean and have significant ancestry from other parts of the world. The other thing is that monogenic risk for things like major psychiatric illness, there's now pretty good data that those are identifiable through a screening strategy. Most people that are running screening projects wouldn't have the default to take that on, but there are individuals that are thinking about taking the single utility by returning those results is a really important task. So again, going back to primary findings from data sets created from four genomic screening is one of the ways that the ACMG group is thinking about it. And in my last slide here, I'll just cut to the chase and going, I didn't know the rules before when I created these slides. So thank you, Rex, for pointing them out, but I'll just cut to the chase down here and I'll say maybe a white paper that I've been thinking about is that a group really needs to put together a timeline and the milestones for completion of the kind of implementation that we're talking about population-wide and then maybe two future conferences since I have the floor that we could think about is, one, we are now in a moment when the projects that are driving populations screening, the big projects, are mostly privately funded. So when I was referring in the beginning about who's ready to do this, there is money from private businesses and other sources that are ready to start doing this. And we have to think about what's going to happen over the next 10 or 15 years. I have no specific complaint about anybody that's done that or that I know is doing it. But when we have a patchwork of the U.S. population, each subgroup of which has a different group of people holding their data, de-identified, albeit, but what happens as far as the data environment and the potential uses of that data. I think we have a giant LC problem that's coming. I don't think we should wait until the first real disaster happens. I think we need to start thinking now about the rules, regulations, penalties, potential criminal penalties for the misuse of data and how we're going to manage that. And then the last thing I'll just throw out there is, I think we need to start thinking about the human genome project part two. And in my mind, what that is, is recognizing people having their own genomic data set as a public good for their own health and sequencing the whole population. Sounds a bit crazy, but so did Genome Project One when it was first proposed. And with that, I'll stop with my comments and I think Les and I are supposed to set up here. Actually, Mike, we made a change. Come back. We're just going to stay here. And so what we're going to do, thank you first of all. So thanks to both Les and Mike for getting us started. Very stimulating ideas. We have a lot of discussion. We have 45 minutes for discussion, which is great. And thanks for staying on time. We're going to alternate. So I can see this side of the table, but I can't see this side, but Mark can see this side of the table. So we're going to alternate. And what would be helpful is, yeah, if you do these sorts of things when you have a question, then we will keep our eye out and we will call on people. That includes people who are not at the, who are sitting in the second row. Please write, you know, he should be able to see that as well. So I can't see the name, but go ahead. I'm Kate Nicholson from the University of Pennsylvania. So I had a comment about Les's sort of thinking about this. I personally, unless you didn't mention, I'm struggling much more with the issue of somatic testing on tumors and then identifying variants that have and reflex testing, particularly, and I don't know how many people follow us at all with the new Esmo guidelines, which are extremely broad in terms of reflex testing. And one of the things I'm personally struggling with and too bad reasons in here is NF1 reflex testing in the context of people without a clinical diagnosis of NF1. So I just want to bring that up as another form of population screening that we should be aware of and considering when you're talking, when you think about this sort of secondary findings, this is another whole set of secondary findings that I think have a big impact. And I'll just make Mike a quick comment that we've done obviously the same study, although not published, looking at our rate of your say one in two mutations and how many know and panits that's much higher. Actually, it's about 50%. So. Yeah, I guess on two responses that of first is that of course, if they're having somatic testing, there's a reason why someone did that. So that elevates their prior probability of disease. So that's really not population screening. That's working up a patient with disease. Given the list of genes now that they're doing, I mean, it's probably. Oh, so for genes other than related to their disease. Oh yeah. I get you. Okay. It's essentially secondary findings. So the list is probably 30 genes, maybe longer now. So that is, I understand what you're saying, but no, this is like essentially like you do someone who has, I'm going to say renal cancer and you essentially find a beer, say one mutation. It's not an associated cancer. You incidentally find it and it is probably from the screen. I'm just saying like that's a whole another thing we're struggling with where you're, how do you manage that? Sorry. Yeah. Thank you for the clarification. So there's two issues there. The number one is that really, we come up there is lowering the probability of the analytic validity of the fight, right? Because that finding that somatically is not as good as finding it constitutionally. So that's number one. And then number two is an interesting debate going on that some, I'm not a cancer epidemiologist, but my colleagues who are tell me there are some questions. Regarding whether or not all genes that are cancer susceptibility genes are pan cancer susceptibility genes with just different relative risks and that our power numbers aren't adequate to demonstrate that, but they think there's a very long tail with a risk. And that's a different issue. I'm happy to have a big debate about that. This is in the context. And then people are getting reflections through the reflex testing. So there's a really broad set of genes now that are suggested where you should have reflex testing. So you're, you are identifying people who have unrelated diseases incidentally through reflex testing. Sorry, just to clarify how that's, that's, that's happening. And so to me, that is another form of population based screening, to be honest, that's, that's somewhere in between a population screening and opportunistic screening. Yeah. Great. I'm going to take moderator's prerogative and ask a question also to Les. I love the concept of the practical, probabilistic model of population screening. And it struck me that we could potentially get to something to look like the Richards criteria for very ancient interpretation where you say, here are the different pieces of information that you need. Here's how you assess them in a rigorous way. So the people could move away from doing them as one office and just kind of say, here's the information we need to find. And here's the information we need to generate to be able to do that. Is that the direction you think we need to proceed? Absolutely. And we're collaborating with, I don't know how many of you know this group at Harvard, Giovanni Parmigiani, who has this website called ask to me, which is called all syndromes known to man, pardon the gender specificity of that. And it's basically epidemiologic meta analyses of cancer risk for all these syndromes by age. And then they back calculate the likelihoods of disease, likelihoods of finding a variant in a person who has disease. So all the data are there. And then again, for the reverend, you just flip that into the inverse probability. You can ask the question if you have disease, what's probably a phenotype, what's probably have the variant and such. So we can do this. And he has a very simple plug-in tool. And we're just going to have him flip that and then provide, oh, I have this person with this variant and they have cancer when they were 63. What's the likelihood or their aunt has cancer. And we can, you know, derive because, you know, the cascade testing that Mike brought up is, is as essential as it is difficult to do practically. But we can begin to derive risks from even family members who aren't genotypes using Mendelian principles. And so you can do simple plug-in computer-based algorithms that will spit out the risk of disease based on that input data. And you shouldn't have to be a Bayesian expert or geneticist to be able to do that. I think I saw Carol say that up. I think Rex was first and then Heidi. Well, he knows. So he took his pick and then I, and that's why. So it was my turn. And then we immediately found a flaw in our system because I'm covering this and somebody in the middle. So I'm going to start with Carol, but I will use Rex next on my next pair Carol. Very good. So both talks brought up a very similar question. In my mind that might follow on a little bit from, from Mark's comments and questions and that's evidence. So the probability of pathogenicity given evidence and the amount of evidence we need to move something into a tier one. And so what are our gaps there? What, what are the areas that we can improve the evidence or the process to determine pathogenicity and what are the types of evidence we need to move something into something like a tier one screen? This is, I love that question. And I think it's one of the problems I have with the tier one. And I think grouping together, for example, HBOC with FH, I think was crazy. And I, the numbers are one thing, but I think what we have to do is, is we do have to understand the numbers and we have to understand them precisely what our predictive power is and how we likely we are to change the healthcare outcome. But we also have to have a really sober discussion of our error modes. What happens when we have a false positive? What happens when we have a false negative? And I would argue that it's a completely different situation. If you have a false positive for HBOC it's not going to ruin anyone's life. I would think very differently about HBOC. And so we've got to understand the numbers and we have to have our community has to get together and make really clear headed decisions about we're going to do this. Here's what happens if we go down this pathway. Here's what happens if we go down that pathway. And I think it's one of the things that we have to do that happens if we go down this pathway. Here's what happens if we go down that pathway. And so it's not only the numbers, but it's a values discussion about what errors we can really tolerate. So I mentioned that one of the evidence gaps, one of the evidence gaps that we're going to face is right now we have to extrapolate outcomes from data that's derived by a different methodology. So we say that BRCA variants are 80% to 90% associated with disease over lifetime. That's based on different data than we're generating through screening. We don't know that number for screening, but we need to get to it. And places like Geissner now have almost 10 years of experience. We probably need 20 or 30 to really get those numbers, but we got to get to them and it's going to be lower. And it's going to be different. Great. Caitlin. Thank you. Maybe we should name sides of the room, right? So team, the team have teams on the middle. I'm Caitlin Allen, and I'm from the Medical University of South Carolina in Charleston. So Les, you mentioned motivation. And I think Mike, you mentioned utility, buy-in. And so I'm thinking about this from more of an implementation perspective. And maybe adding into that equation, we've talked a lot about the clinical utility, but thinking about research utility. And so for our population screening program at MUSC, that's been a really a huge motivating factor for us is that we have now a large population of genomes that we can then go back and do research. And that's been huge to get patient buy-in for participating and also our provider buy-in and leadership buy-in. So just maybe adding that to the equation or curious to hear any thoughts about sort of the research component of motivation and buy-in. So I think in every setting we should be getting participants to be offered the chance for their data to be used as research. Yes. So asking every participant that gets screened to be involved in research is essential. I think if we're going to do this right over time, we need as many people participating as possible. So that motivation, which most people will agree to, is really important. So I think what you're doing there makes a lot of sense. Yeah, which is that I think it overlaps with Genome Medicine 14, which is this silly notion of this huge wall that divides research and clinical practice. We just have to get over that. And every research participant should be seeing the clinical benefit of their participation and every patient data should contribute to every other person's health care through the research mechanism some way. We just have to make that happen and think more openly about that. I can just clarify that Genome Medicine 14 was on Genomic Learning Health Systems, which is doing exactly that, that basically every patient is a research participant. Every research participant is a patient. I just want to make a quick clarifying comment, Mike. There obviously have been several popularly very well done case control studies looking at risk in BERSA1 and 2 as well as other cancer susceptibility genes, which I think is what you would want to be looking at both carriers and bridges, which were published in New England Journal in 2021 and a lot of sort of resulting studies and the penetrance for BERSA1 and 2 done those well done epidemiologically matched case control studies was about 50%. So there really, those data are there and I think that they were done in studies that were really well known like nurses health and things like that. And I'm sorry that Pete's not here also to comment on that. Thanks. I would also just note that Mike's point is still relevant because the case control studies still involve somebody that has phenotype and in population screening, we're talking about ascertainment, irrespective of phenotypes. So there's still a gap there. Heidi Rehm. Thanks. So as we get better evidence and we better understand probabilities, we can better feed into, you know, recommendations for clinical screening and follow-up and we have a great framework and flingent for actionability and all the variables. But I think one aspect of this that I anticipate being a challenge is that patients' perception of what is important to them or not differs. And I think a great example of that is amniocentesis and the risk of losing the pregnancy versus the risk of a child with a chromosomal abnormality and where those lines intersected became the age a woman should get an amnio and before that they shouldn't. As if the risk of loss of a pregnancy was equivalent to the risk of a child being born with a chromosomal abnormality and obviously for each individual woman and couple, there was maybe very different concerns. A couple that took 10 years to get pregnant versus one that took one day to get pregnant. So as we think about getting good at the front end of this, do we just leave it to the physician to have to have long discussions about patient preferences or how are we going to build in sort of scalable approaches to patient preference is my question. That's a great question. That's probably for genetic counselors in the room to respond to. A session. Good. And I think what we need is to understand the basis, the fundamental basis of what these counseling decisions are and figure out a way to make what is currently a 45 or 60 minute patient encounter, which is totally unaffordable, totally not scalable. And if it's required is the end of genomic medicine. Don't want to overstate that. But if we can understand what, how this process works, I would suggest in a major large majority of cases, we could build online computer based online interactive tools that could help people understand where their values and their key decisions that they want to or outcomes that they want to maximize or minimize and say, look, a person who thinks about this the way you do generally goes down this pathway for these reasons. And we, I think we could supplant the, a large majority of this decision making from this super expensive one-on-one patient encounters to tools like to do exactly what you're hoping. I think I saw Jonathan's hand up. Okay. I'm sorry. Oh, wait, that's that side. So I maybe I missed, I maybe I missed Jeff. So Jeff and then Jonathan. Go ahead, Jeff. Thanks. We have a lot of referees around here. Tough crowd. Thank you. Yes. This is Jeff. Actually, it works well because Mike's a good follow up to Heidi. So putting, I'm a public health person at HRSA. And you, you mentioned about newborn screening might fit in, you know, in certain as learning lessons. I was really struck, Mike, by the picture of the woman with a cigarette at 100 years of age and how hard it is right now to get healthy and how hard it is to change. And particularly your comments about clinical utility. So thinking about newborn screening, sickle cells, we have great ascertainment. We've known the gene for decades. We're getting better at some of the genotype, phenotype correlations. And yet still fewer than half of children get hydroxyurea transcranial Doppler is even lower in adults. And there's such a huge health systems issue right now with things that we are very clear about. It's one of the biggest challenges that we face. And I'm not sure that, but just bear all of us to remember that you may have systems in place. Man, they're not really getting to everyone. I just follow up on that. It's a really important comment. And again, that's part of our changing our mindset from dealing with families who come to us highly motivated already with dealing with the John population, which is mostly unmotivated or maybe I don't know, was it negatively motivated? What would you call that? I can tell you in our secondary findings, there's no study of families were ascertaining includes FH exactly zero of the children in those families have been screened for hyperglossomal. And that's an AAP recommendation routine care and zero. We met covered a single one. It has no penetration. So you have to make this as automated and simple as we possibly can. And I just add that, you know, this is one of the reasons why action ability and clinical utility don't equal each other. And we've got to understand the reasons for that and come up with better systems for getting people from an actionable result to a clinical outcome. Yeah, optimistic note though is that and at least the studies that have been publishing their data to this point, more than half of individuals that have been received a result have actually made a health behavior change of some type in terms of at least getting a test ordered or something of that nature. So it is encouraging that it seems like this may be having some effect more so than what we do typically with diet and exercise and those sorts of things. Ned. Thanks. I really enjoyed the two talks, especially being right next to each other trying to navigate the information between the two is fascinating for me. So Michael, you tell us there's a lot of undiagnosed variants out there unless you tell us not all variants are created equally or uncreate. I don't know how to say our have the same risk. Some are more risky than others. And I think about applying that in the setting of population based screening where you're going to generate a lot more tests. That's what Jeff was talking about, the impact on the system overall. I think about clinical utility and what I didn't hear again was the harms. Because if you're going to generate more uncertainty, you're going to generate more positives with variation and penetrance and we're going to leave it up to an unprepared medical workforce make decisions. We are going to buy more mistakes, more errors and more harm. So as we think about clinical utility, always think about not just the good things, but all of the harms that we're going to accrue because of population based screening. And the whole, right, that we all hope, is that when we balance it all out, we're doing more good harm. But I just ask the group never to forget the kind of downside to the results. Great point. Great point, Simba. Related to that, you know, when we talk about false positives, I was thinking as this is being discussed, we need to make sure that we're distinguishing that from non-penetrance. So some people just blend those two together. And so biologic evidence that a variant can cause disease is pathogenic, is different than giving back a variant that the data was misinterpreted and is actually not pathogenic. And that will hopefully help to clarify some of those issues. Yeah, I would just add to that. There's a wonderful quote I came by, came across when preparing the talk. Isaac Asimov said, the uncertainty that comes from knowledge is not the same as the uncertainty that comes from ignorance. And it's really important to remember that. And, you know, we in medicine are extremely good at washing our hands of awful things that happen because of our ignorance. And our only choice here is to make decisions based on no genomic information or based on our decisions based on genomic knowledge. And we can make better decisions with more knowledge. I don't think we should ever confuse that. Not to say that we won't make mistakes. And your question earlier, we have to do error mode analysis and say how many people are going to end up in this error mode and how many people are going to end up in that error mode. And we make rational decisions. We grossly, newborn screening is a great example, right? It has a very high false positive rate. And the whole system is built to work with that high false positive rate because we insist on the sensitivity being high because avoiding a kid having a devastating disorder like PKU is the outcome we really want to avoid. And so we way overdiagnose it and then we get rid of that in the follow-up testing. Not to say we should do that in genomics, but we have to say, what do we want for the false positive rate? What do we want for the false negative rate? And where's the right trade-off? And I would say that those trade-off numbers are completely different for FH than they are for HPOC. And we have to have that discussion and say, okay, where should we set those? Because we don't get perfection. Perfect is the enemy of the good here, but what's the optimal good we can get out of this? One more just basic orientation piece. I think that these types of questions have a very different context if this meeting was about, are we going to implement population screening or not? This meeting is about what are the research questions that need to be answered if we're going to proceed to population screening. So these types of issues where we may have fundamental differences, mostly those differences are being driven by, we don't have the information, which means they're really a high priority for research. And so for those of us that have been tasked to capture information, which gives me another opportunity to be criticized about how I'm doing here, we will be contextualizing these types of debate points into a research agenda. Jonathan. Jonathan Burke from UNC Chapel Hill. The first two talks are why I came here. This is so good. I'm also giving a talk, but this is thanks for the setup. One sort of commentary thinking about the two talks is given the evidentiary requirements for doing this and getting anywhere close to right. It's going to be perfect. I think it really drives us to the conclusion of starting with a very, very small and very well understood set of conditions. Just so for the ACMG committee to think about, let's not go to 70 right away. Let's try to focus it in. And I guess the other sort of point about that is if we're not really close to that sort of whole genome analysis and healthy people goal, as you kind of pointed out, what would you think about at least as a starting point for that considering whole genome analysis in consenting adults before we go all the way to whole genome analysis in all new boards. That seems to me like we could get a long way towards the goals of identifying all those risk factors and getting our feet fully wet before we start into new boards. So we have this interesting issue that will come up. We do a whole genome, and then we limit our analysis to the CDC tier one. Won't somebody take that data and say, let's go for secondary findings here and look for all other kinds of things. So then do you recommend doing just a limited data production to only look at those things? Well, then you're talking about wasted opportunities from a budgetary standpoint. It's a conundrum. One thing that I didn't mention at the front, which is worth mentioning is the ACMJ has been very clear over the years that the secondary findings list was not generated as a population screen list, even though everyone has assumed that from the beginning. But when you think about it, the secondary findings are from clinical data is generated within a patient provider context, where that provider, by ordering that test takes on the responsibility to see through the outcomes related to that test. So it's going to be very different when we're doing screening of populations and no one's really in charge of kind of seeing through all those variants. And we'd rather address that too. George. Mark, thanks. Actually, since I put up my card, my questions have evolved. And I'm not sure I remember the original question I wanted to ask, but I love that quotation about uncertainty and knowledge and ignorance. And I'll be honest with you, increasingly the challenge is distinguishing knowledge from ignorance. A lot of people tell us they've done their research and you dive into it. It gets a little difficult. So let me make it brief and thank Mark for reminding us of the core purpose for what this session is about. And maybe make a friendly amendment that as we address the research directions for when to screen, who to screen, how to screen, maybe also think of the engaging the people we screen and the research questions about implementation. I think they agree that very relevant if we could extend the discussions to that. And then finally, there's something about the automation and use and artificial intelligence to extend how much we can do. All for it. It's really terrific. The challenge we're also beginning to see is that there's a segment of the population that have no access to that. And so as we do more and more of that, we tend to exaggerate the disparities that already exist. What are some thoughts around how we will become that before everyone has full access to automation and AI and machine learning? Yeah, I would say I'm really a proponent of both opportunistic and population screening because I think as it's currently structured, I can't imagine a healthcare resource that is more disparate in its availability to patients than genomics and genetics. But the obstacles that people have to go through to get to see a person like me or Mark are ridiculous and only the most resourced, educated, advantaged people can navigate that. It's ridiculous, it's terrible, and that's disparity. And by making it as broadly available as we can, we have an opportunity. Not to say that it will eliminate disparities, it won't, not to say it will make irrelevant all of the other disparities in our healthcare system, it won't, but it will begin to level that playing field of access to genetic and genomic healthcare, and I think we have to push really hard for that. We have a lot to go through, but we're doing well. Christine. Sure, thank you. So as we want to continue to improve our understanding of, and our communication of risk and penetrance and probability, it seems that the screening is not going to be just a one and done type of deal with the screenings. We're going to have to continue to engage them so that we understand their outcomes over time. And I was interested in what Mark was saying about how, I think you said 50% of screenings are taking at least one action right after the results. But how do we, or what are the thoughts about ensuring that we continue to engage so that we have a fuller understanding of outcomes? I think the easiest way to answer that is to stay tuned because we have sessions throughout the meeting that are going to be really focused on that. So if you're willing to kind of put your question in the parking lot, hopefully we'll get some answers and some interesting research opportunities for that. Is that, that's not an answer, but it's practical, I think. I'm going to go to Terry because there's an online question from April Adams. Because you put, well, yes, I mean, because I'm not in the meeting, so. Can you hear me? I actually just had a comment. I think it's similar to some of comment that was put in the Q&A. And just as we're kind of going through this process and thinking about implementation and how this is going to be widespread and equitable is just, yeah, remembering when we make comments like who is motivated and who is not, remembering that those barriers for motivation may really be set in the healthcare systems and the structures that we've created, right? It may not be the onus on that individual. It may be that our healthcare system has prioritized people who have resources, people who have high health literacy, you know, and not prioritize people who are from marginalized populations. And so I think starting in a room like this with lots of stakeholders and having language that is not putting all of that onus on that individual is a really big place to start with that trickle down of why does equity matter? Why does diversity matter? How are we going to actually engage people in a way that doesn't make them feel more marginalized? And we'll probably talk more about this as we get through the meeting. Yeah, I would just add, we do have, I think, an outstanding session this afternoon that will address at least some of those issues. So stay tuned. I know there was two down here. So here, props, yeah, let you go. Thanks, I'm Kelly. He's from Hudson Alpha. This kind of circles back to, I think, Heidi's comments earlier about patients and bringing their kind of preferences into, ultimately, what turns into outcomes that we can't necessarily script that, that there's all these other variables at play. And I guess just wanted to bring up the additional variable of patients believing the results that they get back and when it does or doesn't corroborate with their story, their narrative, their history. And we bumped into people that get a negative result and they flat out say, well, I don't believe it. I think that's wrong. Or vice versa, they get a positive and they say, well, I don't believe it. And that's going to play into those actions that are taken in those conversations. Sometimes we can change their minds. Sometimes I'm convinced we don't change their minds on whether they believe it at the end of the day. But as we scale things up, just making sure that we're kind of looking for ways to improve that belief and have those narratives discussed and approach that with populations in more scalable ways. I don't know the answer to that, but that's going to play into the ultimate impact and benefit of patients. Thanks for that comment. I have two probably irrelevant questions. One was, Mike, you mentioned at the end this idea of monogenic psychiatric disease. Maybe you could amplify on that because I'm not sure I know what monogenic psychiatric disease is. There's a whole polygenic psychiatric disease problem that people are using polygenic resource to find people with schizophrenia or depression. And that's going to be an interesting implementation issue. That was sort of an information question for me, and maybe I'm the only one in the room that doesn't know what you're talking about. And then the other question was for Les, and it may be that we'll deal with this later. But if you're... I love the Bayes theorem sort of approach to this. It makes perfect sense. If you have a patient who has a phenotype and you're then going to... And the phenotype could be cancer susceptibility. That could be a phenotype. Or the phenotype could be something that you actually put your hand on, like cardiomyopathy or something like that. Maybe this is a question for actually Jonathan Berg too. How far into the long, long, long list of potentially causative or associated genes does Bayes tell us to go? If I have somebody who had cardiomyopathy, there's four genes that I might want to look at that have 10 genes or something like that. But then there's another 150 that somebody wants me to look at. And how do I sort of balance that? How do I approach that? And I'll tell you why at the break why I'm asking this particular question. But I'd be interested in your thoughts. So it's like a monogenic psychiatric disease. Yeah, so I will send that reference around. I've been approached by people, Yale and now at Sinai that are interested in looking into that. But it's a good point. What do you mean by that? And I think we're also, you know, focused on single and nucleotide variant here. But the reality, and this gets into the polygenic, if you will, is that we can do cognitive screening and there's publications out of Geisinger that have shown that we are not doing diagnostic testing. A lot of people carry known pathogenic copy number variants like 22Q11.2. And the vast majority of them is not 100% of them have neuropsychiatric complications related to that. And so if you want to constitute that as a monogenic screen disease, I think there is some evidence of that. Okay. Your second question is awesome. And in fact, we're incorporating that concept is in an upcoming paper and it will be in the version four of the ACMG amp recommendations, which is that negative evidence, if you have a phenotype that has locus heterogeneity, negative evidence at one locus comprises positive evidence at other loci. That was not in version three of Richards at all. But that's what you're saying, right? Because when you start once, was it, who was, was it Sherlock Holmes? Once you've eliminated the impossible. Whatever remains forever improbable. Right. That's it. That's basic. Yeah. And so I, you know, that's why I'm a genome fan because I want all the variants. And then I want to do a probabilistic assessment starting with the most likely culprits. And once I've excluded them, I'm walking down that curve of contribution until I get to the most likely answer. Does that make sense? Sort of. It doesn't tell you when to stop. Because once you have the bird in the hand, the question is, do you keep looking in other bushes? That's an interesting question. Bays won't tell you whether or not to do that. I'll just tell you what the likelihood is that you might find something else if you keep looking. Do I start with the 10 genes or do I start with the 200 genes? I think you should always have all 200, but you should start with the 10. I like that. Jillian Hooker, our concert genetics. I want to go back to the conversation that Heidi started less as response and some colleagues comments about genetic counseling and patient preferences and even the scalability of genetic counseling. And first point out that from a numbers perspective, genetic counseling is one of the most rapidly growing professions and the numbers are a lot bigger today than they were five or six years ago when we were talking about a shortage. And I think many folks have sort of acknowledged or sort of accepted a shortage as a truth without actually like digging into what's behind that. And in fact, I think many program directors right now are worried that perhaps they're training too many people given the contraction in the lab market that they're increasingly worried about the job market for genetic counseling. And so what we have then is not so much a shortage, but a distribution problem of where genetic counselors are, where positions are, which is related very much to the finances of it, which is sort of covering these positions, which are also on their way to getting a little bit better with a new CPT code slated to come out in 2025 that I think will really, really help in that situation. And then to the like we can't scale to individual level to our going on to engage in screening. I think already we're doing that in MRI and echocardiogram colonoscopy. These are individual provider patient interactions, most of which cost far more than a genetic counseling appointment. And so I think there are economic models that might be more sustainable for follow up to population level screening where genetic counselors could actually save costs in a way that would make them more scalable. And so I think we're going to be able to do that in the future. Unless I'm missing something, which I guess is my question, this is their piece I'm not seeing there. Labor will always be the most expensive part of healthcare. And anything that we can do in healthcare that supplants an hour of any provider's time with some technique, some technology, some non-human interaction that can do that, it can do that. And it does, will save money. And you're right that genetic counselors per hour are less than geneticists, but they're still real money. And what do you think an hour of genetic counseling actually costs the healthcare system in totals? Probably $200 or $300? We'll have the answer from the rock. We haven't done that yet. There's a study going out right now. That's actually being calculated as we speak. I think that's a great question. And it will most certainly be less than the cost of an MRI, an echocardiogram, or a colonoscopy. I think that no one would debate that at all. The other editorial comment I just make is that numbers are one thing, distribution is another. And so the distribution of the resource is also something that's important to consider. But I think this will be a great research question, which is how do we actually efficiently deliver this if in fact we move it forward? I'm going to turn it back online to Carol Horowitz. Hi, everyone. I'm so sorry I can't be there in person. Following up on this conversation. I think it's not just how many genetic counselors and the distribution. I think more research is needed into when we need genetic counselors. I was struck when we have now tested many thousands of folks for April one variance. that almost none asked for and had any genetic counsel, even though it was offered to all of them for free. And I think we need to think about when this is, as Terry Monoglio, you mentioned many years ago to me, is able and like a pre-atni test, do we need a genetic counselor to go through that? And how do we start looking at all the different tests we do and what the resources are needed to give good quality equitable care that's accessible to many people? We have Bruce Korf online who has a question. That's gonna be the last question. Yeah, I'm one of those that had an unexpected encounter with an unwelcome small genomes. I'm sorry, I'm probably about 300 feet away from you all, but that's probably what you want right now. Actually, I was gonna make a comment, which is a bit delayed. It was, I think Kate made the point about finding NF1 variants in people who have had cancer testing, usually breast cancer testing. And we've seen a fair number of people like that who most of the time don't have any signs or symptoms. Occasionally they do and they just didn't understand what they were or some of them may be mosaics, but some actually really don't have anything that you can find and most of them are older people. I think it may turn out to be the case. We haven't proved this yet, but that some of them may have clonal hematopoiesis. And we're also doing an analysis of all of us data now and we find a pretty substantial number of known pathogenic NF1 variants where there's no trace in the record of any clinical diagnosis. And I think I'm not sure they all have that because no doubt some of those records are incomplete, but I suspect some too may have clonal hematopoiesis. And although maybe a small point is something just to keep in mind as we look at older populations, if that's what we end up doing with screening. One other point, which is a little bit, which is based on comment just made a moment ago about genetic counseling. You know, I take the point about the distribution, but I would also make the suggestion that we will never ever have enough genetic counselors or medical geneticists to see the people who are involved in screening and that we need to think in different terms in looking at ways to communicate information. I personally think that the genetic counselor of the future will include artificial intelligence-based sort of chats that people will be able to converse with and get information as these systems are improved, which no doubt they will be. Thank you. Any comments from the speakers? Last comments you want to make? Less is good, Michael. Okay, well, first of all, let me thank both speakers for stimulating our thinking in this opening session. Lots of great questions from participants. We did leave some 10 cards up. We recognize there's more interest than we have time for, but this is just the first session and I'm sure you'll work your questions into some future sessions. So we will take a break now and we will reconvene at 11.05 for session number two. Thank you. If folks could take a seat, please, we're gonna get started. Thank you. Well, welcome everyone to session two, Genomic Screening Technologies. My name is Erin Ramos. I'm the Deputy Director of the Division of Genomic Medicine at NHGRI and I'm co-moderating the session with Jeff Roscoe. Jeff introduced himself briefly, but he's a pediatrician and Director of the Division of Services for Children with Special Needs at HRSA. So I will introduce our fantastic speakers, try to keep us on time and then Jeff will mostly moderate the discussion. I moved over here so I can see folks on the opposite side of the room. We'll see how folks. Aang is our first speaker. Christine is joining us from the Department of Molecular and Human Genetics at Baylor College of Medicine. She's also the Chief Medical Officer and Chief Quality Officer at Baylor Genetics. So Christine is perfectly positioned to talk to us about technical approaches and logistical considerations for population-based genomic screening. Thanks, Christine. Thank you, Erin and thank you very much for inviting me to share some perspective on the current state of genomic sequencing from a technical and logistical framework. And I should say that the running title of this talk originally was Nuts and Bolts of Genetic Screening, which may be a more descriptive idea of what I'm gonna be talking about today. So I just wanted to start with a summary of things that we've learned, lessons learned from optimizing high-throughput genomic screening. First of all, the input is very important. So there should be a consistent DNA source and quality. We should also optimize and simplify the workflow with automation as much as possible. And this is especially important for library preparation, which has been somewhat refractory to automation, but there's been a lot of very good advances in that for now. Of course, it has to be cost efficient for the laboratories. And then also there should be a focus on continuous improvement. So there should be continual upgrade process, not only with the testing for laboratory protocols, but also for the analysis. And this can present some challenges for the laboratory because with every new major change or not even so major validation, testing and validation is necessary. So you wanna be very prudent in terms of deciding when and which advances you want to introduce to your workflow and that they must be durable. You also have to define your metrics. These are pretty well understood for next generation sequencing, but metrics are very important to monitor throughout the process. And also you want to monitor them long-term. And I'll be speaking about ways of doing this by looking at the results and the data that are being generated. Analysis, I think we touched upon this a little bit earlier. I'm sure we'll be talking about it more. It must be automated, but there's always going to be a manual component to this as well. And in order to keep your database updated is of the utmost importance, but also to communicate how often that database is updated. And then finally, we talked about the input, but what about the output? There must be clear communication of reporting practices so that providers and patients and screenies understand exactly the result that they're getting. So I just wanted, I am Cleo Lab Director for both Baylor Genetics and the Human Genome Center Clinical Laboratory at Baylor College of Medicine. So I wanted to talk a bit about requirements to launch a clinical test. And I'm gonna focus mainly or mainly exclusively on LDTs, but as we all know, there is more discussion about oversight of laboratory develop tests. First of all, it needs to be performed in a clear environment, a clear certified laboratory. Cap accreditation of course is preferred as well. The indication for testing needs to be clearly defined as well as elements on the requisition such as consent for additional testing or research that we touched upon earlier today. Specimen requirements, the specimens that are acceptable for the test and have been validated for the test. SOPs have to be generated and there has to be a process for validating your reagents, your instruments and your vendors on a regular basis. In terms of the test design and the validation, the technical limitations of the test must be determined and disclosed with the rest of the description of the test. Asset interpretation also needs to be fully disclosed. Your rationale for validation, and this can take many different forms, but especially with the different variant types that you're going to be reporting on. So SMDs, copy number variants if you intend to report on those. Your clinical reporting criteria, again, the limitations of testing and reporting and then importantly, the post launch evaluation. So these tests, once they're validated and launched, they need to be continually monitored for performance as well as for the results that are given. So if you are expecting a certain population frequency, a variant or a gene, you must continually monitor your results to make sure that there are no surprises and that will address some of the false negative, false positive reporting that we've been touching on. So clinical test validation, and this was discussed in the first talk, there is analytical validation, which is of course your accuracy, your precision, your sensitivity, your specificity, reproducibility, limits of detection and the choice of your validation samples. So for a large panel, you must have positive controls and validation samples that are going to be a good representation of the genes that you're going to be assessing, especially the more common ones. And then of course the clinical validation. So what is the purpose of your test? Is it newborn screening, is it family planning, is it wellness, the evidence for including those genes and the action ability that is associated? So in general, these are the stages of validation of a clinical NGS test. The test design, the development and the optimization, and we talked about, I touched on some of this earlier, the test scope is very important. So what is the endpoint of your test? What is the information that you're hoping to provide to the patients? The samples types and the turnaround time. So this should be very clearly communicated to participants so that they are not waiting for their results and they can expect their results within a certain timeframe. The wet lab workflow and any automation or if you can automate the whole process, the QA metrics and performance need to be determined. And then of course your pipeline needs to be established. You may need to develop additional modules for some challenging regions, both on the analytical side and on the technical side. And I'll talk about this a bit later. Your variant confirmation approach. So are you going to buy an orthogonal method or are you going to rely mainly on the metrics of your NGS analysis? The test validations I talked about needs to be very thorough. And we should also remember this in your pipeline, both wet lab and analytical is going to be exposed with high throughput. So you have to make sure that these processes have been tested and stress tested as much as possible with volume, with difficult regions and level of detection. And then the test performance needs to be monitored as I've mentioned with QA metrics established and tracked. And also importantly, sample identity, contamination to ensure that you're not having sample mix up. And I'll talk about this a bit later as well. So this is a very, it may sound simple but a very important logistical concern. So the sample collection, what type of sample is going to be accepted? These are some of the considerations in deciding which type of sample is going to be used for the test. Participant convenience is of course very important, at most importance. The laboratory should provide kidding. So whatever materials are needed to gather the test sample should be provided in a kit that's going to be provided to patients, as well as detailed instructions for self-collection. And I'm talking mainly about where the patient is going to be collecting their own sample, not where the sample is collected, let's say in the laboratory setting of a medical center. Very importantly, there should be bulletproof labeling as well as downstream matching to the patient's contact information. This sounds fairly simplistic but this is one of the major sources of error in a clinical laboratory that labeling is not performed accurately, especially if you're testing partners, multiple family members at the same time. Samples come unlabeled, samples come with labels of the other person who is being tested at the same time. The cost obviously is very important, the ability to automate whatever sample you choose for DNA extraction, but most of these considerations have already been solved. The sample stability during shipment and the time to processing. So they should be able to withstand mailing through the postal service and some delay from the time that the sample is obtained to the time the sample is processed. The failure rate of the chosen method should be assumed as well. Of course, whole blood is the gold standard with I think still the lowest failure rate. Saliva is a little bit higher, maybe about five to 6% or though this can vary based on the quantity of saliva obtained. And then also the ability to store the DNA long-term. So it's stability long-term if a bio repository is to be part of the testing process. So these are sample collection options from non-invasive on the left to invasive on the right. I just put some visuals in terms of the types of instructions that are provided to patients. I think the dried blood spots is one of the ones where you have the card and you write your name on the card. So I think there's less possibility of sample mix up there and misidentification, but I was struck by the differences between the non-invasive and the whole blood method. So if you look at the whole blood, there is a healthcare provider that needs to collect the blood and there are multiple types of medical devices. So there's the needle to pierce the vein, there are the tubes, but then most importantly also there's the bio hazards that are produced. So needle sharps and others that all have to take into consideration as resources needed for whole blood. This is taken from paper, recent paper by O'Brien and colleagues and it's from the Oregon project for population screening of inherited cancer and familial hypercholesterolemia. No, it's difficult to see this, but this is basically their workflow from their patient consenting to the patient asking for kits. About 25% of patients who ask for kits did not return them. Then the sample is processed in the laboratory and about 5% of those samples and this was mouthwash failed the DNA extraction step. And then taking the sample through NGS reporting in this particular example, they did choose to retest a positive. So, saliva kits were sent out to presumptive positive patients and another sample was obtained for orthogonal confirmation and this was done by Sanger. But just an example of a workflow that was in place for this project. Choice of platform for a high throughput testing. So just a couple of examples here, genotyping arrays I think were popular maybe a little bit in the past. UK Biobank project had an array of 800,000 markers and this is perhaps some better suited for biobanks, genome centers and other core laboratories. All of us is using a GDA array as part of their testing process. So they're reporting on ancestry from this array as well as doing concordance. Arrays are cost effective, high throughput, low failure rate, but of course there's less flexibility after the design. Targeted NGS panels can be used as well. Examples are universal carrier screening panels, hereditary cancer panels. There's less data produced which will allow you to have higher coverage and this can result in your ability to accurately call CNBs. But of course there's less flexibility after design and a lot of work and curation that has to go into designing these panels. WES has less data, less cost than WGS. Of course you have the ability to reanalyze but you may be missing some regions that are important for PGX and potentially also PRS. WGS hypothesis free ability to reanalyze the highest cost and the highest amount of data. And then there are some hybrid designs that we heard about last week at ASHG. This is a low pass WGS which can be combined with a relatively low pass or less than a clinical grade WES. Example of an NGS workflow. So the blue is a technical production again to automate this as much as possible. The green is the identification of DNA variants. This can be automated. And the yellow is your tertiary analysis and there are a lot of tools. So this is your annotation, filtering, prioritization and then for more diagnostic tests your variant prioritization. To some extent, this is becoming more automated but there is a manual element as well. Just wanted to- Just to interrupt one minute left. Okay, we do have ability to automate some of these processes and for population screening we rely heavily on the existing databases. Quality metrics that can be applied of course there are quality metrics that can be applied to the initial nucleic acid samples after library preparation. Primarily it's looking at insert size, post sequencing their pass fail metrics including the sample identity and then the post sequencing monitoring as I mentioned before including what I think is important the periodic review of your positivity rates to make sure that you're capturing both your positives as expected as well as not over reporting. Just wanted to give a brief example of our carrier screening panel that we do in our laboratory. This would be a tier four as designated by ACMG. So one point that I wanted to bring up here is that typically these panels are not uni-directional. So it's not just NGS that you're doing. There are difficult genes but ones that have high clinical utility such as Fragilex, FXN, CYP 21A2 for congenital adrenal hyperplasia. These cannot be accurately assessed just by a uni-directional NGS. You have to have as we do in our lab another workflow. So for CYP 21A2 we do a long range PCR and then we spike it into the NGS. So you have to have separate assays and then you need to join your assays together. You also have to make sure you're taking that one sample through all the different workflows and you're not having any sample swaps there. So sample identity becomes very important. And then we do orthogonal confirmations for specific genes, especially the more challenging ones such as SMN1. I am going to... So return of results, we're going to have a lot of discussion about this but of course critical to clearly communicate the result parameters to providers and participants. And then the re-contact, re-analysis and possible subscription has to be defined in advance as well. So just in summary, I shared an overview of the current state of clinical methods for population genomic screening. There are clear distinctions between the reporting for diagnostic versus screening genomic tests, typically pathogenic likely path for screening. Of course, the US is for diagnostic, but I wanted to make the distinction that the laboratory quality measures are not distinct. They must be exactly the same for diagnostic testing as for screening. And that can put some challenges on a laboratory because of the high throughput nature of this type of testing. Thank you. Thank you so much Christine. You were asked to touch on a lot in a short period of time. So thanks. So our next speaker, Bob Kerrier, spent more than 20 years as the chief statistician of the Genetic Disease Screening Program at the California Department of Public Health. Although the focus of this workshop we heard earlier is on adult individuals, the lessons learned from newborn screening I think can and should really be factored into our discussion. So we're grateful to have you here today, Bob. Thanks. Thanks for inviting me. And I have no financial interest to disclose the opinions presented on my own and some of them are pretty strongly held. I should have put up the disclosure of a paper that I published a little bit ago called whose title is Newborn Screening is on a Collision Course with Public Health Ethics, which appeared in the International Journal of Newborn Screening. Anyway, why focus on newborn screening? The main reason is it's already the by far the largest genetic screening program in the country. And in addition, the newborn period is perhaps a unique opportunity to intervene in genetic disease before symptoms develop. So one might say, well, why don't we just sequence everybody, all of the newborns and identify all the treatable disorders and get going? The goal of this, my part of the talk is to just say what a bad idea that would be. But for people who are less aware, newborn screening starts with the collection of the blood card about at one day of age. And usually the parents have little or no involvement in that. So newborn screening as a medical test is unique and not being consented. I'll avoid all of my rants about that in the talk. But it then goes to the laboratory where there's a lot of biochemical analysis and then the results are reported relatively quickly. They need to be reported quickly because the target disorders are serious and urgent. I will say on the way by that this first step of doing the biochemical analysis radically changes the prior probability of the, or the posterior probability of disease among the positives is much higher than the general population continuing the base that's theorem series. So this is a state program. And so in the context of public health screening, generally the state has the usual public health ethics considerations. I want to underline at least three of them that newborn screening really is universal. Essentially every baby in the country is screened, every baby, four million a year. And as a universal program, it also applies to everybody regardless of race, ethnicity, socioeconomic status, insurance coverage, anything. It always happens. And a part of that is that the state program needs to be a trusted partner in the process. And when we come to genomic information, this becomes its own kind of challenge. For newborn screening, the choice of disorders is really important. Because the parents haven't been involved, the disorders need to be certainly serious. The state justifies it by saying, this isn't the best interest of the child and we have to move forward. The baby could have a metabolic crisis at two or three days of age if the MCAT isn't diagnosed. Of course, the conditions screened have to be treatable. But along the way, because the state is screening everybody, the goal is detection of all of the disease. But the screening test needs to have a low false positive rate and consequently a high positive predictive value. On top of all of this, this is a hugely high throughput result. The state of California tests 1500 babies a day. And that all has to happen promptly. I don't think genomic testing is in that ballpark quite yet. But single gene or small numbers of gene sequencing is used in newborn screening a lot after an initial positive biochemical test. And it has two functions. One is to reduce the false positives from the biochemical test. Or the other is to aid in the subsequent differential diagnosis of the positive result. Well, let's take a look at a couple of examples. This is a schematic of cystic fibrosis screening in California. The first box there is the biochemical testing. And only about a percent and a half go on to further testing. So it's already weeded out a lot of people. The first tier is actually a mutation panel. At the time of this paper, it was only 40 genes in California. It's now up to a hundred. But on the way by, I wanna point out that in addition to the usual snips and small indels, cystic there are a couple of whole exon deletions that are sufficiently common to be important. And there's some deep intronic variants. And all of this comes from knowing what's going on with many, many patient samples from CF. In the, if two mutations are identified from the panel, it goes on to the diagnostic test for CF. If there's only one mutation that goes on to sequencing the whole gene. And in that, I wanna point to the, if two or more variants are found, including any BUS or anything, one of them is known pathogenic. So it goes for diagnostic testing. And I will point out that if you compare the panel results, the number of CF cases, that's the N equals 138. And in the results from the variant, the sequence saying that almost the same number of CF cases were identified at those two stages. Turning to Adreno local dystrophy. In this case, the screening test is considered positive because the baby, if it passes two tiers of biochemical assays, at that point it gets referred for diagnostic follow-up. But at the same time, the relevant gene, ABCD1, is sequenced to help among other things distinguish between ALD and other peroxisomal disorders. So, but now let's consider what happens if we started to do primary population screening. That is, just go directly to genetics sequencing. There are a lot of disorders that look like candidates for newborn screening, except there's no biochemical test. There's their relatively early onset. They have clinical follow-up, but there's no test. And the group at UNC has identified over 400 gene disease pairs that would be potentially suitable for genetic screening. Their definition of suitable in mind of early onset are a little different, but okay, fine. But, and this I think is, to me, one of the really important things to think about in genetic screening, that when there is a diagnostic test, you can afford to consider all kinds of variants, including VUSs, that's what the cystic fibrosis model did, that you have one known pathogenic variant plus something else, let's check it out. But when there is no diagnostic test, then you're really relying on the genetic result itself, not just to predict the genotype, but really the goal is to predict the eventual phenotype. And in that case, you have to just stick to what you know, which is probably, I keep thinking about autosomal recessive disease, because essentially all of newborn screening is that. So you really have to just refer, like homozygous cases of a single, of a known pathogenic variant, but given that what's known about the pathogenic variants, that starts to impact your equity. And so this is a really difficult thing. Every single gene is its own screening test. And so you have to know what you're looking for. Some diseases like crepe disease is commonly caused by a large deletion. So depending on the disease, you have to know, you have to know when you have to be able to find copy number variants. And enough about the US is, one of the things that goes with this, because the knowledge base isn't uniform across ethnicities, is that there can be racial, ethnic distinctions in screening. And we'll take a look at one of those. The thing that struck me about this paper was that the number of variants that were found did not seem to vary according to race or ethnicity, but when it came to interpretation and identification of pathogenic variants, the differences were significant. And I think this is probably not a surprise to anyone. I keep focusing on autosomal recessive disorders because it strikes me that the notion of pathogenicity doesn't belong to a single variant in this case. It's the combination. So here these are PAH variants, the causative gene for PKU. So in each group, we have two homozygous variants and the compound heterozygote between them, and then looking at the frequency of various levels of disease. And so for the two groups, the left group and the middle group, there's one bad variant and one, not moderately pretty okay variant. The resulting compound heterozygote is in one case not so bad. And in the other case, almost as bad as the worst variant. The group on the right, I think is the most, it's not troubling and it shouldn't be surprising, but in this case, the compound heterozygote is worse than either of the homozygots. And it just reminds us that enzyme structure and function isn't a linear combination of the variants that went into it. So it almost goes without saying, but compared to current newborn screening methods, genomic sequencing is much more expensive per patient. It's reduced sensitivity and specificity. And it's which is exacerbated by racial ethnic differences. This secondary sequencing is really valuable, but that's not really what we're here for. And there's this promise of newborn screening for all of these other disorders if we could use sequencing. So here's my list of things that really need to be worked on. Many of them are in progress. There are people here that know more about this than I do. I really would love to have screening tests in place that distinguish between soon onset and down the road. Bob, sorry to interrupt, about 30 seconds left. Holy moly, this is the last slide. So let's just say this. Diagnostic testing would allow the inclusion of more VUS. Sharing variant data, not in individual laboratory silos, but across the world really. The interpretation of compound heterozygots. Can we automate that? I don't know. And there needs to be guidance, real guidance on pre-symptomatic management of genetic disease. We don't, I think my impression is that we don't have a good sense of what to do with cases that are identified with a genotype that don't yet have a phenotype. So, and thanks. Thank you so much. So our last speaker of the session before the discussion is Jonathan Berg. I know Jonathan quite well. We've worked together on ClinGen for the past 10 years. Jonathan wears many, many hats at UNC, one of which is directing the program for precision medicine and healthcare, which has implemented a clinical service offering screening of CDC tier one gene. So Jonathan brings lots of experience to discussion. Take it away. Well, thanks for inviting me. And I think I'm gonna pick up some threads from some of the other talks. I'll try to give another way of thinking about this. And the way that I've been kind of cogitating around this is really thinking about the number needed in genomic screening. So there are a couple of well-established sort of terms that are used in medicine, number needed to treat being individuals who already have identified risk factors. How many of those do you have to treat with something to prevent that poor health outcome versus number needed to screen, which would be the number of people you have to screen for those risk factors to find the one that you're going to essentially help by preventing that adverse event. So all of this as noted previously, we're talking about the risk for poor health outcomes, right? Monogenic disease, risk for poor health outcomes, polygenic risk for poor health outcomes, however we wanna formulate it, that's the concept. And can we use that similar logic to examine genomic screening for monogenic diseases that convey high risk? All right, so the first part of this is gonna talk about test performance and population prevalence. I probably don't have to go into too much detail about the way we classify variants, but just to put it a really fine point on it, this scale from benign to pathogenic is about probabilities. And there's a really sort of important zone there in that orange to red where we're talking about high level VUSs and likely pathogenic variants that are particularly useful in a clinical context when you have a diagnostic workup as less pointed out, but also our potential false positives, especially when you start thinking about that low prevalence population. So we're gonna, I'm gonna show you my version of math based on images here in the next couple of slides. So if we're gonna start with a population of blue people where there's a few orange people in there with a monogenic disease, the prevalence of that is fixed. It is what it is in our population. This example is approximately one in a hundred. That's gonna be at the best and in terms of the prevalence of the diseases we're talking about. So this is an optimal situation. And of course we have the test performance, which we've already talked about as being tunable, right? Which types of variants are we gonna include as positive screens? That's gonna yield us a population of patients, some of whom are the people with disease and some of whom don't have that disease but have been picked up because of the way that we set that tuning. And so then we can calculate things like the true positive rate, the false positive rate and sensitivity of the test. And I've given this one an 80% sensitivity. And we can also calculate the false positives and they're in the specificity, which in this one, I've given a 99% specificity. And you can kind of see how those numbers relate to each other. Of course, this is just determining whether an individual has or likely has that disease, right? With all of the caveats about their prior probability to have it, not whether they're going to develop the symptoms of the disease. So we're gonna get there in just a minute. All right, so let's start with those groups. Now we can do our math, right? So you've got your four true positives divided by all of the positives. That's your positive predictive value. So in this particular example, we've got 67% positive predictive value. That's gonna be very good performance compared to the prevalence as we'll look out for other conditions. And the negative predictive value is going to be very high, almost no matter what sensitivity we choose to use as our threshold, because there's so many people in the population and because these conditions are so rare. So what then is the number needed in this case? So I'm gonna use a number needed to diagnose, and I use the term diagnose as a molecular diagnosis, if you will, probability of a molecular diagnosis, that one true positive person. And that's gonna depend on the sensitivity. And so this is an example table, just basic calculations of the prevalence of the condition and then setting out some kind of basic clinical sensitivity and specificity values that relate to each other in terms of the fact that if you're going for 100% clinical sensitivity, your clinical specificity is gonna get pretty low. And if you go for the maximum clinical specificity, your sensitivity is gonna suffer, but that's the trade-offs that we're gonna have to make. And so you can see with a one in 250 condition like perhaps HBOC, something in that realm, if we're going to be going for a midline clinical sensitivity of 90% with a specificity of up to perhaps 99%, we're still going to end up with about three false positives for every true positive. There's just gonna be some leakage of those people into our population. And it gets much, much worse, the rarer the conditions get. You almost can't do screening for a one in a million condition with anything other than the absolutely well-known pathogenic variants or you'll just wind up with lots and lots of false positives. Okay, so what are some critical values? And this gets to the research questions. We have to get to the prevalence of these monogenic conditions. So we know where to start from in terms of those priors. Most estimates we have are pretty much hand-waving guesses. This is about a one in 50,000 condition or whatever, but there is nice population ascertainment now from biobanks where we can look for the path of likely path variants. We saw some examples of that earlier. We could use those numbers to get a baseline ballpark, but of course there's gonna be some questions about whether those people are actually truly having that disease or not based on that ascertainment. But that could give us at least a lower bound estimate. All right, here's another way to think about thresholding the clinical performance of genomic tests. So I'm thinking about this, not now as a single variant that's coming back, but as the population of variants that might come back from a given test. And if you had a distribution, something like this, where it's almost a frequency histogram of number of variants reported, if this were the characteristics of your test and you had a couple of sort of the higher level likely path and path variants that gave you a pretty high percentage of the cases overall, you could get a reasonable sensitivity at very high specificity. But then you'd have to ask yourself, is it worth gaining a little bit of additional sensitivity down into that orange realm at the cost of some reduced specificity? So how much value do you get out of pushing your threshold down? Obviously the better we understand pathogenic and benign variants, the better catalogs we have. Shout out to ClinGen. If we can eventually get the MAV data to really help us classify these variants and separate them so that essentially we're pushing things to path or pushing things to benign, then we will do better with our predictive ability. Oh, and I would just point out, I think this is an area where we could ask our clinical labs to help us with this. So for people who are clearly have a high, high, high prior probability of a given monogenic disease, what kinds of distribution of these variants do you see in that population? Would be an interesting way to look at that. Okay, so the clinical performance is gonna be something we need to learn and understand to know, to calculate the number needed. If you have a variant that's responsible for all cases, then you can really go with that variant. That's gonna give you a lot of information. But obviously most diseases have a much more complex mixture of variants. And that's gonna cause us some problems in terms of really estimating what our sensitivity and specificity is. And I would also add that the complexity of the combinatorial diplotype of recessive conditions and which two variants an individual has even greatly more complicates that. Okay, so part two is about penetrance, actionability and preventing for health outcomes. This is why we're doing the screening. So under normal risk management for that whole general population that are true negative, they're getting their appropriate routine management, they're individualized by family history, et cetera. They're getting average population outcomes, that's great. The one individual that we missed on our test as a false negative is likely getting inappropriate routine management. We might not be able to help that and they do have the opportunity for clinical diagnosis. So there is still a safety net for that person in the sense that they're probably getting at least some amount of medical evaluation and follow up for things generally. Under high risk management, we're gonna have our two false positives who are getting inappropriate high risk management that gets to the harms that are likely going to exceed the benefits for these individuals and they'll have below average outcomes probably. If we look at our four individuals who are positive, now you have the advantage of cascade testing, I'm not gonna talk too much more about that but within those we have what we previously talked about of disease penetrance, right? This is a fixed value for the condition and some of the people will be non-penetrant. That's the one person that kind of comes up to the top there. I refer to this as over diagnosis. They truly do have that monogenic disease. We're just gonna be treating them more than they need to be. Whereas the three people in the bottom circle there are gonna potentially benefit from the high risk management. So these are gonna be the will be penetrant people and our goal is for them not to become penetrant. We want them to not develop that disease either because we've treated them, prevented it, picked it up early, et cetera. We wanna convert those into light blue dots. We're not gonna be successful all the time, right? So in this example we've sort of prevented that poor health outcome in about two thirds of this population likely gonna benefit those individuals more than we harm them and hopefully have above average outcomes. Now this non-penetrant overdiagnosed person goes in with the false positives. This is someone who we are not helping by all of that high risk management. They were never gonna develop disease anyway so all we can do is harm them by whatever it is that we do. So this is a way to calculate the number needed to treat. In this case there are six people being treated. Only two of them benefited. We get an NT of three, right? And so these numbers could actually be calculated through some of our population research to figure out what that actually looks like. Timeline is also important. If we're screening prior to symptom onset then we have a greater opportunity to mitigate those harms. We're gonna potentially have more people that we can prevent that disease in but if we're starting the screening coincident with the onset of disease, well some of those people have already had that disease. It's not gonna change their outcomes at all. And we saw that some of that example in the Biobank data. If we're starting the screening after symptom onset then the best thing we can do is identify people that should have been diagnosed anyway and potentially improve their family health through cascade testing. So what do we need to know? We have to know the penetrance of monogenic disease for each of the diseases we're screening for. We obviously have ascertainment bias from the affected populations that we've studied so far. We're gonna probably have a much lower penetrance for population screened patients. That's gonna increase the number needed to treat since the proportion of individuals that would benefit are gonna start to go down. I think we also need to better characterize the age-based natural history. When do symptoms start? When do people start to develop the poor health outcomes? And when can we time the intervention so that we pick up those people before that happens? Okay, so a couple of strategies. One is to follow up the tests, right? On all positives, like we have a gold standard test we do. Well, that's gonna cost money and we need to calculate that into our cost of screening. But we could potentially resolve some of those people as false positives. They are actually. And then they go on to get appropriate management. There's the issue of how to deal with this incomplete penetrance, right? Well, one way to do that is through what I've referred to as proximal surveillance, some low burden way of following people, maybe generating some additional data, looking at their family history more closely, finding out a little bit more about their risk and characterizing it so that some of those people are gonna go on and get low burden care for their lifespan and not get something definitive done, whereas others might trigger a definitive intervention because of a phenotype that develops, right? There's a slight widening of the aorta. Somebody's following that. It hits a point where they then need the surgery, that kind of thing. On the other hand, you could really work on the refined risk assessment and really try to triage people into those who really don't need a whole lot of additional management and those who do need whatever that definitive management is. And this might be the sort of thing that you would bring into the decision making about something like prophylactic surgery, right? How are we gonna decide which people to remove an organ from based on that risk? It's a medical workup and decision making and I think that's gonna be costly. And I'm not sure we're capturing that effectively in our current estimates of cost. So essentially we need these strategies and they need to be defined for each condition that we're gonna be doing screening for and we need to build this into the cost of a screening program so that we're not just thinking about how cheap it is to do the DNA sequencing but we're thinking about all of the costs of the management. Okay, so critical value to know. This is another way of saying clinical utility. I'm taking action ability and making it quantitative. What is the quantitative action ability of each monogenic disease? Donathan, one minute left. Got it, I have one slide. So how much reduction in morbidity and mortality will we expect to get, right? Can we really put our hands on that? How effective are those strategies to reduce the false positives and mitigate the overdiagnosis? And in the absence of any controlled trials or 20 year follow-ups, how are we gonna estimate what that number needed to treat is really to achieve those, to reduce the poor health outcomes? So going back to my toy example, again, this is sort of a best case scenario because it's a fairly high prevalence condition. I've given it pretty good test performance characteristics, estimating the number needed to screen. We've got about 125 people to find one diagnosis. The number needed to treat is about three. So we would need to screen 375 people to find one person who we're gonna help. And in doing that, we're going to identify people who are false positives and overdiagnosed and we're gonna do them harm. And so we have to make sure that we're balancing that as we think about these conditions. So the conclusions then, we need this key evidence. We need monogenic disease prevalence for the things that we're considering screening for. We really need to understand the clinical test performance and the spectrum of variants that we're getting out in each of these diseases. We really need to understand the natural history, the age of onset and the penetrance for these conditions in the population, not just in our ascertained affected cohorts. And we need these quantitative actionability estimates to know how effective our interventions are gonna be. And then based on that, I think we're gonna do what Les was suggesting is tune the thresholds for what variants get disclosed based on what the condition is and what we're gonna be recommending for the people that have it or that at least screen positive for it. And think about incorporating some of this into the cost effectiveness models to really give us a better sense of where in that process are the key features that we have to really tune to get the best performance out of screening. Well, I'll end there. And I think we have- Thank you, Jonathan. Thank you, Christine, Bob and Jonathan. There was a really neat follow up to what the first talks were this morning. Christine sort of leading us through what happens, has to happen in the lab. And Bob does what happens in the state lab and some of the concerns about that. And then Jonathan was a nice model for how to do it at a population level. I guess one of the things that was sort of missing a little bit taking off on my previous questions about the health system and how well it functions is also how well screening at a population level gets followed up. And so it hurts that we don't only fund the advisory committee in Newborns, which of course, Ned chairs and how the things get onto the rust. But we also fund states for doing Newborn screening follow up. And so that whole system, we didn't really say much about that sort of. So we start off Bob, I wonder if you could say a little bit about your experience in California about what it takes when you do population level screening to make sure that you follow up on all those kids. That's a key part of any large screening program. Yeah, thanks. So all of the Newborns that are identified with a positive screening result are referred to the appropriate specialist for diagnostic evaluation. And that, I think that happens essentially immediately and is really quite universal. I mean, again, it's part of the Newborn screening program. All of that communication takes a lot of keeping crack and monitoring. From after the diagnosis, things start to fall apart depending on how intensive the clinical management becomes and where the family lives. I mean, in California, there are families that live four hours from the nearest geneticist. And so they don't get the kind of care that people in other parts of the state get. And that's just geography. There's also language and, well, just general social economic status and then insurance coverage starts kicking in. So the gaps in long-term follow-up are real. I think I'll stop there. Thanks, Bob. I see Ned and Kailin and then Terry and then Heidi, which is for due. So I think this raises an important issue for thinking about the research agenda that we don't stop our process diagrams at we have a diagnosis and the clinician knows there's a whole lot of things that happen downstream from what we've learned from the white screening is if we don't calculate that into the number needed to treat, we're missing out on something important in our calculations. Ned. Yeah, I appreciate that. And Bob, I appreciated your discussion and I agree with you. I didn't want to come back to something less said that all those false positives, we just diagnosed them. It's way more complex than that. And we're doing research into the harms associated with reporting a false positive result back to a parent and the Odyssey that they have to go on to. So you don't have a disease that's gonna cure your child in the first year of life and you don't need a bone marrow transplant. I think the other thing you talked about is the immediacy of treatment. And so crab A is difficult because the bone marrow transplant has to happen so rapidly to be effective that there's worry about the chances to make a mistake in that diagnosis. So we're not supposed to talk about new screening. Let's think about how these apply to adults because it's not that much different. If you have a false positive result, I would argue that wrong information is always a harm and treating a false positive or an over diagnosis is also always a harm. And so it's always that balance. So Jonathan, the only thing I would add to your wonderful talk is calculating the number needed to harm. All the therapies we're talking about, all the interventions to follow up on a positive genetic test also carry harms. None of our therapies or approaches are harm free. And so always thinking about, yes, I only have to screen three to benefit somebody. How many people do I screen before I end up with a harm? So I just always like to come back to harms. Yeah, I just quickly respond. So like, yeah, I agree with you, right? The number of false positives will, I think if we just trust the math, right? That these results are not 100% specific. If we're talking about like the pathogenic variants, then you could get astronomical numbers of false positives and that would just become overwhelming. And so I think that's part of the issue is tuning the types of results you get back to keep it so that you're at least in a manageable number of false positives and then try to get strategies to re-classify those and sort of that. Yeah, I think your name is Katelyn, I can't read sideways. So I think my question is pretty directed for Christine, but of course others chime in. I'm curious around the sample collection element here. And if you have thoughts or recommendations for criteria that organizations who are trying to implement population screening might use to help make decisions about what sample collection method to implement and thinking about from a practical perspective, we've at MUSC have started with a lot of sample collection and are doing that in clinical settings as well as at home, have not gone into blood yet, but because of the logistical barriers, but just would love to hear your thoughts around the decision tree or decision-making there. Yeah, thank you. You know, I think this needs a lot of consideration, you know, first about what your sample volume is going to be and then what your outreach is going to be. So if they're going to be seen at a medical center and it's going to be somehow combined with a visit, then your options are much greater than if this is going to be mainly by outreach through social media or other types of approaches. I think from studies that have taken place so far, it's been mainly non-invasive testing. And from the side I showed, and again, I was surprised at the level of complexity of the blood draw versus these other non-invasive. So you really have to think about the resources that are needed and whether the incremental improvement you get in sample viability and lower failure rates and long-term storage is that meaningful. Also, the mislabeling and there is no full-proof method for this yet and that has to be developed. As I mentioned, the dried blood spot card where you actually put your name on the card and then collect your sample, I think is maybe the closest, but anything where you have to put a label on a tube is prone to error. And then marrying up the patient information downstream to the sample, that's critical as well. But no, we don't, as far as I know, we don't have guidelines, but I think that is definitely needed depending on the scenario. And just one follow-up, I think it's been, maybe it's post-COVID, maybe it's just getting used to fit tests and at-home sample collection. It's just been really amazing to see how people are responsive to the at-home collection. And so I think that's been a lesson learned and really a surprise at home and then high return of those. So. Thank you. I forgot to mention that, which we might talk about. Yes, I think the at-home COVID tests have really sort of made people more knowledgeable and not so anxious about handling their own samples here. I wonder if we're on the technical stuff, Christine. If you could say a word about reanalyzing for variants and later on how the practical aspects of that, have you started thinking about those kinds of things? Yes, so reanalysis is even today in a diagnostic setting is a challenge. So making sure that patients are aware that whatever result they've gotten is in a particular window of time and then making sure that your reanalysis protocol is very well understood and communicated and that should be and is in our like exome and genome consent forms, how often reanalysis is performed. But in the laboratory, it's also a challenge. You have to have teams that are dedicated to your database and making sure that whatever changes you make to your previous interpretations are well vetted because you're gonna have to reach out to those patients again. So it's definitely not an easy process but you have to have the resources dedicated in your laboratory, the protocols and follow them. Heidi, you probably have some thoughts on this as well. Could I just make a quick comment about dried blood spots that that's what newborn screening is. Those spots if stored at minus 40 with desiccant can be analyzed after 20 years and give good DNA for an exome or genome. Yeah, I wonder if there are any differences between the dried blood spots that are collected by a healthcare professional. So newborn screening I think is usually in the hospital versus those that are collected at home. So I was not able to find any data about that. But do you know what the failure rate is typically for newborn screening programs statewide? Less than 2% have Terry and then Heidi and then Josh. We could talk about reanalysis all day long but I think we'll hold that. I have a technical question but you go ahead. Great, yeah, I wonder, Jonathan if you could expand a little bit on your comment on how do we get at prevalence of monogenic diseases and presumably yes, monogenic but breast cancer isn't monogenic. And yet we're looking for a particular gene variant. So both, how do you expand maybe to complex diseases as well as what would be the approach to assessing that prevalence? Yeah, so I mean I've seen sort of the extrapolated numbers based on, you know, if you say that of breast cancer some percentages are related to this gene or that gene and then you sort of extrapolate from the population prevalence of breast cancer to get the prevalence of that monogenic forms, right? So that's one way to go about it. I think that there's actually, my guess is that polygenic risk is probably gonna be more directly sort of applicable straight into the kind of the risk that a given individual has at a level of polygenic score that they have as opposed to with, you know, with our penetrance issues somebody might have a BRCA variant but then we know what the risk for them is some percent by some age is that modified by their other polygenic factors is it modified by environmental factors sort of how does all of that work together which I think is a lot harder whereas I think with at least the polygenic risk you've sort of evened out across all of the other factors that might be involved in the risk. So I think you go directly from the projected risk based on the polygenic score perhaps combined with other environmental things if you're doing a genomic sort of risk prediction that might be more direct to the risk of the individual. So this one's a little more, oh Heidi Rehm, Mr. Nohassan Badal and Brad Institute. So this is a little more probably directed at Christine's talk. Today we have, and you gave a great example of a carrier screening sort of tier four, lots of genes but you also have to supplement that with certain assays because of more difficult to detect variation and certain genes, fragile X, SMA, et cetera. In the secondary findings world we accept that those tests aren't comprehensive. In fact, certain genes have been left off the list knowing that they're technically challenging. And so we sort of view it as opportunistic if you happen to come across it, report it but we're not assuming it's comprehensive. And I'm wondering how we all think about the middle of the road here where we're all starting to think about run a genome at birth and use it throughout the lifetime kind of cost effectiveness approach. But that means that we'll be taking tests like carrier screening and sticking them on a genome where they're not optimized for the comprehensiveness of when we design a test for a specific task. And how do we think about using labeling those tests that are not perfect but we also, the cost to make them perfect would make them less useful at a population level. And how do we make it clear what this test is compared to the gold standard ordered intended for carrier screening, for example, versus we're trying to do this but it's not perfect. Do you have thoughts about how to label offer that kind of scenario? Yeah, very challenging question. I mean, I think we do run up against this in the diagnostic world already where we have our disclaimers and many labs do this gene by gene and the list becomes overwhelming. Exxon five in this gene, Exxon 52 in this gene are not well covered. And so understanding that and then being able to take that information and translate that to the patient to tell them exactly how good this test is for you is very difficult. I think for, let's say our carrier screening test where patients are voluntarily or to some extent voluntarily taking this test, you want to give them as good a test as possible. And so that's why these orthogonal methods are being introduced in order to get the sensitivity where patients expected to be. But I think looking at a population level, obviously it's going to be different and you're going to have to balance cost and time. Thank you, I think Josh is next. Yeah, I hate Josh Peter, I'm just gonna add Vanderbilt. This is a question. It seems like the sensitivity specificity framework works pretty well for identifying the genetic risk and false positives and true positive sort of thing. But I worry a little bit about using that framework for identifying essentially the connection between the risk and the disease because of time. And every, there's a distribution of disease incidents over time and when you go to apply, let's say the idea of penetrance to an individual person that you're trying to counsel or treat, then you need of course account for how old they are, but also essentially when they got that information. So if we're gonna be screening 18 year olds, how do you counsel them? And what do we need to know? What's the right metric to communicate essentially that risk that connects the genetic risk to the actual time related risk of disease? Yeah, I mean, it's a great question. I mean, that's the big problem, right? Is the goal is to find the people before they have symptoms so that we can do something to prevent them from having a disease so they never become penetrified. And so I think it'll be as we sort of roll out screening in the context of the interventions that we want to do with people, it's gonna be really hard to tell which of these people who have that pathogenic variants are gonna benefit from whatever intervention we're offering them, right? And so that trick of sort of communicating the population benefit that may not actually be an individual benefit is gonna be something that differs from when we sort of think about individualized medicine, right? We're doing this for individual benefit. We're actually not, we're doing it for population benefit. And that's part of the communication, I think of what those results mean for that person. Yeah, maybe this is part of the research agenda. I was just struck by some recent articles that showed that you could show stick figures in a diagram and you get twice the sort of actionability based on what patients like to do compared to, let's say a single probability number. So I mean, it's not only the metric itself but the way that you communicated. And it seems like we really need to know more about that in the context of genetics. And maybe to add to that, it's also the consent process upfront, right? That when someone's signing up, we probably see this in clinical medicine all the time right now that this happens to me as a primary care person where one of the specialists is ordered to test because someone had an eye finding and then they find a bunch of things they didn't know what to do with and they get sent to me. And I don't know how much counseling was done upfront. So it's both once we have results, but even before, I think understanding that better probably makes sense too. I have Erin and then Carol and Mark. Thanks, so we heard someone mention all of us and we know other databases like Nomad are increasing their representation from individuals from diverse genetic ancestries. But what else sort of are we at the point are those will those be sufficient? You know, we heard Jonathan you be saying sort of we have to tune our decisions to maximize true positives and minimize false positives. Do we, can we do that yet? Do we have the data from diverse ancestries to make sure those choices are gonna benefit all? I mean, the answer is probably not yet and certainly not for more of the rare diseases. Again, and I think it's, yes, you have to, if the well-established pathogenic variants are concentrated within people of European ancestry because we've seen them the most and studied them the most, then that's gonna be a problem and we'll have to figure out how to address that me. Make sure that the catalog can benefit most people. And maybe a quick follow-up to that Jonathan, how do we, and maybe Christine can help with this as well and Bob, how do we get these new data streams? So as we get more data, how do we make sure it goes someplace like ClinVar so we can use it going forward? You guys thought about that much. Sorry, I was not paying attention. Yes, so I mean, we're like, for instance, we're right now working on the submission from the All of Us research program which obviously has a larger diversity. We're also working through the Global Alliance globally to try to get every country to submit and support their submissions because you can't submit variants, you don't test from patients. So I think it's just a widespread model to sort of support data sharing at various levels including interpretation ClinVar submission but also the raw data and how we all use that. Just launched an OMAD-V4 last week and the contribution of new variants despite the fact that we dumped huge amounts of European individuals into this database. There's a very small increase in the number of variants from Europeans that were above a certain frequency that lets you exclude it. The contribution of the smaller number, 130K of non-European was massive and it just really demonstrates just how important diversity of data is not only for interpreting the population it's in but interpreting other populations. Thank you. Some question was kind of similar in that, Jonathan, you mentioned Maeve, sort of the multiplex assay variant effect and how data from that sort of project could feed into understanding the effect of all of these diverse variants that are reported in. I mean, what's your vision for how to integrate those types of data into this assay? So I would see the need for a good level of communication between the ACMG committee that's starting to say these are the things that we think are worth the population screening for. ClinGen and expert panels to get really high quality specifications for classifying those variants and the Maeve groups to tackle those genes, right? And between all of that, you should be able to have a really good catalog of clearly pathogenic variants and hopefully not as many that are likely path or kind of in that wishy-washy range so that those things could be rolled out with confidence. That's the goal. And that'd be a really good project. Mark Williams, Geisinger. So this is a comment rather than a question but I'm happy to have others comment on my comment. And this comes back to the idea that when you do screening, the balance of sensitivity and specificity results in residual risk which also needs to be communicated and falls for assurance. And we have plenty of examples from newborn screening and also from even direct to consumer testing like 23andMe where somebody has an obvious family history for coronatory breast and ovarian cancer and they say, well, I don't need testing because I had 23andMe. Or a child that presents with chronic rhinocytositis and pneumonitis and poor growth and floaty stools as well I had newborn screening for cystic fibrosis so we don't need to do a sweat test. So I think one of the other aspects of this that we need to consider is downstream how do we communicate the idea that this is screening we are intentionally going to be missing people some for technical reasons that these are just genes or regions that we just can't get at and others for rare diseases that don't meet the thresholds that we would define for screening and how do we facilitate recognition and testing? So at ASHG last week there was some discussion and some of the sessions about whether genetic counseling is actually needed for negative results. So we're all very familiar with genetic counseling for these types of programs for positive results but what about the negatives? And is that actually just as important or perhaps even more important that participants aren't left with the impression that they're no longer they have to be concerned about this particular issue. Yeah, to add on to our discussion from the first thing if we are arguing as some of us are perhaps all of us are that we don't have sufficient genetic counseling resources through the positives then I think we can fairly well assume that we definitely don't have enough resources for the negatives. So then it raises the question of how do we develop resources that can achieve some, I won't say equivalence but at least some acceptable level of communication for those particular issues since we know we won't have the human resources to be able to do it nor can we afford the cost associated with that. So I'm gonna say that the part of what Les said earlier was kind of these automated and computational ways of doing this and I think that the genomic learning healthcare system is going to need to know somebody's gotten screening what the results were but also all of the other phenotypic stuff that you just mentioned and the probability that that represents a disease, right? So that you can calculate what is the actual Bayesian kind of relationship between the fact that they had a negative screening test with its performance versus all of the phenotypic stuff and does another test need to be done and how can we rely on EHR to pull that information together and fly it for somebody to act on which I think will be a really interesting challenge for it. Yeah, it's well outside the realm of this but I think some of the work that's been done at Vanderbilt and other places to say we've got data and EHR that can make it so that we're not reliant on between the years which we know is going to not be successful to really flag folks and say this is an individual that definitely needs to be tested for CF or whatever based on a very high confidence phenotype that raises that prior probability. Sounds an awful lot like a genome informer is consent from the merge. I'm gonna call it quick time because if you guys purpose in trying to confuse me if your card is up and you don't have a question you can put it back down so I can figure out who's next. All right, excellent. I think it's, I think actually Dan is next and then Kayla and then Terry. So three random comments. One is I appreciate the shout out for the phenotype risk score part but we actually looked at CF and Lisa Basterash really couldn't find any undiagosed CFs using the phenotype risk score. So that's an example of the phenotype risk score not finding extra cases but I appreciate the shout out anyway. The business of counseling negatives I think that comes back to what Les was saying. If somebody has a very clear phenotype so I'm an arrhythmia guy so if somebody has a QT interval of 550 milliseconds and their genetic testing is negative they still need to be followed by somebody who knows something about QT intervals of 550 milliseconds. On the other hand, if it's population screening and they were screened for because they don't have an indication I can't see that we need to counsel those people. And then I just have a comment about the maves. We're sort of, I would say we've put more we put several toes in the water around maves and one of the things I think I'm learning is that what maves do is assign pathogenicity or not depending on a particular protein function that's being interrogated. So you can never be sure if you have a particular variant that looks benign on one assay whether it's gonna be benign on all the other assays that you might want. And obviously the field is moving very, very quickly and maves have nothing to do with penetrants. Leases as near as I can tell. So there's still this penetrance problem where you don't be left with. And the other problem that somebody's gonna have to solve with maves is that there are a couple of hundred KCNQ1 variants. For example, perhaps even less than that that have been annotated by ClinVar and ClinGen. And when we do a KCNQ1 mave map there are 13 and a half thousand variants. And so there's a problem scale and how we're gonna sort of be able to accommodate levels of evidence and just be able to present data in a chewable fashion to the wider community. A separate discussion, but worth, I have to say something. Any of our folks wanna respond to those comments? All right. Caitlin, if you put yours down, we know. Oh, I thought I was allowed to. So I was going to follow on to Mark's comments. I think there are, as far as research agenda and research questions, a lot of opportunities to think about from a behavioral perspective what's happening with people who have negative results. Are they, how are they interpreting that? What are they doing with regular screening behaviors? Are they stopping screening because of the way that they've interpreted their results? So I would just maybe emphasize that as a potential research direction. And then I think too, highlighting some of the work that Kim Kappingst and Guirame Adelphiol from Utah have done in the bridge trial and in their ITCR work with returning negative results using a chatbot for genes associated with HBOC and Lynch. And they've seen a non-inferiority to returning those negative results with chatbot through standard care. And I think that it's not population screening, but is potentially a really good example of a mechanism to be returning and educating folks about the negative results. So quick comment on your first point, which I think is a really good one. If I see a teenager who's obese or overweight and I check the cholesterol and it's normal, does that mean that they say, oh, now I can do anything I want, right? And if someone does a whole genome sequence and it's nothing shows up, does that mean I can drink and smoke into where I want because I don't have any risk factors? So I think research in that area is probably critical as you're pointing out. Comments from our panel. And you guys are quiet. All right, I think of Terry and then at the end of the table, I can't see your name, sorry, Kate. Yeah, so I just was curious, Kate, when what is ITCR? That wasn't my question. You asked me too fast. It's a funding mechanism through the NCI. Oh. And so it's focused on developing algorithms for helping to identify cancer risk and then tools and resources. Great. My question actually was for Bob Currier. When you said that when a diagnosis is made in a child, and again, we're not talking about in birth screen, but we can get lessons from it, they're referred to an appropriate specialist, which leaves, you know, there's lots of arrows along the way to that that you commented on. But I wonder, you would think that that would be sort of the upper limit of who would respond, who would actually follow up, et cetera. And that in adults who have fluid choice, et cetera, it would be much lower. And I wondered, is there any estimate of who actually, you know, what proportion of screen positive actually get into care and get appropriate care? You know, if that's... Well, we do know that I would say over 95% of positives get to a diagnosis. Well, after that, I really don't have, unfortunately the way the newborn screening system in California is set up, after a diagnosis is made, the positive case management is handed off to mostly CCS. CCS. Uh, it's the pair of children with special needs. Oh, really? It's another part of the state health system. And newborn screening actually doesn't get data back about long-term care, long-term follow-up. So it's very hard to really have a sense of that. So I will say it hurts so that we are now funding both propel grants and copropel grants for states to be able to do a longer-term follow-up. And you're right, Bob, that there doesn't generally happen. I will give you just one example that we know of what may happen in Ohio where they're screening for Crab A and there's not a secondary test for sight visiting. They have a high number of false positives. And more than half of the kids get completely lost to follow-up. So identify newborn screening as being positive for, but no follow-up. And so it's really concerning what can happen. Yeah, and just to follow on that, I think there's this connection for each state to have who's gonna follow up these results with the state lab is making sure that those are getting acted on by someone. Will there be something similar in an adult population screening program? Is it the responsibility of a health system to identify those pathways? Is it the state that doesn't? I mean, how are we gonna figure that out? It's patchwork, it'll be a problem. And I think we're gonna have to rely on our primary care providers and educate them on what to do when they get a positive. Because that's gonna be the first person that often sees these people, what to do next. And I think it's an important distinction, right? In the newborn screening, since it is a fairly much state mandated test, you could argue that if the state is doing this, there's some obligation to make sure there's a follow-up. What we're talking about some foreign adults though, it's not a state mandated. So it does probably fall on the healthcare system and likely the DCV. I think Kate was done in there. One of the last questions. Okay, online, okay. So after Kate online and then- Right, just a quick. The ITCR is actually a general technology mechanism for the NCI, I've been on that study section several times now. So all sorts of technologies screening is just one of the many technologies. So the other thing I was just gonna comment about the negative results is that there's clearly a trend, I was just on an ESAP for clinical programs to start returning negative results through portal and through low risk BOS. That more and more institutions, especially because of sort of the workload of genetic counselors have really started moving towards returning those results that way. I realized it might be slightly different for population screening, but I think you're gonna see a real clinical trend towards results being returned. And obviously they, people have a mechanism to ask questions to the portal if they have a question. But my impression is that that is really increasingly being adopted in many, many clinical settings just to put that out there. I agree, it's not possible to know if people particularly interpret negative results. I mean, I think people always say, you need to continue your screening, you need to continue your risk. I do this all the time for melanoma genetic testing in particular an area where you wanna make sure people realize that genetic testing does not change their need to have dermatological screening. And I sort of am very upfront about that. But I think that this is a cat is out of the bag phenomenon that negative results are really gonna start to be generally returned clinically via portal. Any of your panelists like to respond? Okay, there's a card up about happy down. I can't see who that is, I apologize. Introduce yourself, please. Sure, Kelly said something. And I don't wanna, I'm gonna talk more about education and training in a little bit. So I won't steal my own thunder, but one thing I just wanted to mention that was talking about these negative results and the people who might get false reassurance or overinterpret that negative. And you're talking with new board screening at State mandated, you have very broad participation in that. But I think at least our experience, I think others today, a lot of our population screening programs are they're opt in. And there's a inherent ascertainment bias in those populations and people who perceive benefit of that program. And you're gonna have higher rates of people with that personal and family history in there. And so that risk is even kind of exacerbated. And I think we just need to acknowledge that be prepared for that. And when we think about the calculations of those risks that, whether we should really be using population prevalence or should we assume a kind of increased ascertainment bias there? Actually, my comment was somewhat related to your comment. As these population-based tests, let's say for familial hypercholesterolemia or other conditions become more available, we have to make sure that they're used in the population setting. There are some examples from our carrier screening where providers are actually using it as a diagnostic test for a suspected, let's say child with CAH or other condition. And it's just, it's not appropriate because of the reporting structure and so forth. So we have to make sure that providers, even though these tests may be more available, may be less expensive and collected at home, et cetera, should not be used for indications other than what they were intended for. We are at the end of our time. Do we have a chair, Erin? Do we have chance for one more question online that you want to go to our summary? Okay, is there someone that are reading? Yeah, Carol Horowitz, go ahead. Thank you so much. As a primary care doc, half of our jobs are using screening as teachable moments and saying, you know, just because you don't, your CAT scan isn't, your lung screening cancer, your lung screening CAT scan isn't positive. You should stop smoking just because you don't have diabetes doesn't mean you shouldn't eat healthier. So I'm struggling to understand both why we, it almost feels like the concern we have here is more elevated than some of these other times that we screen and have negative tests. And I also am a little bit concerned that the idea coming out is that we want genetic counselors to do everything but we just don't have enough money to do that. And it might actually be that weaving these things into primary care and handling the way we do other things will be viewed positively and we shouldn't view that as a loss. So as I turn it back over to Erin, I wanna echo that and say that PCPs give this kind of counseling all the time. And if there was some tool that let us know, you know, what the data showed, we could probably do it. The research agenda though really should tell us more about whether negatives change behavior in a negative way. Erin. So I'll just do a 30 second wrap up. I'm not gonna give a robust readout of all the good points that were raised. So Christine covered important concepts regarding the validating and stress testing, the pipeline, especially when thinking about high throughput screening. It's critical to have robust systems in place to monitor performance, both of the tests and the interpretation. Have we really thoughtful when deciding when new advances plan to be introduced into the clinical sequencing workflow? We didn't talk about that much during the discussion but that same goes for sample collection and choice of platform. We hear from Bob, the new board screening considerations are around serious, urgent and treatable disorders are applicable in the adult context. We really need to figure out how to hand off positive test results to clinical providers in the context of the US healthcare system. Disparities exist. We need to do better to include ancestors from underrepresented populations. We're seeing the value of that particularly like Heidi described with Nomad and the contributions to increase the number of fairings that we can classify as benign. It's imperative to have better estimates of prevalence of conditions, natural history of disease and age-based penetrance. We need to calculate what is required into following up on positive findings, better understand harms of reporting false positives and I'll stop there. Thanks. Thank you both very much. I would note the lunch is up in the far corner there. The hotel people may come out and pull the table out so you can do that without spelling anything. So, yeah, that's right. And we'll be back at 1.20 please, 1.20 to start the next session. Thank you all. Okay, if you could all take your seats. We're about to get started. Okay, welcome to the session three of the meeting. The session is gonna focus on the logistics of population screening. I'm Carol Bolt from the Jackson Laboratory and a member of the Genomic Medicine Working Group. And we're gonna start off this session with Melinda Massart from University of Pittsburgh Medical Center and Melinda, turning it over to you. Okay, great. So it's hard to be the first speaker right after lunch because now everybody is sleepy, but it's really an honor to be here today and to talk about opening the flood gate of results and are we ready and how will we handle this in healthcare? As you can see, I am a family medicine physician. So that probably gives away the punchline of what I'm going to talk about today. But I do want to just sort of acknowledge that I think, I may be posing more questions than answering them in my session today because I think there's a lot we still really need to think about. So I think we all recognize that there is a tipping point and that's why we're here today to really be thinking about the difference between population screening versus risk-based screening. And in our primary care precision medicine clinic, we have the great opportunity to be able to do very extensive primary care-based pedigrees with our patients. And although patients are coming to see us for an indication, we often find one to four additional genetic indications once we go through that pedigree. And even with this level of detail, we know we are missing folks. And so in our office over the last year, we have started offering population-based screening to our patients, even if they don't meet specific criteria for indication-driven testing. And I think that's really because the cost of testing has come down so much that we are even able to consider this. However, this is just on a very low, small scale. So how do we really think about when is the tipping point in the larger scale and going nationally with population screening? So a couple of things I wanna talk about first before talking about how do we handle the results is one, this concept of democratizing genomic testing and using genetic testing as a tool. And someone brought this up earlier even I think using some of these exact same examples, but really genetic testing is a tool now. And we really need to think about how to scale that and put that in the hands of all clinicians just like radiology is a tool, cardiology is a tool, when I order an MRI or a CT scan or an X-ray, I don't refer my patients to go see a radiologist, right? I order those tests myself. And same with many cardiology tests, if I want an echocardiogram or a stress test to restratify someone, I order that myself. And then I take that test result and based on the finding refer to the appropriate specialist to help manage that particular situation. I think we need to start thinking about genetics and genomic testing in that same concept. And many of us have acknowledged the challenge in scaling up access to genetic counselors, which I think also is probably an impossibility when we start talking about population level screening. And so again, how do we restratify patients and then get the right patients to the right level of care after that testing? The next concept I wanted to just point out, and again, this has also come up is really thinking about a single test future. One of the challenges right now around democratizing genomics is that testing is so highly nuanced. And this is really a barrier to most clinicians being able to utilize genetic testing or screening. But if we move towards a single test model in the future, then we can also think about how does the single test be applied across the lifespan? And when is it appropriate and relevant to unmask certain results? Even if we have them all, initially we don't have to interpret and analyze them all initially. And we can think about appropriate times across the lifespan when they are relevant and appropriate or when there's an initial clinical indication. And in the future, this should be able to happen really in seconds, right? When there's a clinical question at the point of care. So what do we need to achieve population scale genomic screening? And again, lots of folks are talking about these different elements today. And this is certainly not exhaustive, but I think we have some critical ingredients that are needed in this recipe. We need national buy-in. We need community and foreign processes. We need integrated clinical decision support for management, informatics, infrastructure and educated workforce, simplified testing, data sharing mechanisms, patient empowerment, enhanced protections through GINA for privacy and security and funding. And if we are able to achieve this, who and how would we handle all of these results? So would testing be centralized like newborn screening within state labs would health departments be responsible for notifying positive results? What would actually trigger the interpretation at what stage of life? And what would we consider actionable and when? Would those results then go to the relevant specialty care providers? I think that this would be very challenging because this would require a chaotic network of referrals and often great delays in being seen as we already know is happening nationally in genetics clinics. And if it doesn't go to the specialty care, does it go to primary care? So I obviously am going to advocate that, yes, the answer is it should go to primary care. And I know many of you are thinking about this as well. I think the primary care workforce makes the most sense. We truly are the orchestra conductors of health. Primary care includes pediatrics, family medicine, internal medicine, obstetrics and gynecology. We are the first line of medical care and have the lowest access barrier. We are available across geography. We provide care across the continuum of life, the age span, it's multi-generational with broad scope of practice. And we are the home of preventative medicine. We have multidisciplinary care models that already exist, including pharmacists, nutritionists, therapists, social workers, and now in our clinic we're adding genetic counselors to explore what this looks like. And patients have honestly already expressed and it's been documented that they have a preference for keeping their genetic concerns within the primary care space. Primary care also the scope of practice aligns with genomic screening, right? So in primary care we do preventative care, which are risk testing panels. We do prescribing management, which is pharmacogenomics. We do routine cancer screening, which is genetic cancer risk assessment and multi-cancer early detection technologies. We do prenatal care, which is prenatal carrier risk and NIPT. We do newborn care, which is following up on those newborn screening results. And we do chronic disease management, which will be in the future polygenic risk scores. Also screening already lives in primary care, right? This is all the national guidelines around screening. And this is already done all in the primary care space. So is the primary care workforce ready for this? No, they are not. I mean, I wanna be positive, it's not an F. I did not give them an F. You know, there's a lot needed to make this happen. And I think the next question though, is primary care able to do this? And the question is emphatically, yes. I absolutely believe that primary care can do this and should do this. So what do we need to do to prepare the primary care workforce? Some of the critical needs that we have are time-saving efficiency measures, right? Time, time, time. There is never enough time in primary care. Knowledge, we already know that's a major barrier. Confidence to manage all of this, both on the side of the providers and the patients. And really, I'm going to lean heavily on robust, informatic infrastructure to support data integration, reanalysis, curated updates, and portability of results. So possible solutions, we need clear and concise, just-in-time clinical decision support to manage results across the lifespan. Clinicians in primary care are never going to have all the knowledge needed. They just won't. They cannot add it to their already very full plates. So they have to be able to lean on clinical decision support and trust that it's up to date and have the confidence that it's going to support them in the algorithms needed to manage all of these screen conditions so that we're actually doing something with those results. We need minimal viable product for supportive management and counseling. Primary care providers should not become genetic counselors. That informed consent and guidance is really the secret sauce of genetic counselors. We should not ask primary care providers to do that, and they don't have the time to do that. So what is that minimum viable product that they need to do informed consent to be able to integrate results into the healthcare records and into the care and management of their patients? And then when to refer those patients up to that next level of care. And of course we need enhanced referral systems for management beyond primary care and better electronic health records ready for the needs of genomic medicine. I think this has already been acknowledged today, but our current electronic health records don't even help us manage diabetes at this point or routine cancer screening guidelines at this point. How are we going to ask it to add all of these additional components? I don't think that's a defeatist kind of thought. I really just think it's a challenge that we have to elevate these EMRs to do what they truly can do. And finally, community informed models are necessary to prepare the public to get to this level of advancing genomic screening. We need to think about who are interested parties, and there are many interested parties that belong on this list, but the two I really want to highlight are the community or public themselves and the clinicians who will be out there doing this work. So how do the key interested parties want this to happen? What should the models look like? Are they federal, state, local, regional? What models will be acceptable to the public and readily adopted? Are they going to be universal or population specific? And how do we ensure diverse and equitable uptake of the models across the population within the US? We talked about these classic screening criteria earlier today. And I guess the question for me every time I read these is who actually decides these answers, right? They kind of pose the questions, but who decides the answers? And I really think that needs to be discussed and engaged with the community, both the population at large, as well as the primary care clinicians who will implement this. And then finally, I'm gonna strongly advocate that we not do a fire hydrant model that we don't prep everything and then just release this massive onslaught of results and information, but instead really think about proposing a trickling faucet model. We pick one high value, high evidence screen. We A-B test this out in different mechanisms in different places with the community, both the patients and the clinicians. And then we layer on additional screening tests when pilot phase is deemed successful. And with that, I'm gonna pass on to the next person. Thank you. So next we have Peter Kraft. Peter makes his way. Who's gonna talk? Whom to screen, when and how? Well, I'm doing this over here for a second. So thanks, it's really great to be able to join this meeting. It's caught the last five minutes of the last session. It was already an exciting discussion. And first the disclaimer, I'm a statistician and a genetic epidemiologist. So I'm primarily interested in gene discovery and gene characterization. So estimating penetrance largely in the general population. So I think about clinical testing and genetic testing both in the clinic and public health, but I, most of the clever things that I have to say about it, I've learned from reading articles or talking to experts like many of the folks in this room. So if I say manage to say something clever, impact yourselves on the back. If I say something foolish, that's entirely my responsibility. So I wanted to start with just reviewing a successful non-genetic screening program. So the widely accepted mammography screening for breast cancer. So the thing to note about this is it does vary across time and across different contexts, different countries, different public health systems, but there's a general agreement that for the general population for average risk women, they should start screening between their late 40s or early 50s. And this is based on a balance of sort of risks and benefits both to the individual and to the health system and society as a whole. So I mean, it fits the classic screening criteria. So it's an important health problem. There isn't accepted intervention treatment. We know something about the natural history of the disease. And I think importantly for this context, the case finding is definitely not a once and for all project. The guidelines call for getting repeated mammography. It's not just you show up. We don't find any evidence of cancer in your breast today. Congratulations and have a nice life. You're screened repeatedly every two or three years. But because a lot of the guidelines are aimed at the general population, there's still a recognition that there will be some people who are at higher risk than average risk who we might be missing. And the question is how do we identify those folks and clearly genetic screening, genetic testing is one way of doing that. And the US Preventive Services Task Force has already made this an area of important research, something they're looking into. So I wanna talk a little bit about genetic screening and how it might help in this context. But before I get to that, I just wanna flag the sort of coverage or the uptake of mammography screening. So it's not enough just to have a set of guidelines that everybody thinks are good guidelines. You actually have to implement them. And even for something that is pretty uniform at age 50, start screening every three years, there are still coverage gaps. And these are influenced by a number of factors. It could be social and economic status. It could be distance to the nearest screening center, the middle panel there is highlighting immigration status or time since moving to the United States as a potential barrier. So all of these things should be kept in mind. We've already heard about a couple of those in the previous talk and I'll come back to this again. So when thinking about the potential utility in a population level genetic screening, we should start with the baseline, which is sort of the current guidelines or the current practice. So this is a schematic that's sort of describing genetic testing, clinical genetic testing for three tier one CDC conditions, hereditary breast and ovarian cancer, Lynch syndrome and familial hypercholesterolemia. So and basically for testing, based on a personal family history of disease. So those bottom three arms there, no pointer. So the bottom three arms are for folks who we've identified should be tested. And of course, if you're tested, you might end up having a pathogenic variant. You may have informative negative test. So we sort of know what the genetic variant that's segregating in your family is and you did not test positive for that. So we're fairly reassured that you don't have the high risk variant. But a lot of folks end up in this middle category where either you test positive for a moderate risk variant. So think check two in the case of breast cancer or it's uninformative. Like there's something going on in your family but we haven't been able to pin down what the genetic variant is. And this is the setting where I think there's been some, there's actually some clinical invitation now. Some companies are offering, some clinics are offering polygenic risk scores to help differentiate those moderate risk folks. So for example, taking that check two example, if you didn't know anything about the polygenic risk score or other risk factors, you're sort of right on the bubble of guidelines in terms of MRI screening. But if you had additional information about the polygenic risk score or other risk factors, you may end up being fairly comfortable that you're actually well below that or well above that thresholds and you could take action accordingly or be, so that's schematic that I was showing. Oh, and the other key thing on this schematic is of course the folks who we didn't test because they don't have a positive family history or haven't been diagnosed with the disease yet or for other reasons, we haven't been able to get them to the testing clinic. So there are folks who are walking around with a variant who haven't been identified. So there's a missed opportunity. So, you know, that little schematic was already somewhat complicated. And when you go to the actual guidelines, they're even more complicated as I don't need to tell many of you. So, you know, before we even get to population screening, there may be an intermediate step which is to automate some of the tests to really identify people like actively go out and look for folks using EHR or other records to flag people who should be screened. And my colleague at the NCI, Kertina Goddard, has implemented a pilot study, Charm, which is the Cancer Health Assessment, and I can't read my handwriting, so RM, which basically takes sort of the typical process where people are sort of opportunistically identified and makes it a little more systematic. And they were able to show that this was able to get more people in for screening who should be for testing, who should be according to guidelines, especially among folks who are underrepresented. So moving to the other scenario where we did undertake population screening for these three conditions. So now everybody gets tested and you can end up in sort of three bins. The middle bin there is folks who would have been discovered using sort of current care. And you can compare these two arms. What would happen if we kept things the way they are versus what if we implemented genomic screening? And in this one particular case, this one particular paper where they did a simulation model, it turned out that the population screening was able for these three diseases together was part of the key argument they were making, was effective, you were able to identify and prevent more cases of cancer or deaths in cardiovascular disease. You had a better quality adjusted life years. And what's particularly interesting or relevant for this session or the title of my talk anyway, is that when you did the screening or when you, maybe you did the testing early but you've unmasked these results at different ages, 30, 40, 50, affected the utility of the screening program. So in this case, starting screening at 30 years old was the most effective. They did look at what if you started it earlier, 20 years or older or so in this particular setting, the gains were marginal and were not necessarily offset by other factors. So starting at 30 made sense. So I'm just gonna come back to this picture and mention some of the coverage gaps that are, I should have changed the title of the slide and that's the, the titles aside but the point is I'm thinking about the gaps of what might exist for population genetic screening. So there's going to be costs. There's gonna be, how do we cover this through insurance or not? There'll be barriers in terms of transportation getting to the testing center, the time to undergo the testing. And there's gonna be a burden on the healthcare system in terms of returning these results and the subsequent follow-up. So all those things will have to be considered. How am I doing on time? Good. And then I just, when I was reading the Wilson and Younger article or the retrospective of the article in preparation for this meeting, it really made me think of the Jeffrey Rose 1985 in our article, Sick Individuals and Sick Populations where he talks about sort of two strategies to lowering the burden of disease. There's the high risk strategy and the population strategy. So the high risk strategy, I think sort of what we're talking about, let's identify the people who are at higher risk and let's intervene on them in the case of cancer screening early if we catch the disease early, versus the population which is about shifting the underlying risk if there's an environmental exposure that's driving a lot of the population burden. Let's change that. So the high risk, the individual gets a big benefit from that intervention but most people will not benefit because we're focused on a small proportion of the population for any particular disease, I should note. Whereas on the other hand, the population, you have actually a big population impact but the difference for any one individual in the population might be small. So this is another slide I borrowed from Katrina sort of making that point contrasting individual level interventions on tobacco control in this case versus sort of population level and you're sort of getting, there's more of an impact for the sort of broader population approaches. But I guess the one thing that I wanted, the point I want to make here is we shouldn't forget about those other approaches especially when we're thinking about complex diseases. So again, breast cancer, there's lots of, there's it's multifactorial, people get breast cancer for all kinds of different reasons. It's a small proportion of people who get breast cancer because they carry pathogenic variants in BRCA1. So there's a potential for intervening in other ways but we should be clear that it's not an either or, right? It's a both and that both of these strategies can be in play at the same time. So thank you very much. Very good. I believe our next speaker, April Adams from Baylor is online. So can we bring up April? April, it's all yours. All right. Can everyone hear me? Yes. Okay. All right. Well, thank you for having me and accommodating my need to be virtual. I'm April Adams. I am, this is professor at Baylor College of Medicine and I'm primary clinical and I work as a reproductive geneticist. So today I'm going to talk to a little bit about addressing the challenges of genomics, screening and populations underrepresented and genomic databases. I'm sure that I don't have to prove to the audience here that the US is a diverse population. This is just a snapshot of the race and ethnicity prevalence by state from 2020. And you can see it kind of goes from the most highly represented group over to the second, third and then kind of a diffusion score of the populations, the lower prevalence. And interestingly, so when you look at this data and you compare it to 2010, you can see that our reporting at least or how people identify and the admixture of people that live in the United States has definitely changed since 2010. So the largest group represented being white identifying is not Hispanic comprised 57.8 and 20% in 2020, but that was down from 63.7% in 2010. And then you can look at this in another way looking at over the different age demographics in the United States. And you can see that also that diversity is reflected in younger populations as well. But the thing to consider with self-reported race and ethnicity is that it can often be pretty unreliable and it may not capture the true diversity of a population. So when you're thinking about what is our actual genetic diversity look like, you find is when you look at people across different populations in different locations with different cultural backgrounds, you see that those processes impact genetic diversity significantly. So even a population next door to each other with the different cultural practice may have some different rate of changes in their genetic diversity. And this can't be reflected in our race groups that are categorized pretty broadly as white, non-Hispanic, black, non-Hispanic, Hispanic, and then we've categorized everybody else into this other group. So just to kind of set that stage up, we are diverse and we're probably not even really capturing how diverse we are. So our identity is multidimensional. So our identity is our ancestry, our genetic lineage, our genealogy, all of those things that we have no control over and then it's still also impacted by population movement and cultural practices and all of that. But then we also have these other categories that are placed on an individual such as race, which is the construct that we create based on physical attributes or things like that, that can also change over time and depending on where you are and where you live. And then ethnicity as well is an area in which it's a construct that really looks at your language and religion and nationality and it can be self-reported, but it is also something that can change. And also people may not know enough about their prior history to really determine their ethnicity and there's really no consensus on what race and ethnicity should look like. And so I say this all to say that we have to be very thoughtful about how we look at race and ethnicity when we are looking at clinical implementation and designing research studies because it's information that we may be building upon the bias that already exists there. And I say that looking at, okay, how are we doing in representing people from diverse backgrounds? So we're probably under-representing our diversity and then we're also not even capturing that diversity that exists. And so when we look at some large studies and large databases, we see that we've made some improvement, but overall it's still heavily weighted as has been previously mentioned towards individuals who have a European ancestry. And that also is going to still limit the amount of diversity that we're seeing in those studies. And so this lack of representation in addition to our inability to really give people the appropriate categories when we're looking at ancestry, race, ethnicity leads to this perpetuation of these health disparities that already exist, which is really opposite of what we want to do with genomic screening and sequencing. And so what are the drivers of these disparities? And a lot of the drivers of these disparities have to do with the social and cultural context in which we live. And so looking at things like social determinants of health and how do these experiences lead to our health and well-being or to decrease life expectancy, higher healthcare costs and things of that nature. And so what we are missing is we are lacking on that diversity of people included in studies. And when we do that, what we find is that you're going to also miss out on the genetic factors that are interacting with our environmental, behavioral and social determinants of health to lead to that disease. And so we know that the parts of genetic diversity that are gonna flow through with our population and cultural processes without including those people, we are gonna miss out on figuring out where we can make that actual impact in health. And I'm sure that I don't also have to explain to the audience that there are clear disparities in our health outcomes. And just a couple of examples of when we look at race and ethnicity, obviously with the caveat of we're mixing race and ethnicity together here, which may not be the same thing. We see that there are big disparities in groups, specifically in non-Hispanic black individuals, the rates of death, secondary to diabetes and heart disease far exceeds the individuals that identify as non-Hispanic white. And so how do we make that gap smaller if we are including people in studies? And not only does this disparity impact adults, this disparity actually starts in utero, right? So if you look at fetal mortality rates, the rate of birth in non-Hispanic black fetuses is twice that of our non-Hispanic white population. And then this still translates to the same when we look at infant mortality as well. So with that, I just wanted to take a case example of looking at reproductive carrier screening. And the reason why is because it's an interesting area of screening. It's an area in which a lot of people are captured, but it's not really done in a population health sort of manner, but pregnancy may be the first time they actually encounter a genetic test. And so with carrier screening, it's really looking at, can you decrease the morbidity and mortality of fetuses and infants by screening asymptomatic individuals for autoresomal recess of an X-linked conditions? And ideally you wanna do this in the preconception period and give people the opportunity to make reproductive decisions based on that information. And traditionally this was done based on, but that was limited because it only captures a small amount of diseases that may impact a fetus or infant. And it also, as we mentioned, people don't know their history or their ancestry or very well. And so you're excluding a lot of people who may actually be at risk. So looking at that and looking at, as we talked about, there's different admixtures, patients can't identify their ethnicity. When you look at something for example, like sickle cell disease, people really targeted African-American patients, which makes sense, but then you're missing lots of newborns who don't identify that way, who also will end up having this condition. And so this is what led to a more expanded carrier screening approach. And when we look at expanded carrier screening, we're gonna be able to look at more diseases, do a pan-ethnic survey, and really try to capture an entire population. And then what we've found is the more diseases you screen for, the more people you'll find are carriers, which is great. And so then we get these guidelines that say, okay, these are the conditions we should be screening for. So frequency, carrier frequency of greater than or equal to one in 200. And thoughtfully, let's make it equitable. And all pregnant patients or those planning of pregnancy should be offered at least this tier three or a carrier screening frequency greater than or equal to one in 200. However, that is a little bit more difficult in actual practice. So when we look at some of the challenges in expanded carrier screening, I really wanna focus on the fact that one, we are counseling people who may not be represented in a lot of these screening panels. And then we really don't have the right access to do this at a population level. So just an example of a study that a systematic review that looked at carrier screening research studies. And in this study, what they really found was that it's been curious, we needed a great job. However, these were small studies and even in these small studies, they only had a very small percentage of patients who identified as not having non-European ancestry. So there again, you can see people are just not being represented and not recruited into these studies. Additionally, when we talk about access, we have these huge barriers and access when you see that patients who identify as non-white are going to have a harder time accessing care due to costs related to care and also have higher rates of being uninsured. And so that makes things that are like carrier screening which can run anywhere from $200 to $2,000, very cost prohibitive. And then another piece that is also gonna be a barrier to care and barrier to access for patients is going to be provider bias and discrimination. And what you see is that not only are patients having barriers with access to cost and insurance and all of those things, they are also seeing barriers in getting the same care offered to them that would be offered to somebody who is a part of the majority. So they may be less likely to get a referral to the genetics clinic. They may be less likely to get a genetic evaluation or to even be offered a screening test. And additionally, they may not access care because of prior experiences with perceived or discrimination and poor treatment in those facilities. So when looking at our criteria for the population-based screening, I think some things definitely are already there but some especially are, can we offer this to anyone who wants it? No, a lot of people still have a lot of lack of healthcare access. So there's a big lift to figuring out how you can offer this in an equitable way. And can you, are we actually offering it to individuals in a routine and an equitable manner because many times patients may not be offered a test just by based on their providers perceived bias about their decision-making, their race, ethnicity, all of those factors? And then obviously as we've discussed, lack of representation in genomic databases really does lead to difficulty in counseling and less downstream research into how do you mitigate the outcomes of positive results. But all of this is not theoretical. There are many, many carrier screening tests out there. You can also do preconception, genome sequencing, exome sequencing. And so the cat's definitely out of the bag and people are being sequenced. We just are a little behind in figuring out how do we implement this in an equitable way? And so right now we're in this phase of kind of having this increased gap in who can get it, who can't and who's benefiting from this process. So in looking at that, we took kind of a step back and said, what are the things we need to be thinking about when you're going to provide equitable genetic services? And kind of looking at the NIMHD framework for equity and looking at the different domains and how do you impact the lack of knowledge about genetic variation and all those different levels of engaging individuals, their families, their communities, educating healthcare providers, building trust in healthcare systems, as well as making sure that people have access to culturally sensitive care, to affordable care and that their wants and desires are incorporated into how we disseminate this kind of care. And so just moving on from that, looking at what are some principles of equity when we have patients who may have been, who may be underrepresented, who may have a marginalized population and really focusing as we develop clinical implementation strategies and research protocols, incorporating person-centered models to help empower marginalized individuals and communities. And I know we're going to talk more about that in later sessions, acknowledging historical harms and using that knowledge about those things to build better systems, right? So when we talk about things like AI and telehealth and all of that, making sure that we are not building that same bias, that structural racism into these systems and having that thought before, instead of an afterthought of, oh, now we realize that this is a disparity, let's try to fix it later. Really having respect for individuals' choices and not creating shame around their choices and their decision-making and being creative about meeting people outside of the healthcare system because sometimes that first step is getting people to trust the system and actually enter it. And then a big piece is also looking at how do you better support health literacy, language, cultural context because uncertainty for one person based on their language, cultural background experiences may be very different than the next individual and understanding that there are definitely first-boat burdens in being able to do all of that. So the gaps that I think need to really be addressed are one, increasing diversity and inclusion in the workforce because a lot of these principles of equity can be done when you really engage what patients need and patients tend to do better and feel less marginalized when they see that there's not only representation in the patient population or the research population, but in the stakeholders and people sitting at the table and making decisions and making policies. Also identifying and limiting barriers to participation as I mentioned, so maybe just getting out into the community being engaged and understanding that there are gonna be multi-modal ways that patients need to be engaged in care and not immediately thinking, oh, they don't want to do it because they didn't show up because maybe they just needed a ride there. And incorporating the principles of equity in all levels of implementation, it really should start with the hypothesis, right? It should be from the very beginning, the first question that you have, how do we create this in an equitable manner? And then the big thing, which is a really heavy lift is how can you expand the on-race? How can you incorporate social determinants of health with ancestry into these research questions and so that people can be moved outside of those boxes to really understand what are the drivers of their health outcomes? Thank you. Thank you, April. And so our final speaker in this session before the discussion is Kelly East from Hudson Alpha. All right, good afternoon. I'm Kelly East. I'm the Vice President for Education at Hudson Alpha as well as a genetic counselor. So I'll be kind of wearing both of those hats in this talk and I really appreciate the opportunity to be here and be a part of the dialogue and share some of our stories and data from our experiences engaging both providers and patients in population screening initiatives. And so one of the things kind of, as I think about population screening, there's some wins in the education space in terms of barriers that, by deploying things on a population level, there's a few less things that a provider necessarily has to know and make decisions around to get patients access to genomic care. But at the same time, population screening requires more education and knowledge and skills and confidence for the patients who are getting that testing to have the maximum amount of benefit and the least amount of harms involved. And so I'm not here to, I mean, everybody in this room, I think would agree that more education and training is needed. But what I'm hoping to do is to share some of our stories and some of the themes that have come out and emerged as places where we can do more and themes that should be a part of the interventions that we need to deploy to provide better education and training around genomic screening for these audiences. And so just as a bit of a context setting, some of the studies that we've been engaged with that I'll be sharing some data from, there is a cancer risk population test that we've been doing for a number of years. So Hudson Alpha is in the northern part of Alabama. So from a context standpoint, the patients and providers we're engaging with it is the Southeast part of the United States where that issue of access to genetics care is kind of at an increasing pinch point. But this is a population consumer directed test that we've been implementing where consumers can go and self-select to have the testing and then we go and engage their providers. But it's the cancer risk gene panel that is definitely meant as a screen not as a diagnostic test. Another study that we've been doing is the Alabama Genomic Health Initiative and I think we'll hear some more about this tomorrow from Dr. Korf. But this is an array-based test that we've been doing again for a number of years for individuals in the state of Alabama for adults. Initially it was actionable disease risk and then more recently we've added some pharmacogenomics to the study as well. Of note from 2017 until COVID in 2020 we were operating under a at what we call our population cohort where we were embedding recruitment sites out in the community, trying to engage Alabamians in the study. But then after COVID we relaunched as a clinical cohort where we started recruiting patients out of specific primary care offices. So it was much more integrated in the patient's clinical care. And then finally, this is another study that we've been engaged with called SouthSeek which is not a population screening test but the reason I included it here is we've got some really interesting data that I think is useful as we think about educational needs and misconceptions in something like population screening. SouthSeek was a whole genome sequencing study for affected infants and NICUs. Although it was a very broad set of inclusion criteria, it was we focused more on which babies shouldn't be enrolled rather than which babies should. So we had a really diverse set of programs that were engaged in that. And also specifically with this study we were interested not only in getting diagnoses for the probands but to test a result delivery model through non-genetic providers and doing a clinical trial around that. For all of these studies, as we thought about scaling up education and scaling up genetic counseling, we focused on how can we best use our precious resources of clinical genetics professionals, largely in all of these studies, the front end, the informed consent conversations and decision making was not done through a individual interaction with the genetic counselor. It was done in different ways for different studies and I don't have time to go into a whole lot of detail. But we focused a lot of our education interventions around the return of results and thinking about on the back end, once you've got results, how do we manage them and make sure that the patients are getting the right downstream care. So the way I wanna frame this is just talk through some of our lessons learned and things that have come up that have framed how we educate our providers and our patients and things that I think we need to care take even more maybe. This is one that we've been talking about already quite a bit today and that population screening is going to identify a lot of people with an unmet need for diagnostic testing. I think we all know that there's so many people out there that should be getting diagnostic testing and they aren't for a whole variety of implementation reasons. And so in a couple of our studies in particular and as I pulled this, I checked myself over and over that these numbers were indeed the same in these two studies and they were. But for the Alabama Genomic Health Initiative and that information is power, cancer screening, cancer risk screening test, both of those we included family history as part of the intake process. So when patients were enrolling into the study or were signing it for the test, they provided some information about their family history. And one of the roles of our genetic counselors was to go through and review this, although we've gotten a little better with automating that as well. But going through and using this as an identity to identify those patients that kind of regardless of the outcome of their test result, there's some additional actions that should be taken based on their family history. And it was almost half of our participants had some kind of what we considered a flag. And generally speaking, those flags were things that came up that would have made them a good candidate for genetics evaluation or additional genetic testing. Another thing that continues to come up as we've been looking at results that have come out of these studies is that oftentimes the people who get positive genetic testing results, it does not corroborate with the family history and personal history that they have told us about. Which not terribly long ago where we went through and looked at all of our positive test results in our in our actionable disease risk in that population cohort of the Alabama Genomic Health Initiative and looked back and said, well, how many of them would they have been flagged? Was there something there that indicated that they were at a heightened risk of getting a positive result back? And the answer was not many. And it also, the answer was it varies wildly by the gene that you're talking about where there's some of the genes that we reported that the corroborated family history or it was 0%. There were others that were 100 and kind of everywhere in between. But what this leads to are really surprising results that we can't necessarily predict or counsel or educate well around at the beginning before you have these results. And in working with our providers, we did a fair amount of education and talking with our providers at the front end. And then as they started getting these results, a lot of them got really uncomfortable about thinking about having to talk to a patient about a genetic test result that seemingly is completely out of the blue. And it also brings up that issue that we've been talking about some today about what do you do with management in a unselected population and someone who doesn't have that personal or family history, you know, what is their actual risk for disease and should they be following the management guidelines that have been published that have come from those high-penetrant or more high-penetrant families. Another theme that we've seen come out kind of over and over from providers and patients alike is this overinterpretation of a negative result. And the risk that those people that have that personal or family history that getting a negative test result doesn't necessarily decrease their risk, especially just depending on what the testing was and how good it is at picking up the known risk factors. And so this is where I wanted to talk a little bit about that South Seek study where we've got some of the, we've got some really robust data around provider errors, provider misconceptions, which are really big opportunities for identifying places where we can do better in terms of education and the tools that our providers need. And so in South Seek, we took all of our, the pro bands, the infants that were having whole genome sequencing and we randomized them to either get their results from a genetic counselor at their site or from a non-genetics provider, most of whom were NICU physicians that we had done some level of training with. And what that looked like was a half-day live training. Also our genetic counselors had a role where we were writing the result reports for the study and going through and kind of creating a bit of a roadmap in those result reports that providers could use that has had a lot of the talking points in it that went beyond just this is what the result means, that included some more things about what this means for the family and contextualizing that result for them a bit more than I think is typical. But so the patients would get their results from one of these two entities and we audio recorded all of it, which has created a really robust data set that we're just now kind of scratching the surface of and dreaming of what else we can do with this data. But the reason that we were recording it was we wanted to be able to look for errors. It started as a safety requirement for the study and then it turned into a really interesting research question. And this is a kind of busy slide, but the take home point is that non-genetic providers they were significantly more likely to make what we consider to be a major error. When we were listening to them, we categorized them as major or minor depending on whether we thought it would have a profound impact on decision-making. So there was a higher percentage of major errors in our non-genetic providers, but 92% of those non-genetic provider disclosures did not have any major errors in them. Of note, we went through and did some thematic analysis of the errors that we identified and the one that came up the most kind of over and over and over was this overinterpretation of negative results. And there's some quotes up here from our non-genetic providers about, that them wanting to kind of think genetic testing is better than it is. Many of us in the genetic space know the nuances and know how much genetic testing can't find. And these were in affected infants that had a suspicion of a genetic disorder and that we had providers that would tell them that this ruled out genetics or something along that theme or that the future children are not at risk. And so that's something that we, and certainly was part of our education. It was on that report that we sent back, but this is still what was happening in those conversations. And you've got providers and patients that are really over-interpreting these negative results. And I don't think this is unique to diagnostic testing and it's something that's gonna be a major part or needs to be a part, a major part of education for providers and for patients when you're giving back negative or non-informative results. We also have also data here from our inherited cancer risk testing where this was a survey, it's probably hard to read, but this was a survey that we sent out to patients after they had gotten their results and they had them for a while. And this was a knowledge question of asking them what does it mean if someone has a genetic risk factor for cancer versus what does it mean for a person to not have a genetic, the reason I point this out is that there's a more accurate for positive results than negative. So almost 100% of those patients are saying that they appropriately picked the right answer for a positive result, but only 72% of people selected the right answer for a negative result and a notable 27% of people who filled out the survey who most of them did indeed have a negative result said that a negative result decreases your risk compared to the general population. And that's problematic that even for people that don't have a family history, getting a negative result is not gonna take you down below the general population risk. But the fact of the matter is that the correct conception for a negative genetic test result really is not the same for everybody and it largely depends on that personal and family history and it needs to be contextualized which makes broad education messaging challenging because you're going to potentially over alarm or under alarm people with that messaging. We talked earlier, a couple of people mentioned cascade testing and I think that's something that from an education perspective, genetic testing is a little bit unique and when we're talking to providers, making sure that we're calling out the fact that getting these test results go, the impact of that goes beyond the patient in front of them which is kind of out of scope and it's a little, providers are not necessarily thinking about or equipped to handle that kind of downstream testing or talking about those risks. When we look at what patients are doing after they get genetic testing results back from a population screen and I think this corroborates with other studies as well, patients talk to their families a lot more than they talk to their doctors about it and we have an opportunity, I think with working with patients and providing them the tools and resources they need to really facilitate those conversations with family members in a way that I think we can do more to provide those tools rather than expecting providers to do that. And finally, I think integration with clinical care is really important to both increase the access to testing as well as the potential, maximizing the benefits of follow-up. So this is just looking at those Alabama Genomic Health Initiative enrollment data and the blue of these donut charts are Caucasian or European individuals and this is the difference in, both of these have thousands of people in these data sets of in the population cohort of the study. We had, and we had people from all 67 counties in Alabama in the study but we were out in the community recruiting patients into this array-based population screening test and despite our best efforts, we still ended up with a fairly non-diverse data set. Meanwhile, in the clinical cohort, these are patients that are recruited through primary care offices, so it's embedded in care and we've been able to strategically pick and partner with diverse populations in the state but we've had a much bigger uptake of diversity and a much more diverse data set after integrating this testing into clinical care but I'd be remiss not to mention that there's probably a whole bunch of people that don't have primary care doctors and so if you only go that route that you're potentially also, you're creating some barriers to access as well. But so getting the test, getting people in the door but what you do with those results, having that integrated in clinical care is also, it makes it much more likely for people to get that benefit either the follow on testing, further evaluation and then the care based on their results but this is where for those things to happen, providers have to have the knowledge and they have to have the tools to be able to do that. And this has got to go beyond providing CMEs for providers and thinking about the things that have already been mentioned today with EMR integration and AI and telegenetic counseling resources and all of that and thinking about from an infrastructure standpoint of how to make this as easy as possible and as leak proof as possible for patients to get the information and the care that they need. So as I think about some opportunities based on the things that we've learned and things that we need to be paying attention to as we go forward, certainly interpreting and communicating negative results needs to be emphasized. The vast majority of patients that do genetic testing are going to get negative results and we need to make sure that we are caretaking those and making sure that we're not treating them all the same and finding ways to contextualize them for the patients that are getting those results back. Thinking about program infrastructure that can help support that family communication and cascade testing that we're thinking about that downstream benefit and there's a lot of education needs in that not only for people to be aware of their risk but where to go to manage that risk especially when family members may live in wildly different places. Infrastructure as part of our screening programs to use that as an opportunity to catch people that should be on a diagnostic genetic pathway and figure out how to make those referrals easy and more likely to be done whether that's a referral or whether that's providers that are able to then add on those additional levels of testing. And then finally, scalable processes for clinical genetic professionals to provide support. I'm very much a proponent of non-genetic provider engagement in population genetic testing. I'm really interested in thinking about how we as genetic providers can become a safety net for those providers and become a resource for them in helping on an individual patient level providing contextualization and guidance but not being the one that is necessarily trying to use our knowledge and our resources as judiciously as possible but thinking about who are the patients that would most benefit from seeing a genetic counselor and how do we focus those resources together? Kelly will have to call it there we're a little bit over time. Okay, sure. I'll leave this slide up for a second but these are when I think about the places I wanna be doing more research and I think we should be coming together to do more research if you're right here. Thanks so much. So Pat, can you help me by, if I miss somebody who has a question on this side I can't really see them and that would be very helpful. So we're open for discussion and questions. So I'd be interested in sort of Kelly following up on your last slide. I'd be interested in hearing from the other presenters about sort of research agendas that can address some of the, we've had a lot of discussion of gaps, right, and barriers. So I'd be interested in hearing some of your thoughts on research agendas that could address those barriers and gaps and maybe we can go in order. Melinda, maybe you could start us off. Yeah, so this is something I think about a lot and I think a lot has been published on the barriers and gaps and really would like to start moving the conversation towards solutions. So we actually just held a day long session using Human Center Design to prioritize solutions and possible interventions for integrating genomics in primary care. And I think that someone mentioned earlier putting out a white paper about these potential solutions and really starting to have teams that are collaborating across the country to test these various different solutions to really find out what works best in different types of care settings. There's no one care setting in our country that is one of the biggest challenges that we face. And so we're probably going to need several different models to find solutions that work successfully for various different care models. But that's one place I would really like to see a lot of research moving forward. Great, thank you. Peter, any comments on? Sure, I mean, two things that I think came up in the previous session were around understanding penetrance in the general population. And I think April mentioned sort of being able to interpret particular variants that may be very rare in the existing databases. But we don't want to say that those data is an artist or a representative. So we made the flag something inappropriately as being pathogenic because it's rare, but in database, but it's actually not that rare in these samples. So I think those are still important areas to research. I do think we're filling in some of those gaps. So certainly for the more common conditions that we might be screening for, I think we are getting robust estimates of population penetrances. And those were, you know, as biobanks and cohorts get bigger, we're going to have better estimates of those. And I mean, Heidi can maybe speak more to the reference databases. So I know Matt just had a big update and there may be some interest in sort of better assigning folks using genetic ancestry as opposed to population levels or self-reported race and ethnicity, though that might help with the interpretation. And that goes to my other thoughts on research areas would be sort of on the informatics and both sort of how do we get the informatics in place so people can interpret these things? And then when we're going to be probably there and maybe Kelly can speak part of this is we will be relying more on some sort of automated reports as a way of sort of getting that conversation going. And then maybe the clinician or the specialist would be sort of more answering questions that folks might have, but there might still be some informatics. And I just want to make a shout out to the leaky tab. So that's, you know, part of the research will be the implementation as well. How do we do this most effectively? April, would you like to make a comment on research agendas that you think would be essential to addressing some of the logistical issues you identified? Yeah, I think that everybody's made really good points. So I think when we talk about how do we, you know, it can engage more people and utilize the workforce appropriately, all of that. I think really being thinking more out of the box about how we're getting out to other communities. Because if you look right now where all the geneticists and genetic counselors are, they're in major urban areas. So, you know, I think definitely focusing on how do we utilize technology better to be that consult or that safety net person for somebody's primary care home? Obviously that implies that people have a primary care home, which is a bigger issue, but utilizing those structures that work and maybe that's an example to grow that type of infrastructure or more. And then I think that also just, yeah, to reiterate, I think that looking better at how we more thoughtfully design studies to look at diversity and how we are categorizing people. Because I do think that we are, you know, the genetics community is definitely aligned in a space to change that perception and help people understand better why ancestry matters and how we utilize it in a, you know, equitable and thoughtful way. Thank you. Kelly, any comments from there? Sure, I agree with everything that everybody just said. And I think when I think of the things that I'm most interested in and think would be really impactful is we're just talking about thinking about how to pilot and test different models for interpreting and managing genetic test results. Thinking about how the combination of the testing labs, the primary care provider and maybe a clinical genetic specialist that how those could come together to create pipelines that they get the providers what they want most, which is an answer for what does this mean for this patient, not what does this result mean or what does it mean for this patient and what are my next steps and thinking about whether that the combination of tools but also where the human brain can supplement those tools in a scalable way. So one of the things, oh, I'll go ahead, Terry. And then Mark. Sure, so Pete, I was delighted to see somebody quote Jeffrey Rose. It's wonderful to see his name again. And I was just curious, you gave some good examples of population wide efforts in smoking tobacco control. What would you think would be population wide efforts in genomic screening or identification of genomic risk? Yeah, so I guess I think of the population screening for genetic risk is falling more in Jeffrey Rose's high risk approach. So it's identifying those folks who are at high risk. So which we're sort of doing now but there's a lot of people we're missing as galley source and data where there's folks who aren't being recommended to testing who would benefit from it. So yeah, I don't know if that, so I guess sort of exactly what this meeting was talking about is what I would think of as genomic screening. Epidemiologists sticking together. So the rest of us are going, what the hell is Jeffrey Rose? So the question I would have for each of the speakers is, I think you've raised some really interesting research questions that are worth exploring but I'd be interested if you would be willing to propose some research methods or some potential projects that might get at the specific aspects that you were talking about. So a little bit more of a translation of the questions into a project or a methodology. Yeah, for having thought for 20 seconds. I'm gonna go back to what I really emphasized in my presentation, which is clinical decision support. And I think that after 20 years of trying to encourage primary care doctors to adopt genetics into their scope of practice, I realized that that is just not feasible without these tools. And so I think I would highly propose an implementation study around two to three different models of clinical decision support that both provide just in time information to the clinician and education to the clinician around how to interpret and integrate one specific screening, which could be something like pharmacogenomics, could be something like hereditary cancer testing and really look at the success of that implementation and comparing those different products as well as strongly exploring the user experience from the clinician's standpoint. In addition to that, I think I really would encourage us to engage the community of patients and we're not hearing from them today specifically, although I think all of us are also community and patients, but I think we really need to find out what this means to patients and how they would like to see this impact their care going forward. So short answer. Carol, can I hop in here for a second? We've got a fair amount of experience with providing clinical decision support to primary care providers. We've been doing that for quite a few years, both in the area of pharmacogenomics and the other area of genetics, genomics at large. And the experience that we have is that they don't want it. So how do we get over that barrier? So I don't know if you mean the experience in your specific institution. Our specific institution, but I would say more broadly, I can report that at least I would say across the emergency network, which has got eight or 10 clinical sites, that's a fairly consistent result. I mean, people, pharmacogenomics is a good example. We provide pharmacogenomics clinical decision support and they don't want to pay attention to it. They want to just click it off and ignore it. So how do we get past that? Yeah, so I think that's where that end user experience is critical because I think we, a lot of us designed that clinical decision support with interruptive alerts and primary care providers are overwhelmed by interruptive alerts and don't like them. And so I think that, again, with implementation science rolling out an intervention that won't be adopted or received by the end users is gonna always fail. I think in addition to that, historically primary care providers have not felt that genetics has yet reached the level of value for their patients. And until we show the value of integrating this, that it's worth their time, then they're gonna continue to think that this is more of a nuisance than a benefit. Yeah, I really want to echo your second point there because the first point, we all understand alert fatigue and what the impact of that is. But our experience is the more fundamental problem is people don't even believe the evidence. And so how do we get, I mean, when in fact, I mean, the experts believe the other. So what do we need to do to fix that? Yeah, I would weigh in and say that, while I agree with what you were recommending, I think we've got one step before that that we have to do. And this came out of GM 13. So I'm referencing a prior. And that happened to be on Informatics research agenda. And one of the things that clearly emerged there was the idea that people really don't like an impositional model, which is, hi, I'm from Genomics and I'm here to help you. I'm gonna solve all your problems. And they said, well, we don't have any problems for you to solve, thank you very much. Whereas we actually sit down and say, what are the things, what are your pain points and what are the issues? And so I'll just use one example from our institution where there's an institution-wide initiative to improve colorectal cancer screening. And so there's everybody's bought in, there's a big quality initiative. And then we say, let's build a piece in this for those people that are at the highest risk. And then let's sit down with our clinicians to say, okay, you tell us how you want us to do it. And then when we do that, we get actually much higher uptake because they're part of the design process. And then of course we also follow up with the user testing afterwards because as we say every time when we roll something out, we guarantee it's wrong or your money back. The second piece is we'll fix it. But I think we really need to think more about that early engagement in defining the problem and defining the solutions. Carol? Yeah, I've been trying to say, I'm actually, we've had a lot of discussions with our primary care providers about this and they're really not interested. And I have to say, and they're scared, but our specialty providers are extraordinarily interested. We've really intentionally moved this and the things that we've done to mostly to specialty care providers. And I just wanna sort of contrast that because we've really found that offering, our specialty providers are more and more themselves ordering genetic testing and offering them support around what they really wanna do which is order genetic testing has been much more successful than our primary care providers who do not feel comfortable even with placing consults for genetics. And so I just, I'm pointing that out as some alternative models of thinking about how we do it. It's possible that a research topic could be contrasting doing this in primary care versus doing it in specialty care and seeing what the uptake is because my impression would be the uptake and the interest is quite different. Thank you. Yeah, I just, I wanted to touch base with you, Kelly. You brought up cascade testing a couple of times in your study. And at least in our studies, there is a lot of family communication and people are talking about their results positive, negative that their family members, but that doesn't translate to cascade testing. And so I wanted to get your thoughts on how cascade testing fits into this idea of at some point we will be scaling up to large-scale population screening. But in the meantime, as only some people are getting testing, that is a great way to identify families at risk. But there are huge barriers there. And so is that part of what you think should be studied more in depth as we're getting to this larger scale population screening? Like, is that gonna be kind of an interim approach? Yeah, I think one of the big barriers there is the fact that when you think about a family, they're widely spaced out. And even if you can communicate a risk to another family member, the roadmap for that family member to go get the follow-up care and testing that they need, there are hoops upon hoops that they potentially have to jump through to do that. And some of that I think is thinking about how can we as in the disclosure, in the communication to the province and people that we're engaging with, are there better ways to pave that path for those relatives and passing off resources that are not just for the patient, but are actually for those other relatives to make that easier, making it more clear and making sure that no matter where that relative lives, that there is a way for them to get that care that they need and building those pathways and pipelines and communicating that through the patients themselves. I do think that's an area of right for research that if we can improve that, we can exponentially improve the impacts of population screening, that by we can fix that pipeline and make that easier, you can impact many more people by every person that we're identifying through population screening. And I know there are some efforts out there through AI chat, those types of things to help facilitate that. And it's been a really interesting research. And I think that's a place where we need to do more and that we can do more testing of different models to figure out what works best. Carol, Jillian has a question. Yeah, go ahead. And then Ned, did you withdraw? You're good, okay. All right, there we go. And then after this, then we'll go to George. So this question I think is primarily from Melinda and Callie or maybe others in the room who know about this. Do you think of what you're doing in primary care under the umbrella of collaborative care models that are being done in psychiatry, mental health with primary care? And do you think that there are learnings from those models that we should be adopting? Because it does seem like they're solving some pretty important problems in the space, including like even CBT codes for how those models can be sustainable. So I'd be curious for your thoughts on that and also where those models may not apply in genetics. I mean, I think the short answer is yes. I think that there's a lot to be learned there and thinking about what are all the other issues that come along with that in terms of liability and this and that and how those things play together and how we can build models that are scalable, that you also have to figure out how to have those things being reimbursed. But at some point that has to get paid for the effort of these providers in that more support role. Yeah, so I agree. I think that the collaborative care models we have a lot to learn from. And I think just like you were saying is that the big takeaway from collaborative care is giving the primary care provider that confidence, that safety net. I think someone used that term, maybe you used that term earlier. And I think that's exactly what I'm advocating for and replicating now in scalable tools that are creating that same degree of safety net and confidence in the tools that can allow us to scale even beyond. But I don't think that I don't think it's wrong at all for us to be starting in this intermediary space with this collaborative care model and just like we have a pharmacist serving perhaps 20 different clinicians that we could have a genetic counselor serving 20 different clinicians as that resource. So I think it truly is that concept of having that expertise and that confidence building to be able to move forward, that we need to really be learning how to integrate and to replicate the scale. So George and then April and then Karen. Thanks George Manson from NHLBI but maybe to paraphrase Mark, I should say George Manson from the federal government and I'm getting help. But you know, the challenge you described extends beyond primary care. And I mean, I see the same thing in family medicine and the comment I want to make is I hope we would capture that as one of the critical challenges that's really right for research. So for example, the National Academy of Medicine's conceptual model for meaningful community engagement rather than working separately and then bringing the answer to here I have a model to help you, it's not gonna work but it definitely would work if we re-approach it by using a similar conceptual approach of meaningful engagement with whether it's primary care or family medicine and really developing the solutions that they've created, that's more likely to work. I think community engagement and just a teaser for the next session coming up and implementation research can really be very helpful in addressing it. Thank you. April. Yeah, I just wanted to mention when we were kind of talking about engaging primary care and how it's easier with sub-specialists and things like that. And I think that, you know, I think we have to really be careful about focusing a lot on sub-specialists because that's where a lot of people have bigger barriers, right? People have bigger barriers getting into sub-specialty clinics. And so really thinking about, what are we thinking about the long-term thing? So what are we gonna be doing in our medical schools in our residency programs and all of those things to make sure that people who are coming through understand how genetics is incorporated into whatever specialty they're a part of and having that being a more robust, ongoing education so that we are not kind of losing that, you know, interval time period to say, okay, even though you're gonna be a family practice physician, you should still know these are the top-down levels of where genetics is gonna be a part of your patient's care. So it's not an ad for them, it's just a part of their regular flow the same way we order a check-sex rate to evaluate for pneumonia or whatever. Thank you. Carol Horowitz. Yeah, thank you. And thank you for what you said, George. I completely agree with you. I, you know, we do our work in many academic community and safety net, family medicine and primary care practices. We're, and I'm interested in learning from you what the problem is. We do what George said, which is we bring those frontline clinicians in to develop everything with us every step of the way they're saying, this will work, this won't work for me. The message they give us is we care about this, we just don't care about as much as you all do, because we have 50 other things to care about at the same time and we balance it out. So I'm interested from you, are you finding that people are not interested or that they're balanced? Carol, you cut out there just briefly at the very end. Could you restate that? Oh, I was asking if you're finding that the primary care folks, clinicians are not interested or are they just balancing with everything else on their plates? Would anyone like to respond to that? Rex, maybe? Yeah, I can at least tell you what our experience is. And, you know, I think it's both of those things. You know, they're very busy. They have, what is it, seven minutes per patient that they need to get them through. So that's a problem. And then the increased demands I think of, you know, messaging through the EHR and dealing with all of that has put additional stress on people in the primary care community. So I want to acknowledge up front that they've got a lot on their plates already. But what I'm struggling with is the disconnect between people saying the primary care community should be doing more of this with the experience that we've had, which is they don't know how to fit it in. And I think several people have said this. They're maybe a little nervous about making a mistake because they're not well enough prepared for this. But then I think all of it, the second piece of that though, which does surprise me a lot more is, and maybe it's unusual at our place because we're an academic health center, right? We're not a community-based organization. And what we've experienced is they take initiative. They go out and they read the papers and they're unpersuaded by the literature that persuades the rest of us. And that's a separate problem that I don't know how to overcome. And, you know, is it just that we're true believers and since we're true believers, we're willing to accept it? Or is it that we still need it to the idea of a research agenda? I think this was off the table though, but that we should, we need to generate better and more convincing and compelling evidence that what we're proposing is of value. Is it ignorance or apathy? I don't know and I don't care. So I was just gonna add a few comments there. So I think there's a couple of things. One, I think there's a difference between thinking about overall integrating genetic testing beyond genetic specialists. And I totally agree that the subspecialists are highly engaged and highly motivated to do that because there's so much that's impacting their ability to actually do diagnosis now and genetic guided treatment for those patients versus screening, right? And population based screening and really the population based screening being in that primary care wheelhouse. But one of the things that, you know, we need to do to make, I agree that the providers are interested but busy and overwhelmed but we're not helping them prioritize this because there are no guidelines, right? Their societies don't have any guidelines, you know, until it becomes a checkbox that the USPSTF requires primary care doctors to think about, that's really hard for them to think about and then there's also no incentive. And so we are paid to check other boxes. And unfortunately that's the reality of how medicine is practiced. And so we really need to think about not only proving again, getting back to that value for their patients which I think primary care providers really care about but then value to themselves and guidelines for which to follow to do all of this. So, Jessica, did you have something else? And then at the end of the table here, I think. And Jonathan. And then Jonathan next to you. Oh, Jonathan, how do I miss you? And then Dan. Yeah, so I just wanted to say we did a study within Kaiser Permanente of who was helping patients at high risk due to genetic variants with their follow-up care. And everyone pointed to the other primary care point here. The specialists, the specialists, but they're not coming back to us. They're getting their diagnosis and they're leaving and doing chart reviews. A lot of times the guidelines in their record are outdated because there's no contact to help them update it. And so I think it's a problem on both ends. And who do we expect to be taking care of them and the follow-up and is that who we can guide to get the testing in the first place? Can it all be grouped together as part of care? And so anyway, I just wanted to emphasize that's even in a well-resourced system of everyone was pointing at the other. And at the end of the day, the patient was saying, I'm taking care of myself because no one's helping me do it. And I don't think it was for lack of interest. I think they generally thought someone else was doing it. At the end of the table there. Hi, okay. This is obviously me, Alana at Geisinger. And as I wanna echo also what Jessica just said, you know, I did this in the collaborative model with psychology co-located in primary care. When I was at Kaiser, we actually experienced sort of the same thing and our co-location collaborative model ended up with a visible in primary care. How I have no idea, but it was what I wanted to get. So that is one issue. And I wanted to get also at what George and Carol were saying too though is, you know, we've heard this co-creation and co-, the engagement in the co-creation and how important that is. And it sounds like there is some movement in it working. And I wanted to hear what has been created then. And so, you know, we keep talking about what we're doing so far isn't working. So we need to co-create things that could work. So what have we found so far that may be working or maybe those first steps on the pathway? And maybe you haven't gotten that far yet with that. You co-created it, you haven't actually tested it out yet and that's okay. But I'm wondering if there is anything what is working so far when you do that co-creation? Comments from any of our speakers to that point or anybody else in the room? So when we were setting things up for our screening program, the question was, does it sit in a clinical situation and go through clinical care or does it become a research protocol? And through the co-creation and discussions with primary care specialists and others, they wanted it as a research protocol. And so we pulled it out of what would be clinical care and made it into an IRB protocol. It's a whole separate team. It's ordered by one physician that is the PI of NRD and ASC. And so that's not maybe answering the exact question, Alana, about like primary care, but that was sort of a community engaged approach where ultimately the decision was that it's not gonna go into the clinical pathway, it's gonna go in the research world. And did you, in that co-creation process, do you have the why they wanted it as a research? I think a lot of it is what we've been talking about, just being concerned about feeling comfortable ordering, the volume, our physicians are happy to talk about the program, but then to take it to the next step of actually being responsible for some of the ordering and follow-up, they wanted that to be really outside of what they're doing on the day-to-day. So we've exported that to our research protocol. So it wasn't about that we need more research, it was about that they was practical. Practical, needed somebody else to take on the burden. Yep, exactly not. So we're getting pretty close to the end of time. So, Kelly, if you wanna make a comment and then we'll do three quick questions and then I think we'll be pretty close to the end. Sure, I was just gonna echo that we've had very similar feedback. And it comes down to logistics of that, just in talking with our providers from an implementation standpoint with, I was working with primary care clinics in Alabama that the providers see a lot of value and benefit in having people there that are managing this project that are, they do that, you know, nurses and people that are part of the research team doing that navigation that can do this all day long and get really good at this talking points and talking to patients about it and kind of having that as a unique role that's kind of sitting in that and it's not falling to the providers that is paid for by a research study though. And so at some point you gotta figure out how that can become part of clinical care but thinking about who's doing that work and whether there's a role for somebody doing some of that effort specifically. John? So my comment is on something a little bit different it's been a great discussion but April there was something that you said about drawing lessons from ancestry based carrier screening and I think we could probably all agree that we want to avoid going down the road of ancestry specific screening programs but I wonder if you would comment on whether you think that including, we certainly want to include conditions that are of importance to particular groups. So thinking about April one for example is that something that we can envision as a vehicle for community engagement around a condition that is of importance to that community but also then broaden the opportunity for screening for the other conditions that are thought to be important for population screening. Thanks. So your question is really like if we take this condition that's known to impact this population and then use that to start the conversation for what's next, right? And I do think that that is a really a good point. So like for example, if you take sickle cell, right? So you take sickle cell disease which impacts African-American communities and there's a big disconnect between I have diagnosis and I get treatment and what those barriers are. And so talking about what we know we can screen for this and identify you and these are the strides we're making to improve your health via this and showing you're building that trust in the system that maybe we forgot about this here but we're coming back and we're including this population now. And so building that trust that's definitely gives you that opportunity to take the next step. So I definitely agree that's a very good thought process and how can you incorporate some of those things. But I think the other piece of that though is you definitely have to have somebody who's in the community who's also already has that trust, right? So you have to kind of have multiple layers of building trust with people who maybe don't trust the system that exists. Yeah. Despite you can be changing the subject or sort of opening up a can of words but I'd like comments from any of the speakers about the mechanics of cascade screening. So the issue is you have to rely on the family member to get the other family members to do that. There's a real communications problem there both in terms of what they tell their family members and whether they tell their family members and whether the family members understand or respond. And I think that they can sort of a gap in the way we deliver genomic care. And I'd be interested in hearing how you think it, how you deal with it and how you think it ought to be dealt with. Yeah, that's a great point. I think that evidence from a number of studies is that families, I mean, there are absolute exceptions to this rule and lots of reasons. Some families don't communicate. But largely when we ask people, who did you talk to about your results or that you had this testing? The overwhelming majority of people are talking to at least some of their relatives. Whether those are the same relatives as the ones that would be at risk, I don't know. But I think we have an option. The spouses are unrelated to them usually at least in, except in Tennessee and Alabama, maybe by. Yeah, right, right. But I think that there is, it's not a perfect solution, that there's an opportunity to help improve that normal communication that is happening. But thinking about how not to rely on a patient describing those results correctly or what they should do with them and additional tools and resources that can be built and handed off where that all you're relying on the patient to do is to pass this to this other person and that there's enough information there for them to not only understand their risk, but to know what on earth to do with that risk if they live in Alabama or California or wherever else. But I think it's also important to note there's been two or three studies now where direct contact has been used where they haven't gone through a familial intermediary other than make initial permission. The cascade testing rate is exactly the same as familial communication. So there's something beyond just communicating within families that is depressing the uptake of cascade testing. Contact to family members that permitted with permission from the family. Yes, yeah, so that's allowable with permission from the program. In every state. But also those family members are going to have the same barriers as well, right? So, I mean, you have to consider, like, do those patients have access to a lab? Do they have access to pay for this testing? So every barrier you bring down for that initial contact has to be done for all those family members also. And then also you have to consider that some people there may be stigma associated with having an abnormal test result, all of those things. And so even if they've told their family, maybe it stopped there because they're like, we don't wanna talk about this. And so I think, yeah, there's probably a lot of other barriers as you go down that cascade that are gonna continue to pop up. It's, this is one of the value ads of population screening though, right? We actually take that burden away from families. You know, if we think of newborn screening, we don't do, we don't really do cascade testing on newborn screening results because we have confidence in the system and that everyone is accessing their screening. Yeah, it becomes a transient problem in a future. And one more comment? No. So I think we're up to time here. So just quickly on this top of the logistics, this was a far ranging discussion, but earlier we had a comment on logistics by Christine about sample collection tracking and all that stuff. That is an important part of the logistics that I think we're talking about here. And in this session covered a wide range of topics. Who to test, when to test, who does the testing, how to report, who reports, who takes primary responsibility for the care, is it primary care, subspecialties, preparing the workforce, the timeliness, how it's delivered, when it's delivered, the need still for the informatics infrastructure to align to make that information exchange seamless. I think we could talk for days on that. And I would love to. Sort of informing the public, conforming positions, informing families, rolling this out in an iterative fashion versus trying to do it all at once. We talked about risk estimates and the idea of maybe systematic or automated processes for implementing guidelines on who to test rather than having very complex guidelines that may be difficult to actually manage. And the idea that a combination of approaches population wide and those focused on high risk groups will be needed to do this effectively. We talked a lot about sort of the challenges of underestimating the diversity in patient populations and not capturing that diversity adequately. So the multi-dimensional nature of identity that crosses race, ethnicity, and ancestry, we don't represent that very well. And it is an important part of the logistics of actually delivering genomic medicine at the population scale. And there were a couple other points in there. The barriers to access the cost and everything is not equally shared across the population. And there's logistics around circumventing provider bias and discrimination. And then finally, understanding provider training and patient education and communicating, especially I thought from that, an accurate picture of risk based on screening so that people don't underestimate their risk or overestimate their risk. And I think that balance and that how to communicate that is definitely one of the logistical things that we need to tackle. And there's probably great research project around that. So that's my quick summary of the logistics. Super, thank you, Carol. So at this point, we have a break. We'll break until 325 Eastern and then the back for our final session today, which is on community engagement, which has come up quite a lot. We have several speakers and then a panel as well. So see you at 325. And thanks to all of our speakers. Okay, ladies and gentlemen, the speaker is at the podium. So if you could get to your seat, we've had a really wonderful session already today, series of three sessions. The good news is this is the final session and we're gonna do our best to get you out of here. There are gonna be three presentations and we're followed by a 30 minute panel discussion and Rex and I are gonna do our best and we promise to get you out on time. The first presentation would be given by Vanessa Huretsuka. She's from the Center for Human Development at the University of Alaska Anchorage. She has extensive experience in the ethical, social and legal implications of genomics research and precision medicine, especially in indigenous population. So we're really delighted when she agreed to get this first talk, which is on American Indian, Alaska Native community engagement preferences and tribal code requirements. Vanessa, over to you. Thank you so much. Thank you for joining us here in the room in Bethesda as well as to those of you that are online in Zoomland and on what was it? Genome TV, I think is what I heard. Sounds so cute that name. I'd like to say thank you so much for having me here and allowing me to amplify the voices of the Alaska Native American Indian peoples that I've been working with and what they've been saying about genomic medicine, about precision medicine, about genetic testing. I come to you from Anchorage, Alaska where I am now with the University of Alaska Anchorage have been for the past three years. Before that I was community based in place. I was a senior researcher at South Central Foundation, a tribally owned and managed healthcare system. And what I'll be talking with you about today is largely from that time, as well as some of the work that I've been doing along with others at the Center for the Ethics of Indigenous Genomic Research. My pronouns are she, her, hers, and just a visual description for those of you that may not be able to see as well as others. I'm an American Indian woman with dark brown skin, brown hair that now has some gray in it here and there and more there and more here than I would like. I'm plump, I'm wearing a flower shirt that is somewhat colorful on a black background. I have glasses on and I'm standing in front of a podium very nervous. I would also like to do a moment to acknowledge the lands of the Indigenous peoples that we're on. I will massacre the names of the peoples because I'm not from these peoples. And I don't think I've met anybody from these groups either, which is a shame and also part of the historic trauma that Indigenous peoples have as part of our history. But I would like to acknowledge the Anacostin and the Piscataway people whose lands we're on here now. I am a grateful guest to be here as are we all, I'm sure. One of the things that I had wished to do was to just quickly describe that a woman, when I'm speaking of Indigenous research, I'm speaking of the American Indian, Alaska Native peoples. As a Navajo and Dine, so Dine is our tribal name and when I'm a Mwentu person, when I fill out my demographics for a research study, I will go ahead and mark down American Indian or Indigenous or Native. And what that means is that I belong to a particular tribe. I am from the federally recognized tribe of the Navajo Nation. And there are 574 federally recognized tribes. When I mark that down, it's often in this race ethnicity category, but it's also for American Indian, Alaska Native, Native Hawaiian people, a political category. And I really wanna state that outright because much of what I'll be talking about and much of what our participants in our empiric research have alluded to is coming from these 574 tribal nations as sovereign nations that have the ability to and are enacting research policy as it pertains to genetic research but also to other forms of data sovereignty. So I think that's an important context point and I do wish to belabor it because so much of the research and so much of the community engaged work that is occurring amongst Native peoples, both here in the United States where we do have that political designation as well as Indigenous peoples worldwide is with an idea that we are at one with other human beings but also with the spirit, the land, the cosmos, et cetera. And that brings us to a different set of values. Here in this slide, I am showing a slide from the Alaska Native Knowledge Network that's describing some universal values of Alaska Native people. In our state, we have over 200 tribes and like any group of 200 groups of people, it's hard to find universal stuff but this group did and I think it's important to contextualize when we're talking about peoples that we talk about the values for which they hold and some of what I'll be covering to use some of this language that I realize many of you are using other forms of language in the way of our technical terminology as it pertains to genetics, genomics and precision medicine and you may be a little bit less inclined to your humanities. And so let me just give you a reminder about a few of the things that you might have forgotten or just put aside since freshman year of undergrad. So I'll be speaking today about terms that are reflective of ontology, the nature of reality and of what really exists. Epistemology, relationship between the knower and what is known. Axiology, what we value. And of course methodology, those strategies that we use to determine to justify the construct of a specific type of knowledge. And I'll just say it outright, the methodologies, those methods that I use in my research are both post-positive types of methods, the scientific method, but I also utilize an employee within my research because it's of and for indigenous people, indigenous research methods. And all of this is to say that these methods, these paradigms are grounded in the principles of relation, respect, reciprocity, responsibility, resistance, resilience, resurgence, restoration and reparation, all are words that you can use when you're doing these categories. But also very important principles, particularly to indigenous peoples. And so I realized with 15 minutes to, summer, a body of research is a little bit hard, but I did want to leave you with a few of those ideas that what I intend to speak about is how the people that I have come to serve in Alaska, the Alaska native leadership of those tribes have sought to create our research codes and research teams and the questions that we ask in Precision Medicine and Ingenomics Research in a way that where researchers and those being researched and those that are caring for those being researched are working respectfully, ethically, sympathetically and benevolently. And I say that because the way that those terms and what's elevated ethically is determined by the culture there. And that's why I wanted to show terminology of universal values there in English, even though they come to us in other languages. So again, for those of you that are not living in the world of social engagement and community engagement, I had wished to put forward a broad definition of community engagement, the process of working collaboratively to proactively seek out community values, concerns and aspirations. And I deliberately use those words. Proactive is what we've been asked from our tribal leaders in the Anchorage area to work on. So when there as a concept of a research project that that ought to be brought forward and under discussion. And under discussion with whom, I think is probably the question that many of us would ask next. In the case of the Alaska native peoples and many American Indian tribes, we've entered into contracts and compacts with the federal government to engage in healthcare services on with and behalf of our peoples. And so tribal leaders are hiring and administering and managing healthcare systems now. The healthcare system that I had worked for is one of those healthcare systems. So the stakeholders of interest are indeed those administrators that are tribal leaders, those providers within the healthcare system and the community members. And many people that are those community members are both those enrolled tribal members from that particular tribe in that area or people like myself that are guests or moved there. And so I am not from the area that I'm currently residing. But I've been grateful to be able to be accepted there. Some of the terms that we'll be discussing today are listed here. And I just wanted to say that these terms are influenced by community members. And again, when I say community members, I'm thinking of not just the everyday person walking around, but also the people that are wearing special badges when you go into the clinical health system as different forms of providers and support staff, as well as those leaders that have hired those providers and support staff in the health system that I worked in, we were called customer owners to further acknowledge that power relationship, not just a patient, but a customer owner, a person that owns the healthcare system and is also receiving services from it. So our community context is influenced by the hopes and expectations, but also by financial resources, by job and family expectations and by the broader social context of which we will hear more on. And when we talk about participation, I like to reflect on a little thing that's from 1969, but it holds true now. And this is the latter of participation that Sherry Arniston had presented in a journal article back in the, back in 69 in the Journal of the American Planning Association. And I don't know about y'all that are in the room. I think those of you that are on Zoom and in Genome TV can see it a lot better. But here on the ladder, the bottom rung of the ladder is, there's 0.1, manipulation, 0.2 therapy, and those two rungs are considered by Arniston, non-participation, areas where it's not exactly participation in the engagement process. And then moving towards number three informing, number four consultation and number five placation. All three of these are described as tokenism. And then area six, partnership, seven delegated power and eight citizen control. And those are referencing citizen power. And I ask you to consider thinking in these different areas as you're thinking about community engagement. If you define that community engagement as engagement of primary care providers or of another stakeholder group, I tend to think of this across stakeholder groups and triangulate between those various groups that I had mentioned. But this is just something I put forward to you. If you haven't seen it and haven't engaged with it or haven't engaged with it in a while, it's a handy way of thinking about participation and in aspects of engagement. So in brief, how do we do engagement? And I think again, this is about something that's been mentioned several times. It's that forming of trust. And for the work that I've been encouraged to do and required to do by our tribal leadership via tribal codes, it's been to have active engagement of various stakeholder types in all phases of the project. I know during lunch, I was just talking and we were talking about, who it is that I work with. And now I'm at the university and I don't work with students. I work with community members. Now the work that I do is with individuals in the disabilities community broadly. Prior to that, I would do work with various members of the indigenous community as co-researchers, developing those research skills and training. So when we talk about active participation in our tribal codes, we speak of conceptualization of the project as an area for active participation, the conduct of the project, both in the way of how we manage the process of research, how we manage resources themselves. And by that, I mean money and how we manage conflict. So who decides what goals? Community engagement can look like a whole lot of things. We've mentioned several of those things here. Depends on the project. And I've listed here in engagement methods, some of those more post-positivistic types of engagement methods. And these are several of the things that we've done through Seager. Some of the work that I've been participating in in this century has been around bio-banking that's owned and managed by Alaska Native people through the Alaska Area Specimen Bank. And then that led to additional work that was through an R01, looking at community-engaged research methods around precision medicine, where we had conducted multiple public deliberations in a variety of different tribal settings. And then some survey work, as well as focus group work, and interviews, and all with indigenous researchers, led by, with the data owned by, the various tribes in tribal consortia. I've provided here a few different participatory research practices in empiric form, and some discussion for you to look at in the future. And just to note that I wanted to end on in the way of a scoping review. So a literature review. And this is something that my team had conducted looking broadly at participatory research in American Indian, Alaska Native communities. And what we had found after this thorough combing of the literature was there was, and these again are papers that were published, things people wanted to talk about publicly, right? And in this, we still see that there is a need for community engagement in the community. A need for community engagement in the early stages of the research processes that there's an importance of guidelines for American Indian, Alaska Native communities, more tribes, and putting forward tribal code that is very specific to various issues. Vanessa, could you wrap up on this please? Yes, I can. Thank you. And then the last thing that I wanted to leave you with was just some things that we each could do in our own work regardless of who we are. When you're working with tribes to respect sovereignty, when you're working with any humans to respect self-determination, to follow the lead of the community, practice transparency, humility, to acknowledge the harms that maybe you have not caused but that exist for the people that you're working with and those that may come yet to build local capacity, make long-term commitments, and be flexible and creative. Oh, thank you very much. Great. Thank you very much, Vanessa. The next presentation will be given by Crystal Totsi and Crystal is an Indigenous Genesist and Bioethicist from the School of Life Sciences at Arizona State University and she's gonna talk about the opportunities for meaningful Indigenous community engagement for population genomic screening. Crystal, go ahead, please. Yeah, thank you so much for the kind invitation and welcome. Yet, thank you all, hello to my people, my genetics people. I am going to do this rapid fire and I have a timer to keep me at 15 minutes. But I do wanna start that even with the recently expanded ACMG panel, only a handful of medically actionable genes have very information specific to those of Indigenous backgrounds. Well, how does this affect clinical utility? What does this do and what does this mean for equity? Now, we've seen variations of this diagram through the wonderful GWAS Diversity Monitor and this other, which I've circled here, has remained stagnant over the last couple of decades as it pertains to the inclusion of Indigenous peoples in GWAS studies or really any type at large of genomics research. However, this is not a matter of engagement via recruitment nor is it selling Indigenous peoples on the benefits of precision medicine and health. It's a matter of overcoming mistrust thinking more approximately about health and empowering data decision equity. This tells you about the structural inequities in terms of how healthcare is funded. Not only, this is a pre-pandemic statistic in that before in 2018, the rate of funding funds to an Indigenous patient was 2.4 times less compared to the dominant population. So before we sell the next innovations and promises of precision medicine and health, we also have to reckon that there is a severe barrier in the amount of promises that the federal government has long promised Indigenous peoples in exchange for resources. And also, we have to, this is a subject of a paper, that when we keep telling Indigenous peoples that they're gonna miss out on precision health benefits if they don't engage, without changing the power and balances that created research harms to begin with, what we're doing is effectively engaging in a cycle of victim blaming and coercion. I also want to say that simply dropping clinical genetic tests into our communities is not gonna solve the health equity problem. This is just a general clinical pathway of care for most rural travel patients that want to seek genetic testing. In this case, an Indigenous patient has to refer it out by their IHS provider, often located 100 plus miles away, which means that they're taking time off of work. They are driving, if they can find a car, pay for gas, get to a hotel, pay for several nights in that hotel just to see a provider, sometimes appointments several months down the road. A question, of course, is whether or not there's a genetic counselor that's available to contextualize clinical genetic test results. At the end of the slide deck, I have a very sobering map that shows you how very few genetic counselors are available in rural areas, in rural states, in states that have Indigenous communities. Are these results interpreted against that relative lack of information on Indigenous-specific gene variation? What is the potential impact in terms of the validity of the information that we're delivering? What training is available to deliver cultural-specific care to Indigenous genetic test users? Are patients being fully informed about default data sharing? And then here's an emerging question that we should all be considering. What are the risks for Indigenous patients who are not covered by federal privacy laws through GINA, IHS patients, and also many of our veterans, a huge portion of whom are Indigenous, are not covered? This is a catch-22, actually, that it's a decision tree that many Indigenous patients face on asking the question of whether or not to contribute DNA for clinical genetic testing. So on the one hand, some patients might decline because they can't afford access testing or they're concerned about data usage. On the other hand, those that do agree implicitly broadly consent to any secondary data usage and data ownership by testing companies. In any scenario, this is why it's a catch-22, they derive little to no clinical utility due to lack of informative relative variants specific to their peoples. This is outside the consideration of many clinical care providers who, of course, are focused on quality of care. But by simply using a clinical genetic test that means that commercial gene testing companies can co-opt and clean ownership of Indigenous peoples' genomic data. This is something we don't talk about enough, that this data will be deposited into public data sets like ClinVar, even if patients know to ask to opt out of dating sharing due to tribal data sovereignty rules, they don't have this option. Urban tertiary care centers that see Indigenous cancer patients may also be biobanking samples and data for research under broad consent without tribal data nation approval. What does this mean in terms of taking advantage of vulnerable populations that don't have any other alternatives? What is this also should raise ethical concerns about the conflation of research consent versus the consent to care for minoritized communities? There have been many studies that show that when you insert a form that consents to having your data being used for other genomic studies amidst your other forms on intake and at a point of care that many patients implicitly trust that white coat. Do they know what they're signing? On that topic of equity, this is something that commercial companies have told me as a co-founder of an Indigenous-led biological and data repository called the Native Bio Data Consortium. They asked us when we founded this nonprofit research organization, how many Indigenous patients have crossed what phenotypes? And we pushed back. Why don't you use our people's data and our DNA to study conditions that specifically impact us? And of this, we had a list of childhood conditions and metabolic conditions and what you all as a dominant population would call rare variants. And we were told this by every single company that it's not profit generative to use our people's DNA to create therapeutics that specifically impact us. So recruiting more people into data sets is not gonna solve the health equity issue. Dropping genetic tests in their communities is not gonna solve this issue. Using the word democratize is a false synonym for equity. When it's a system that's gonna benefit most, it's gonna continue to disenfranchise us as Indigenous peoples. So this is something to consider when you're writing that next grant call. There are issues in terms of using population descriptors. Like all processes of gene flow and drift, Indigenous peoples have had longstanding systems of kinship and relationality that are not mediated by blood, such as marrying into neighboring tribes and also adoption. And our clanship systems acknowledged this heterogeneous background under unified identity. Didn't have people to navigate people before contact had over 400 clans. Many of these acknowledged that we acquired people from Pueblo individuals, Acoma peoples, Zuni peoples, Mexican peoples, Mexican peoples, nation peoples. Yeah, it wasn't until 1934 that the Wheeler-Howard Act was stamped upon us and posed these racially defined population groups meant to define us and dilute our rights to these resources. But these ended up getting conflated under these eugenic notions of race. And it's a dialogue that we are constantly in consideration and I'll bring that up in a moment. But it's very interesting because of a few data sets that are out there. These are from Indigenous groups with completely distinct genetic and cultural histories from our own. From those disempowered groups that probably are not even recognized by their own colonial governments and in Mexico and further south and Central and South America. And yet these are the available data sets with sometimes as few as 30 Indigenous individuals meant to impute something, make some inferential statement about US Indigenous peoples. I don't have to tell you about some of the issues with using PCAs for principal components analyses for ad mixed populations. For Native Americans, we are often left out because our analysis on our small data sets necessitates a different type of small statistical sampling procedures that are incongruent with standard QAQC pipelines. Or worse for people that don't understand how unique our 574 plus barely recognized tribes in the US are, we get lumped into the same category which doesn't benefit anyone. It also reifies assumptions of biological purity. When we are looking for to recruit least ad mixed Indigenous peoples, we are also ignoring the real life the lived experience of other Indigenous peoples in regions like the Southeast and East of individuals that have had more contact and more recent colonial settler history. It also ignores real contributions of inequities due to social and structural determinants of health. We need to rethink one person, one tribe because we often are categorized as one tribe affiliation. Again, our populations are not stagnant and not even since 1934. We do a disservice when we default to colonial definitions of dignity. Do not acknowledge that peoples can belong to multiple tribes. I'm so grateful for and thank you for one thing that I am loving is this move away from genetic ancestry to genetic similarity. This type of type of logical move is something that I think should make transparent some of those statistical inferences that are irrelevant sometimes for our Indigenous communities. But in terms of like how members feel, the job thing that was a topic of my own research recently because we often default to tribal leader to leader our conversations. Forgetting of course that leaders are elected officials. And we don't have the longevity of office or the transmission of knowledge from those individuals from one administration to the next often time. But those realities are consistent with community members. And I simply ask them, will my genetic data be used in other studies without knowledge or consent? That was actually the top concern. Even more than will the research benefit my tribe? That community members rated job and education opportunities created by health research higher than benefits of researching disease or condition. And this is because we think in terms of seven generations. We take seven generations out. We're not thinking about our pathway of innovation. We also need to think about all the other dimensions of equity to include data decision equity. And this is an awesome article report on which I was a committee member. Crystal, could you please finish on this, please? Yes, thank you. So we are now thinking about using AI machine learning approaches, their blockchain federated learning to advance data consenting and data sharing, creating our own portals that advance on group consent and broad consent through dynamic consent, thinking about metadata labeling for the labels on repositories that we don't own. But we're also thinking about building bio economies that are structured at all these different labels, levels of policy and governance. We also have to ask what happens to indigenous peoples outside of federally recognized territories. And I just asked that when we're thinking about equity that we're also thinking about benefit equity as well as decision-making equity on top of engagement equity. Akiahat, thank you so much. Great. Thank you very much, Crystal. The third presentation will be given by Minkid Lee. Minkid is the deputy chief engagement officer for the NIH All of Us Research Program. And she's gonna be speaking about advancing genomic research through community engagement on the All of Us Research Program. Minkid. Thank you. Good afternoon, everyone. Thank you for the opportunity to participate today. I've been hearing a lot of different perspectives and I love the discussion, the active discussion going on. I'll try to address some of the points where all of us was mentioned today and where I think all of us could provide some insight into best practices in community engagement. So the All of Us Research Program launched in 2018 nationally with the mission to accelerate health research and medical breakthroughs, enabling individualized prevention, treatment, and care for all of us. And we aim to do this by nurturing partnerships with at least one million or more participants who reflect the true diversity of the United States, which we've been hearing about a lot today. Ultimately, with the data that comes from those participants that they've so generously donated, we hope to deliver one of the largest, richest biomedical data sets that is available to researchers. So how are we doing? As of last week, we have reached over 500,000, so half a million participants who have consented to the program, consented to sharing their EHR, and also completed initial steps of the program, which includes a basic survey and also providing their physical measurements and biospecimen, either blood or saliva. Over 728,000 participants have created an account, so enrolled and consented to the program, and we have over 400,000 EHR connected to the database in our platform. And then, diversity-wise, we are also doing pretty well while the All of Us Research Program is not solely a genomic study, with the numbers that we have currently, it does make us one of the most diverse genomic studies so far. We have over 80% of participants self-identifying as coming from underrepresented communities in biomedical research. Our program defines those by both age, race and ethnicity, geography, if you're rural, or access to healthcare, disability status, sexual orientation and gender identity, and educational and social income as well. And then by race and ethnicity, we are almost at 50% by the information provided by participants. The main contributor to how we reach this diversity really goes back to the program's intentionality from the very beginning. I was lucky enough to have been a part of the program as part of the Division of Engagement before national launch. And I can contest that. We had funded and we had selected and funded inaugural community engagement partners even before we enrolled our very first participant into the program. So we began with the Asian Health Coalition, PrideNet at Stanford University, the National Alliance for Hispanic Health, 50 forward in Nashville, Tennessee, and Delta Research and Educational Foundation. Each had a specific community that they worked with. And since then, we have not only continued those five partnerships, but we've also expanded our engagement ecosystem to now include 18 nationally funded community partners. If you count all of the sub-awardees through the different partners, we have over 150 partners across the country. And we know that the messenger matters, especially when you are engaging these diverse communities. So we have a grassroots local organizations as well as national and regional organizations, again, conducting various outreach and engagement activities across the country. And as with the expansion, we were very mindful of certain gaps and needs and priorities of the program. When these partners go out and do their work, we work with them within this framework that we've developed over the years. And in our program, we define engagement as bi-directional relationships, development of those relationships, and also maintaining those trusted relationships. Often their work and our work begins with outreach and awareness, again, focusing on fostering trust with those participants, potential participants, and community members. We build upon those activities and go into education and increasing access to the program, which I'll speak a little bit to in a few slides. We focus on engagement activities that can ultimately lead to enrollment and retention in the program, but also again, focusing on maintaining those trusted relationships with the community members and participants. For participants who have enrolled into the program, we really approach them as true partners. We have several community members who have identified participant representatives. Into the program, we have a panel of participant ambassadors and also participant partners who are serving in various program governance committees. So I know earlier we had talked about co-creation prior to the inception of the program. There were 17 community engagement studios across the country. One of them was with Urban American Indian Alaskan Native communities and those community engagement studio sessions really fed into the development of the program protocol. And we've had participant ambassadors serve on the very first steering committee meetings as well. So we are very big on co-creation and co-development. As an ultimate outcome of all of this engagement, we do focus on knowledge mobilization, where either it's the data or the research outcomes that come from participants. We want this to go back to either the participants directly or the communities that they have come from so that this benefits their communities in the end. So from here, I wanted to share some best practices that I think have worked great for our program in hope that other studies can benefit as well. An example that worked from the beginning is really having those partners at the table. So these partners not only go out into the communities and conduct outreach and engagement as our trusted messengers and liaisons in those communities, but they are also at the table at the program level where they are co-creating and co-developing with us. PrideNet is a great example where even again, before the launch of the program in developing those basics survey questions, the questions that collected information about gender identity or sexual orientation, sex assigned at birth, PrideNet worked with us as a co-developer and helped us revise those questions and really perfect them for the participant that we were hoping to engage. Our more recent partnership with American Association on Health and Disability, we worked with them to develop disability related questions in the basic survey last year, which went out and those have been helping us collect disability UBR metrics. And then also we worked with AHD on a communications toolkit. For example of how fast our program works to really implement feedback from these community partners a couple of weeks ago at our programs face to face, one of the participants noted that we were missing closed captioning and they needed it for the meeting. So within a few hours, we were able to get closed captioning added. And when they presented on their disability community toolkit, they noted several best practices not just being 508 compliant and being ADA compliant when you develop assets that person was disabled that could be inclusive of people with disabilities. And they can often tell if the person who is in the picture is really disabled or if you're just putting a person in a wheelchair. So they said that true, truly being inclusive really matters a lot at all different levels. And those are the kind of values that we have been trying to really incorporate into our program at all levels, not just participant facing, but again at the co-development and program level as well. And then selecting these partners, we're also mindful of being culturally congruent in our engagement strategies. Delta Research and Educational Foundation is one of our partners who was doing that kind of engagement. They partner with HBCUs across the country and one of their programs Delta 2020 plus researchers provides educational activities for students and network members within the sorority. So they present a speaker who comes and discusses their perspective research which is aimed at improving healthcare that is relevant to the people who are attending those sessions. Our other partner Asian Health Coalition is piloting a language working group which right now the program is available fully in English and Spanish and they have been advocating to add additional languages. So they're working on a pilot where they will provide additional assets in the program in these five Asian languages and that's under works right now. And they also partner with their community based organizations. So again, partnering at different levels throughout the community is really important and it has been working really well. One of the biggest barriers that I know I've heard today and I don't think we need to mention again is transportation when it comes to participation is a big barrier in times of time, cost, disability, various issues might just be inconvenient. So one of the models that our program noted that works well is to have this mobile engagement asset. We started with one, we realized that having physical measurement and biospecimen capability onboard would help enroll participants from rural and other areas. Those who didn't really have access to an enrollment site, it worked really well. So now the program has two of those vehicles. It's called the all of us journey. If you go on the website, you can see where it's traveling through the country and that journey partners with community partners again and they host them in these big parking lots. The picture here is the one at Stanford University with PrideNet. One event that was really successful was recently earlier this year in Brownsville, Texas and four different partners, the Alliance and AHD and others partnered to host this vehicle. And within a few days, they were able to have 150 accounts created and also almost a hundred physical measurements and biospecimens collected at that one visit. The Alliance calls it the one stop shop model, which works well for a lot of our participants from our UBR communities. So we've been trying to implement that and expand access to people through that model. And I'm gonna dwell a little bit longer on this slide than I had planned. So I hope I'm doing okay on time because I think this is the most relevant to the different population screening and return of results that I've heard throughout the day today. So the all of us research program began returning health-related DNA results to our participants about a year ago now in December of 2022, prior to that we did launch return of genetic ancestry and traits results in 2020. So as of last month, we now have offered genetic ancestry and traits, hereditary disease risk results, which tests the 59 genes of the ACMG panel and then also pharmacogenomics results, which tests the results of seven genes. We've offered each of these results to over 180,000 participants. And you can see how many have viewed their results here. And as of now, we are offering health-related DNA results to about 5,000 participants per week. And while we still have more work to do in expansion and also providing equitable results experience to all eligible participants, I do wanna note that we estimate based on the results that we are returning in terms of the hereditary disease risk reports that about 3%. So I think 2.9% of participants may receive pathogenic or likely pathogenic results. And that when we reach a million, that translates into almost 30,000 participants receiving actionable results. The program does provide clinical validation testing as a follow-up to those participants who do receive pathogenic or likely pathogenic results. And that is actually an outcome of a series of listening sessions that we held with our participant ambassadors and community partners very early on in the planning stages of genetic-related return of results where we heard from our community partners and participants and consortium members that people without healthcare or lack of resources, the results can be very burdensome. So clinical validation testing was added on in response to that feedback. So that's something that we really are, from the division of engagement, we were really proud of that we were able to achieve that outcome. And then all of us participants have access to our genetic counseling resource center. So they can go into the participant portal and schedule a phone conversation with a genetic counselor. You can also request a interpreter into over 200 different languages. And you don't need to have a specific result to request those sessions. So that's also available as a resource. Thank you. Could you sum up in about a minute, please? Yep, I'm almost done. Thank you. Thank you. And we also recognize the importance of diversity on both ends of research. So when the data platform for the All of Us research program launched in 2020, we recognized that we also wanted to have a community of diverse researchers using the data. So we've put forth efforts to help that happen. You can see the number of researchers currently registered as users. And we currently have 97 minority serving institutions registered with a data use agreement. Our division also supports a lot of research engagement work to support that, recognizing that there is capacity building efforts and things like that that are required for researchers to use our data platform. PrideNet held a researcher boot base camp earlier this year. We have a partner at Baylor College of Medicine who hosts annual faculty summits. It's a three day workshop, essentially training people to use the workbench, but also providing mentoring and support for like grant application writing, things like that. And they follow through for the whole year with these research teams to ensure that they are getting what they need in terms of support to use the workbench. And then we have one other partner, a University of Utah who hosts is a high school student, high school teachers every summer to come and use, learn how to use the workbench and learn about the program. And these are just highlights. I'm happy to talk to you more about other partners as well. And that's it. Thank you. Thank you very much, Binky. We're now gonna move directly to our panel session and I'll call on my twin brother Rex to take it away. Thanks, twin brother. So we've heard three really interesting presentations and I wanna make sure everybody writes down the questions that they have relative to these three presentations because we will be including them in the discussion after we have the panel. And I did wanna recognize actually that the title is fairly uninspired for this panel. And so I wanna thank Caitlin for suggesting a potential alternative title too late to get into the booklet of course, but beyond actionability, discussing the who, what, when and why of expanding population screening to new conditions. So maybe that's a better title for us to think about going forward. So I would invite our four panelists who are Caitlin Allen, Ned Kalange, Jessica Hunter, and Bob McNallis to go up to the front. We're gonna actually use the panel chairs that have been there all day long. And what we're gonna do is we've asked each of the panelists to in two to three minutes and we're gonna try to keep them to that to just make an opening comment about how they think about expanding population screening to new conditions and I should say also to new communities. And we'll start with Caitlin and just go in alphabetical order. And then once we've had the initial comments, then we'll have an opportunity for some discussion about the points that emerge from that. And then we'll open it up to the floor and the folks online. So Caitlin. Thank you. So we at Medical University of South Carolina have recently as of 2021, November, 2021 implemented a population wide genomic screening program. And as I mentioned earlier, this was a decision that we ultimately made was that we would do this as a research protocol as opposed to clinical care. And our goal is to enroll 100,000 individuals into the program and we're at 35,000 participants so far over the course of about two years. We do whole exome sequencing and we are working with Helix as our laboratory partner. So they're CLIA certified and all of the results for this program are returned into the EHR. And we also provide free genetic counseling for all positive individuals. So I'm saying that to just kind of lay the groundwork for where we are and how we, I'll let you know kind of how we got there and to Rex's point about future expansion. So currently we're looking at or we're screening all adults in South Carolina. We are doing this, we're returning tier one conditions and the hereditary breast and ovarian cancer, Lynch-Genderman familial hypoclustralemia. And we did this, made this decision as an institution because these were well vetted CDC tier one conditions. They're considered actionable. All the reasons we've sort of talked about today, but that was something that leadership felt comfortable with moving forward with the CDC tier one conditions as our package. And then considerations that we had early on are a lot of what we have been talking about today. So ensuring that participants are understanding their findings, both positive and negative results. We're making, really trying to make sure that we're be able to connect individuals to clinical care and being able to then ultimately track those downstream clinical outcomes. So future expansion that we have in mind but have not done are actually thinking about pediatric populations and potentially expanding to provide this type of free screening service for our pediatric population, prioritizing more rare conditions. We're also thinking about and actively expanding in the pharmacogenomics space. And so returning results for genes that are associated with our priorities as far as pharmacogenomics goes, as well as returning secondary findings in particular APOL1, PKD1, PKD2. And part of the reasons for focusing on those particular genes is because of our state's sociodemographic and sort of health concerns with kidney disease morbidity as well as disparities in that space. And then I think other considerations as we're thinking about expanding are that we have this under a research protocol, which is great in a lot of ways that allows for us to re-contact individuals who've participated, but we ultimately would need to re-consent them to provide them with additional findings. And then we're also thinking about just what the right combination of considerations are for expanding and looking at ClinGen guidance to help us think through that. Thanks, Caitlin. Next, we can move to Ned. Do you wanna give your introductory comment? Hi. So I have kind of three things to talk about. EBM, old, EBM new, research and equity. So there are evidence-based methods, evidence-based medicine methods that have resulted in recommendations for genetic screening on a population-based level. So recognize that the USPSTF does that. Their recommendations are population-based recommendations to be offered to everyone in the groups that are identified for benefit versus harms. And they have one recommendation around genetic screening, which I participated in years ago and they're re-looking at it even as we talk today, which was BRCA one and two. So I think hereditary breast and ovarian cancer screening has a footprint with evidence-based behind it. And there was a method that got the task force there. Then the EGAP working group did its work and from those, I was there as well, we came up with the other tier one tests, Lynch syndrome and familial hypercholesterolemia. Then I retired from genetics for 12 years. 12 years later, we have three conditions. The tier one, I'm just a little amazed. And that's so that tells me it's time to rethink about the methodology to kind of look at what Mike wants to do and look at what Les was talking about and saying, we're not talking about lowering the bar, we're talking about criteria that would help us get the evidence necessary to add additional tier one tests when we start to think about population-based screening. So what I heard from Mike and I heard from Les, we're kind of where the uncertainties are and can we identify those uncertainties and get a set of tests that are almost ready for prime time, that is we can specifically identify what gaps would have to be filled in to get us closer to meeting those bars of either the EGAP working group or the USPSDF and therefore provide additional tests that have that kind of, sorry, we used to say that at the clinical guide, that kind of bulletproof recommendation that you know if you do this, people are going to have better health. So those are those issues. How does that relate to research? I'm glad North Carolina is doing that. I would make a slightly different recommendation. South Carolina, oh, sorry. Remember I'm from Colorado and I know the Mississippi is between us and that's about it. I think the other approach is to think about funding integrated healthcare systems to pilot these. So we do this, well, we don't pay for it, but we do it in newborn screening in order to be considered to be added to the routine uniform screening panel. There has to be a pilot study that's identified at least one case. And CDC may be paced for that every now and then and NIH may be paced for it every now and then. But thinking about taking on a guy singer or a Henry Ford or a Kaiser Permanente and I don't mean to leave anybody out. Inner, inner mountain, there's three or four or others. Well, that would be a great setting for saying we're going to have a pilot of a population-based test for genetics. So Terry and Eric's gone. But if you're thinking about things that you could actually screen, I'm sorry, fund that might end in moving the bar a little forward towards new conditions being on the tier one. That's just a strategy I think about. Then finally, we were reminded so well by the articulate and amazing speakers in this last session that there is this incredible opportunity to make recommendations that increase inequities through genetic testing. And so all of the work we do needs to have the component of community engagement and the lens of how can we do this in a way that does not increase health inequities. So those are my comments. Thanks very much. Yes, the hunter is up next. Yeah, so I've been coming at population screening from the context of clinging. So I lead the actionability work where we're doing evidence-based assessments of actionability generating reports that summarize actionability of gene-conditioned pairs. We score four factors associated with actionability, how severe the outcome is, how effective the intervention is, the nature of the intervention to the patient and how likely the outcome is to occur, AKA penetrance. And all of the scores, these semi quantitative scores and our final actionability assertion are available on the website. We've developed this framework and the context and secondary findings, which is why most of the domains are based on the individual. But we're recently shifting to more population-wide and we've been adapting this framework for the context of polygenic risk scores and now shifting as well to population screening and thinking about as part of that effort what factors need to be added to this framework in order to account for a population-wide impact. Severity, at that point, it's not just severity in the individual, but how impactful is this conditioned at the population level and those kinds of things. And so, yeah, we're beginning to think of what factors we need to include in that framework, a lot of which we've learned through the polygenic risk score effort. But for community engagement, I will say that we have to have this effort to be quite, sorry, that's a little distracting. Somebody logged in. Good afternoon, everybody. And forgive me, I'm just recovering from a cold. So my voice is a little smokers cough. I'm from the Office of Disease Prevention at the National Institutes of Health. And unlike many of you, I don't speak genomics as well as you do, but I do speak prevention. And I was actually quite inspired by Les's talk this morning about that genomics is not an exceptionalism. There's no exceptionalism there with regard to prevention. The Bayesian Theorem still apply to all of the work that you do. At ODP, what we're interested in is sort of assessing and facilitating and stimulating prevention research across the 27 institutes and centers at NIH. We do that partially through developing and coordinating and then implementing some of those prevention programs. We work closely with the US Preventive Services Task Force and the Community Preventive Services Task Force and healthy people to really identify key evidence gaps in prevention. And I think there's some other opportunities here to work with all of you to identify some of the key evidence gaps that might exist as you think about population screening. For us, I know as we look across our prevention portfolio across NIH, of all the prevention research, about a third of it actually looks at leading risk factors. About a ninth of that prevention research actually tests a randomized intervention for that leading risk factor. And about a 20th of that actually tests that randomized intervention for a leading risk factor in a health disparate population. So really, very small representation of health disparate populations, even in traditional, or should I say, just run of the mill prevention research. I think that provides us some opportunities. We've added a whole area of our strategic work that involves advancing research in health disparate populations. And really, I think the important thing that we can offer is ways to understand and address research gaps. And I heard a lot of research gaps today. And I think I can offer a couple of reflections on that a little bit later. But just to let you know, one other sort of interest that I have is implementation of this work. And we heard a lot today about how it's gonna fall on primary care clinicians. We've got a little bit of experience with that. I'm reflecting back, not quite 12 years ago, but 2000, 2005, the American Academy of Family Physicians held its educational focus on genomics to try to help family physicians embrace the era of genomics. I think here we are 18 years later and I'm not quite sure AAFP has, our family physicians in general have embraced it at quite the level that they might have hoped. So I think we do have support there, but I think there's lots of opportunities too. So with that, I'll stop and look forward to the discussion. Okay, so before we open it up to general questions, I wonder if each of you could just take a minute or so and comment about what aspects of community engagement you've used in starting up your projects and how you do that in an ongoing way to make sure that your work is continuing to be relevant to the communities that you're doing your research in. I shouldn't say partnership with. So for our work at MUSC, we developed our own community advisory board. So there were existing patient family advisory councils that the hospital has in place and we felt strongly that we needed to have our own community advisory board specifically for this project. When trying to identify people that were going to be part of the community advisory board, we were very intentional about representation from across the state of South Carolina. So MUSC is in Charleston and mostly as a footprint there but our screening program is designed for the entire state. So we really wanted representation from across the state. We also wanted to find folks that would have differing perspectives about genomics and screening and not all of our community advisory board members agree or that they have varying opinions and that was intentional. And so that's been important for our work as well as the engagement of our clinicians early on and some of the co-creation and decision making we had about how to just set up the program. I guess I'll talk from my role with the state health department and say that as we put together programs now that are designed to improve delivery of care in historically marginalized communities, we make sure first of all that our advisory committee includes a diverse group of individuals with lived experiences for those that are members of the White that still makes up our state. So I think that outreach and inclusion is important. And then the next step is as you design a program even with that representation to take it into communities to a broader discussion with more people who have lived experiences differed from the ones on your communities so that you make sure you have enough input to tailor the delivery of a program that's gonna be most successful for a given community. Well, and I touched on this a little bit already and getting community engagement on kind of validation of our metric. And I think that will be particularly important as we move forward towards population screening. But for our individual reports that's been a bit more ad hoc on when a little red flag pops up maybe and this is an important report to get feedback on but I think perhaps going forward we need to be a bit more systematic about that. And I think that would be a really important addition to our efforts. Thanks, and I'm not speaking for ODP because I'm not sure we engaged public quite the same way but coming off a couple of experiences that I've had recently with us in some research areas. I know Colorado actually has University of Colorado because you're thinking of the boot camp. Yep, really good results. There's a team at University of Colorado that engages patients in development of educational materials. And they found better uptake of whether ever the intervention was whether it was blood pressure screening, cholesterol, whatever it was getting patients directly engaged in developing the educational materials are used is an effective way to help engage patients and then help improve the educational process. The second one is from the organizational standpoint and I'm not quite sure what ACMG's approach is but there are other groups that actually at professional meetings bring together patient clinician or patient researcher pairs, diads to help make sure that the patient perspective is represented in every aspect of the work. Not just as somebody to come in and just get their perspective but actually fully engaged in the work. And that also seems to be a very effective way to keep people in line, not in line but engaged in every, along every step of the way. Thank you very much. So we heard a really nice description from Minky about how all of us has reached out and engaged communities. I'd be very interested to hear from Vanessa and Crystal how that might look if we thought about engaging your communities. I was really struck by the fact, Crystal, that you said that only five ACMG medical action, medical actionable genes were even relevant. How do we do a better job? There's two questions there, first of all. I wanna highlight something really important and I'm actually on the Indigenous Research Working Group for the all of us research team. That's where I stepped out about an hour ago and I had this exact conversation that there is a distinct difference between engagement versus research capacity and training. Now, they can be part of the same spectrum but they are two polar opposite sides of the same of that spectrum and you cannot use one versus the other. So when we're going into talking with community members and tribal leaders, we cannot say, oh, we're training students, checkbox, engagement. That is a completely separate activity than what we're calling for, which is the more difficult activities related to conversations that are difficult, building trust, good quality of research and not just surveys and focus groups at democratic deliberations. And Vanessa has given us a couple of great examples, several great examples, the Seeger that we have trialed in our communities as well. So I just wanna have those two distinctions. The other part about the medically actionable genes yes, so of those, those are mostly centralized on colorectal cancer, gene variants are especially related and there's a recent report that stated that that is now the number one cancer in our communities. And so there's obviously a point of intervention here but there's also a lot of concerns. So in terms of when we refer an IHS patient out, what tests are we giving them? Is it relevant? How are those tests results being returned back to the community? Are they validated in a research setting versus a clear certified setting? I know it's not usually a concern for most institutions but it is a concern for institutions that are smaller and that our community held. So there's a thing of scale here but then also what does that mean for meaning for one community versus another, right? And there's just a lot of concerns about Gina, a lot of concerns about harmonization of data. The other definition that I wanted to just state is that we're talking about population screening in like research domains and then public health domains but those two spheres are separate in terms of their ethics and in terms of their regulations and in terms of how those are sanctioned. So, and especially at the tribal IRB, IRB realm, I mean, that there are rules and like past practices for engaging with tribal IRBs when they're recognized by the NIH because under the common rule and single IRB mandate they're not always recognized. There's one like 33 of them. IRBs are not recognized, other forms of tribal research governance are not recognized but we're talking about public health now. That's an entirely different animal. We're talking about different federal agencies. We're talking about different legal proceedings should something go wrong. And then how the IHS and pathways of care especially even for, you know we're just talking about rural tribal settings there's also separate considerations for urban individuals of which according to the last census now 80% of indigenous peoples of AIAN communities now reside in urban areas. There's a lot of issues here. So sorry, I just want to unpack that. Crystal, I'm gonna put you on the spot. Crystal's a great follow on X by the way formerly known as Twitter but you said you had a post earlier today not from this meeting but from another meeting about indigenous leadership. And I think that was, you didn't quite go there. I thought you might but I'd be really interested in your thoughts about leadership of research as opposed to partnership and research. I'm gonna point it back to you. Can you clarify a little bit more? So you basically said that for a particular proposal that it was discussing indigenous leadership within the proposal and the proposals reflected engagement but not leadership. And so that seems a different level of engagement and you were starting to get there but you didn't quite get there. And I think you have some thoughts that would be important as we think about the research agenda. I have a lot of thoughts. Okay, well, right now we're presuming research from a very, our common pathway of these are research proposals initiated by individuals with PhDs working in academic centers and then they're reaching out to communities. And even the work partnership is either thrown around really loosely and it's not really respected in the same way in terms of resource development. The post that I mentioned earlier or wrote earlier was something to the effect of we can no longer call things indigenous led when institutions have 57% indirects or sorry Hopkins near 100% indirects and those go to PIs when most of the grassroots activities are actually from community members themselves. And they're the ones that are generating the data, the protocols going through all the approval processes and they get a tiny fraction of that. And this is something that we need to consider for the NIH models. Now, NIH has been really great in the last two years. You've started to really consider that you do not need to have a terminal PhD status, which is important when you consider that the Seattle Indian Health Board Director who just stepped down recently is a master's level investigator. So this meant that for individuals like her, myself, Joseph Earshada, who's the executive director of the Native Biodata Consortium who were at me recently master's level, we couldn't even be PIs of our own research. So, and also what that meant is in terms of contracting with universities, we were always having to have our lawyers stuff in which we have lawyers, thank God, but that took a long time. And that's the first time and not the last time I'll say thank God for lawyers. But we have a single IRD mandate which has, you know, and common rule, trans language about respecting tribal sovereignty. That only means for tribes that have an IRV that has an FWA, which is like a small, small fraction of the 574 recognized tribes. Those then, you know, you could still work with a large-name institutions that just happen to have institutional standards of refusing to see review to external IRVs to include tribal IRVs. So like how do we win? Even if we wanted to partner with universities, we are constantly at the wrong end of that decision-making equity and power and authority. And there's something to be said of the distinctions between respecting versus operationalizing indigenous data sovereignty. Okay, thank you. I wanted to make sure, Vanessa, you had a chance. Just to ask about the question of what do we need to do a better job? How do we need to think about different ways of thinking? We've heard some great ideas already about helping address the inequity that we have inherently with indigenous people. Well, first off, I don't believe that the inequity that the American government has with indigenous people is alone to indigenous people. I think we were talking about historic inequities with many, many groups of people. And so there are some concepts that ought to be considered, and which is exactly why I put together that I highlighted that ladder of collaboration of engagement. At the very least, we should start attempting to climb such a ladder, recognize that that ladder exists. And then when we move forward on our work, be it as an individual or as a system, that we move up it. I think one of the things that I was hearing from Dr. Lee was, in the example of an individual with disability that is being highlighted on something as simple as a flyer, that indeed the community will recognize if a person is an able-bodied person, that is a model that's sitting in a wheelchair versus an individual that has had that full-lived experience as a, for instance, which is a tokenism, right? Straight up. And also the material. Recording in progress. To the other underrepresented minoritized groups and also the ones not fairly recognized. Can I just confirm? We gotta note that the folks online are not getting sound. Is that still correct? Sorry, Gertrude, either of you, I think what you just addressed relates to that and you could briefly summarize for us. So, oh, you want me to rephrase the question? Okay, it's really interesting. We talk about biology in terms of distinctiveness across populations, like between and within, but then we don't even talk about relationality in terms of how, like in terms of respect of our kids. And I think this is really important because, you know, we have federally recognized tribal nations, but we don't acknowledge their cousins, their neighbors, their, and which have distinct and different types of historic harms, but we try to legislate across genomes and genomes don't care. So this is where I think in terms of the principles that Vanessa brought up in terms of thinking, in terms of respect and humility, but also extending that to how we are thinking and globally and ethically. So I would just like to add that one of the things that I hold to with the work that I've been able to do is the role of sovereignty and how our tribal nations are able to force the issue of the research codes, of the meeting behind those research codes and other marginalized groups don't have that opportunity. And so it doesn't mean that they don't have similar values, that they don't have similar desires to be included in research, to drive the research, to have power sharing, to be all of, to do so many things, all of those Rs that I rattled off earlier. Tribal nations instead have that political authority to engage in tribal consultation, right? One of the things that I had the blessing of being part of was developing the tribal consultation policy for the NIH. That's not something that other marginalized groups get to have because there is no authority to do that. It doesn't mean that there's not the moral imperative for all of us to follow through on such things. And so that's where my mind goes is that we have a responsibility to our various marginalized communities to inquire in the same manner, be it required or not, when we are engaging and seek to engage in whatever type of work. Even in the absence of a legal, no, that's great, thank you. So I think we'd like to open it up for a broader discussion. So go ahead. So I think one of the kind of underpinnings of dissemination and implementation research is that we wish to make the fruits of research as broadly disseminated as possible. And that's a little bit different than thinking along the lines of basic science research, right? But it's engaging with communities and it's sort of finding ways to make something work, right? And so do you see a difference in the way that engagement should happen, say let's say in my state of North Carolina, we do have Native American populations? And if we wanted to say let's have a statewide effort to do this genomic screening, because we think it's useful. So how do we go about doing that with the patchwork of constituencies that we would want to engage with? That's the best way to do that. Yeah, if I could go ahead and start the response. One of the slides that I glossed over quite quickly as I frantically pushed the forward button was a slide that I had to deal with a newborn screening. And in Alaska, one of the secondary additions to newborn screening was a, in what was it, 2003, 2006, somewhere around there was a CPT-1A Arctic variant. And the addition of that variant as well as materials that were developed for the public, those that had screen positive, those were developed between the state of Alaska as well as the Alaska Tribal Health System. And they were directed to providers of care for Alaska Native people that were within the Alaska Tribal Health System. Alaska Tribal Health System is, again, managed by Alaska Native people and is caring for about 40% of the state population when you look at managed care and clinical care there. Kind of sticky thing there is that the materials are being mailed out when a person has a positive. It's a CD, like a DVD, hasn't changed in all these years. And it's not something that was, those materials as patient education materials hadn't been tested with everyday people that were receiving them, but instead were to inform the providers of care for those individuals. It was until a couple of years ago that the health system had put together a research project that was NEH funded to do some community-based inquiry, both with the current providers as well as with the patient population, particularly those that had received such things in the mail and had received information that their child had this particular variant in a double whammy sort of way, right? And these focus groups had indicated that indeed the people had, and the providers still had a lot of questions and the materials weren't quite as helpful as we had all in public health had hoped they would be. I see that little story here because there are procedures that we can all think of now after the fact of what engaging with individuals, with parents that are receiving this information ahead of time and developing it. I mean, goodness, even the title of the materials is something in regards to the metabolism, personally, in the way of the materials. But anyway, I use that as a little case study in the way of what could be done different in the four community engagement on something that is highly specific to an indigenous population and Inuit population, not just here in the United States, but other Inuit populations where that is there. So we've got four hands up and there's one hand on the line. Yeah, sell that one. So Mark, Pat, George and Bruce, and if anybody else wants to see a line, make sure you get your card up, but Mark, you're up. Thank you. This is to the panel, although the handicap here is that my question's about somebody that's not represented at the meeting, and that is we're talking about engagement and there's one entity that has been specifically mandated to do patient-centered outcomes research, that is PCORI. They also have a mandate related to rare disease, which they've had a real challenge meeting their expenditure in the rare disease space. And it seems like there's some opportunities there, particularly since it's for engagement research. And so for any of you on the panel or Alana, who I know has done a lot of work with PCORI that can comment on whether there could be some opportunities for engagement with PCORI in this space, I'd be interested to hear your thoughts. Yes. Yeah, no, I see. Hi, yeah. Could you elaborate? Yeah. All right, I cough when I laugh. PCORI is great. I mean, and they have now years of experience doing this kind of engagement work. I think they're a great target for us to work with. We've worked with them on a couple of different projects and they bring, I think, a unique perspective to it. They build a patient advisory group with each project as it comes along. I think there's some real models they might be able to tap into and use. So I think that's a terrific idea and you're right. I wish I thought of that earlier too, to mention it. Yeah, I just wanted to add to that too. I think we have seen some, I mean, the fact that we're talking about this here and as a grant reviewer, both for PCORI and for NIH over the years, I've seen some of, a lot of the PCORI thinking move its way into the NIH grant review process. We're like, hey, they didn't include, there is no engagement at all in here in an NIH grant. So I think that, yes, there's definitely we can do more, but I think there is a little bit, and I think just what we've seen here and that we're talking about it is the right steps and I'll let the questions continue. So Pat, I think you're next. Okay, yes, my question's for Ninky. I had a question about the all of us research program. So I thought you did a great job of giving us a really good overview of the diverse engagement efforts that all of us pursue. And then I was personally just interested in the last slide that you showed that said for both hereditary cancer and pharmacogenomic variants, then only about half of the participants view the results. So my question is, was that in line with your expectations and do you have any under, have you explored the reasons that only half have chosen to view the results? No, it wasn't, especially since it was a long time coming. So a lot of our participants had voice that they were waiting for it and it was taking too long. So we definitely expected a higher percentage of participants to view their results, especially with all the support that came with it in terms of genetic problems and the reports themselves also look very nice. They're visually appealing. We had a lot of community partners review the language on them. It comes with information on the gene and then just ways to understand the information better. The team is looking into how to increase that. But yeah, right now that's where it seems to be. And for some of the participants, if we anticipate a pathogenic or likely pathogenic, sometimes we do, those are the cases where we do require them to set up a meeting with a counselor in order to receive their results. So that might be an additional threshold in viewing their results. But overall, once they do have a consultation with the counselor, the satisfaction score is very high. It's around 60%. So we don't think it's something to do with the experience itself, but something else must be in action there. And we are having to look into it. It hasn't been less than a year. So maybe it'll just take more time. I'll just make a quick comment that that percentage is actually higher than people see with Biobank return of results. We were just discussing, like for both Penn and Geisinger, that's 50% is actually much higher than we see with trying to return Biobank results and whether people respond to that. We do acknowledge it is high. Bruce. Yes, thank you. This is actually a somewhat technical question, I guess, for Crystal and that is you mentioned the use of blockchain and the very, very, very little bit I understand about blockchain has always made me wonder if it's a way for people to control their genomic information so that they can share it when they want and with whom they want. And I'm wondering, first of all, is that accurate? Is that how you're using it? And then if so, is it scalable and could it help to get us out of the quandary where we have genomic data frequently kind of locked in place, like within an academic medical center, for example, where it isn't easy to port it to other settings? Yeah, so blockchain is more at the infrastructure level and there's smarter people that work within data systems around the table. But basically, first I would say that dynamic consenting is something that provides an individual level of sort of decision as to whom somebody might consent to have their data shared with types of data, they are willing to share rules in terms of data dissolution, are there cultural rules that they would prefer to enact that they wanna remove their data or draw down their data and you can employ dynamic consenting models actually at the earlier design. So I think that UK BioBank employs this type of approach. This is also an approach that we develop at the Native BioData Consortium as part of our infrastructure so that we can license it to other travel nations worldwide for free. The thing about blockchaining is, and it's very important here, is you can sort of define different types of user access, read a rules, but those type of levers are usually done by whomever is designing it, right? But, and then again, I'm speaking more from a community centered approach, you can actually have community members act as the authority nodes for deciding who gets access to certain types of data, I know what conditions, what study types, et cetera within partnership with tribal nations. So sometimes this is like positioned as an individual versus group consent, speaking to an author of a great paper here that interviewed tribal leaders and they say, and one of the leaders and elders or was older and older, both, okay. That stated, it's not individual or group, it's both. And it's actually up to tribal nations to decide how they're gonna make those decisions not outsiders. But in terms of how tribal nations can govern, you can have both an individual level of control and as well as at tribal nation control. Blockchaining is one way, another way is federated approaches where you can actually like read and maybe to execute without having like full access to all of the data within independent nodes that are governed and housed by communities. And this is something that many other systems are employing as well. I can't tell, is it Alana or Kate that has their card up? I made my comment earlier that okay, sorry. Return of results was just like 50% pretty high. Chris? Yeah, I had a follow up question for Mickey Young. So I think you mentioned from your presentation that genetic counseling was offered to all of us participants even if they did not have the results returned. So wondering what kind of uptake you got on that and what were their questions if you know that? Yes, I'm thinking the only information I took away from that in preparation for this talk was that the CSAT scores were high, the satisfaction scores were about 60%. The specific questions I would have to look into that in terms of what they're asking. Yeah, I don't have the feedback off-hand right now, sorry. Is that a second question or no? Great, so I wanted to ask Vanessa, you mentioned in the list of things early on in your talk that should be considered, you mentioned thinking proximally about health and I didn't quite understand what that was. So could you explain what you mean by that? Honestly, I don't know what I meant by that right now. I'm being honest, I'm not reflecting, I'm not recalling where I had that particular point. So I apologize. So did it, I did it, it made, you know, because to me, I might interpret it as the health needs of the community that are primary to the community, but I could put any, you know, interpretation, but maybe you know, or you have some thoughts on that? I have that sentence on a couple of my slides. No, often and unfortunately, the Human Genome Diversity Project, the thousand genomes, genographic projects are great, great unfortunate examples of this in which researchers would enter into communities, promising things that therapeutics will discover the thing that is causing, for instance, the NIH NADDK made a similar promise to the often peoples like in 1965, we're going to treat diabetes and then we not telling disclosing that maybe it'll take a long time for these interventions to actually be translated back to their health and meanwhile, community members are wondering what happened, where'd you go with our data? So that's what I meant by thinking more proximally in terms of the promises, messages, engagement, et cetera. George? Thanks, this has really been a terrific session and as I reflected on the values and the principles that you mentioned, I was reminded of a major principle that we learned as part of the NIH-wide Community Engagement Alliance, particularly during the COVID era, was the issue of trust and we think we know when trust is enriched or when trust is eroded, but from a research perspective and in your experiences, do you have to worry less? I mean, we definitely want to be trustworthy and we want to do everything possible to get there but how do you measure that and what have you found useful from a research perspective in answering questions about trust? If I might go first. Within, let's say, the organization that I had worked for, South Central Foundation, there was a requirement, a research requirement that researchers were to put forward their concept and a concept proposal of one pager. So they could submit, let's say, their specific aims page as well as a cover page, saying this is what my intention is and why we think that we'd like to partner with this agency and cure the outcomes of interest and the procedures that might be used. Again, very broadly, I would say presence or absence, even of the request is an indicator of trust. We had many researchers that once they discovered that that was a requirement for approval, they were out, not interested in participating, not interested in partnering. So just to get presence or absence to engage in a process that's community-led would be a very low bar, one might think, but it's an indicator. There's an emerging field of equity metrics. We keep talking about equity, how do we measure it? That's actually a very interesting question. Some of this is rooted in the health economics. I have a concern that health economics is really short-term and also may substantiate systems of that are incongruent with actual community health, but that is a starting point. In terms of tracking whether or not these researchers are delivering on the promises that we have made, one way to do this is tracking research dollars into communities. So for instance, with Indigigata, which is an indigenous data science education workshop that I started, we actually shifted all of our research compassion training outside of universities into tribal lands. So we're funneling outside dollars from NSF and also philanthropic orgs. And other than paying for or not paying for airlines, everything else goes to tribal nations at a rate of 67%. That is something that we is like above the standard of other types of training opportunities that have federal sponsorship in some form. We can do this with similar other types of grant mechanisms when we start tracking where are those indirects going, where are those directs going, who are we actually training, et cetera. And again, also aligning that education is not the same as a direct, thank you. Perhaps could I have a quick follow up from Vanessa, please, because something she said actually just triggered a reflection, very often when we put out a notice of funding opportunity, we don't always do it, but sometimes we ask for a letter of interest. So did what you say apply also to letters of interest when before we even ask you to come in for the application? Yeah, so let me just step back. The process that I was engaged in with the tribal leadership of South Central Foundation, which gets their tribal authority from the Cook Inlet region, began with a concept proposal phase. And then if approved, the researcher and their team would, and this was also true of those of us that were embedded within this health system. So it wasn't, they do that and we do something else. No, every single researcher, student as well, would complete the concept proposal phase that goes all the way to the board of directors. If approved, then the person would move forward in partnership to develop a full proposal. And then after that had been written and approved, that full proposal would go back through a full process. Now once everything was in hand in the way of this is what the engagement would look like, this is whatever recruitment flyers and materials would look like, these are our research team members on Nazium, on and on, these are the actual majors. Then at that point, then that's when there would be a suggestion of moving forward with funding application. So again, this is a really detailed manner in which engagement would occur where there would be a hand and this required is how that group had chosen to have a hand in the development of the research question, the processes, the et cetera, et cetera and embedding that information within the healthcare system. All right, well, thank you everybody. This has been a terrific session. Appreciate both the, all of the panelists and the presenters as well as the questions. And I feel like I need to summarize but at great peril here because there's so much to be talking about. But first and foremost, I think one of the most important things I heard is the importance of building trust with the communities that you work with. Second, I think I heard, we use the word engagement a lot but that engagement's not enough. We really need building on trust but we need to really think about true co-creation from the beginning of the project. I also heard that I think we need to recognize the need to have a different view of what community leadership means because the community may not always have the same educational opportunities and the same socioeconomic opportunities that the traditional academic community does. So I think that's something that was important. I also heard the importance of recognizing marginalized communities that are not only marginalized but don't even have a voice at the table which some are fortunate enough to have. And so I think we've heard the need in the context today that we really do have a lot of work to do in terms of thinking about making sure our databases include all participants, all humans and make sure that we have a commitment to doing that. And I think I will stop there but I'm sure George, do you have anything else you'd like to add to that? Not a whole lot Rex, I think you did a terrific job only to highlight the importance of the principles and the values and the terms that Vanessa mentioned I think trust is in there. The second thing was also many of the challenges that traditionally in our NIH funding we haven't paid attention to in working with community partners or what can I think it's a little different for sovereign entities but really rethink how we fund and how we support particularly small community-based organizations and small partners. I think the final thing I wrote was paying attention to the power imbalances. We may not think of them that way but we have our structures and we go through with our structures but they may really reflect serious power imbalances that go against meaningful community engagement but this is really been a true of rich dialogues so thank you all. Thank you all and I will just say I think all of us that work with participants in general regardless of where they come from we should put a copy of that ladder in front of our desks. Great, so that thank you very much to everyone. Let's give them a round of applause and thank you very much for sticking it out to the bitter end here of today. We have tomorrow and we'll start at nine o'clock with the session 8.30 with breakfast. No, I'm so sorry. So this is why I have my friends here. Yes, so 8 for breakfast, 8.30 for the session. Thank you very much, goodbye.