 Hello everybody and welcome to our genomic innovator seminar series. This is our final seminar in this series. So I really appreciate everyone attending. And thank you for being here today. So my name is Chris Gunter. I am the senior advisor to the director for genomics engagement here at NHGRI. And I'm excited to introduce you to my co-moderator and our speakers. So if you remember we decided to come up with this seminar series because what we wanted to do is highlight some of the early career scientists whom we have won some of our genomic innovator awards previously. Those are awarded for early career ideas, which are look like they have the promise to be very exciting. So my co-moderator today because we're focusing on genomic medicine is Dr. Rob Rowley. He's a program director in the division of genomic medicine at NHGRI and an internal medicine physician who joined NHGRI about five years ago. So you'll hear more from him later because he is going to be asking questions of our speakers. So the kind of run of show for today is that I'm going to introduce our speakers, and then they will each speak for about 20 minutes. And then Rob will ask them some questions that he has come up with and looking at their work. And then at around the four o'clock ish mark we're going to take questions from you the audience we really encourage you to ask us questions. The chat function is disabled, but you could submit questions via the Q&A so I encourage you to do that and I'll remind you of that throughout the seminar. And we will start looking at those around four o'clock ish so please feel free to put those in as we go along so that we can be collating them and then Rob will be asking those of the speakers. So let me introduce you to our speakers our early career investigator is Dr. Jason Bassie who's an associate professor of medicine at Harvard Medical School. He's also in the section of general internal medicine at the VA Boston healthcare system, and he is a founding member of precision population health at area the labs. So he's a practicing primary care internist and researcher in the implementation and evaluation of genomic medicine interventions, which is what it's going to tell you about today. And our more established researcher who's going to add a lot of valuable context to more work in this area is Dr. Katrina Goddard she's the director of the division of cancer control and population sciences at the National Cancer Institute so right nearby is here. She was appointed that director in October 2021 and in that position she oversees a division that covers a wide range of scientific domains and disciplines including epidemiology behavioral science surveillance and statistics cancer survivorship and health services and outcomes research. So we are honored to be here and have them today I hope you will ask some a lot of good questions, and I will turn it over to Jason thank you. Great well Chris and everybody thank you for that introduction thank you for this opportunity. It is, it is truly a privilege to be able to share with any she or I and larger community. I've been able to do with this opportunity that was given to me by NHGRI so this really, really was kind of a really was a career defining award for me in many ways and so I'll be able to tell you a little bit about about that experience and some of the findings we've already learned from this process. So let me get my slides up. So, as Chris mentioned I'm a primary care provider, but I am interested in the implementation or maybe the mainstreaming of so called genomic medicine so the use of someone's genotype to help inform their medical care, often thought of as a way we don't do it in in in current practice. And so, over my career, I've looked at various flavors of genomic medicine pharmacogenetic testing return of monogenic results. But you'll see that there's a there's been a little bit of a kind of return to my roots in polygenic risk scores that I'll tell you a little bit about and what we're doing currently in this space. And I should also say as a VA employee. My, my views I express here are not representative of those with the government or the VA specifically. But I do want to give what I what I think is the scientific literate literate audience here, a very brief overview of what we mean by polygenic risk scores. Then I'll take a step back in and talk about defining clinical utility. And I think this is where Katrina and I will have some really good conversations about what exactly that means she's done a lot of scholarship in this area. I have some thoughts she has she certainly has some thoughts. That'll be a good, good conversation. And then I'll illustrate some work I've been doing in clinical trials, including, including the Genova study which is funded by by this award from an 2005. So, you know, as was mentioned, I'm a primary care doctor. And so what's a primary care doctor doing in this space. So, you know, but I think really my clinical practice informs my research interest and then vice versa. So you sit down with patients and you start to then accumulate clinical experience and you've seen your 100th patient your 1000th patient your 10,000th patient. Certainly the individual characteristics of that patient in front of you matter but then you start to think about patterns. The patterns of the patterns you taught in medical school the the what does smoking lead to a higher risk of, and there are some then there are some patients that really don't seem to fit the patterns that you're taught or that you started to accumulate from just clinical experience. And that's when you really start scratching your head what is it that we as scientists as clinicians don't yet understand about risk, and what predisposes somebody to risk of disease. You know, not only your diet and exercise as I've illustrated on this slide a part of that but you also then there's that nagging suspicion, I wonder if this is so called genetic, or if it's hereditary or it runs in the family. And I think really that as I was finishing my internal medicine training, and then moved into a general internal medicine research fellowship, it was that interest in epidemiology and increasingly was becoming an interest in genetic epidemiology that kind of led me down the path that I'm on now. And interestingly, one of the first fellowship projects I took on in 2011 was a return of, it's almost almost not fair to say polygenic risk scores, because we this was a score for predisposition to type two diabetes that consisted of 36 SNPs that were known at the time. You can see this was an early report that we gave back in a clinical trial at Mass General to individuals even you snip by snip we told them what their results were, and whether that was associated with an increase or lower risk of type two diabetes which you know is a common complex trait. The hypothesis of this randomized trial the GCLC trial that we did was to see, alright, could we use that information to motivate individuals who we already knew were at risk for type two diabetes because they had metabolic syndrome or overweight could somehow this genetic information be a motivating factor, make them attend more weight loss classes and a curriculum they were a part of, or lose more weight or adopt healthier behaviors. All there were some signals towards changes in attitude but it turned out to be an all study in terms of meaningful weight loss or, or other stages of behavior change. But it did start to wet my appetite about what would it take to start to move information like this into primary care where we could use it as a clinically useful tool, and what how would we even define what clinically useful means. This is my one slide to orient you to what a polygenic risk score is, I will not be able to do this full topic justice but suffice it to say that across an individual's genome. If you look at the millions or even you know millions of different markers across the genome, you can do millions of different case control studies for their, their likelihood of having a certain disease or not. So you then take the effect sizes of each one of those low side across the genome and you can add it up as a weighted score, assign each individual a score and that would place an individual somewhere on a bell curve of genetic predisposition to a disease and these are used often used now for for complex common traits, continuous great continuous traits such as LDL cholesterol, but also dichotomous traits diagnosis so cancer prostate cancer yes or no type two diabetes yes or no. And these can be calculated from genotype array data so called snippet chips, or increasingly genome wide genome sequence data to. So you could present the, the, what we'll call the clinical validity, or the association of these of these PRS with disease in a few different ways. So if you look, this is from a kind of seminal paper from Karen chaff chaff in from a few years ago. If you look at the panel on the left, if you imagine bell curve of these polygenic risk scores, you could define what you think are clinically meaningful cut points and then you could identify the percentage of the population that's above that cut point for a certain kind of risk in this in this case it was for coronary disease, those different cut points corresponding to three fold four fold five full risk of the disease. You could portray it as cases and controls and how the polygenic risk scores among cases of CAD and controls of CAD are portioned across the population you can see here that this is not a diagnostic test there are lots of cases that have PRS lower than controls and vice versa. So this is clearly not the disc the ability to discriminate between ultimate cases and controls is not perfect. You can see you could represent the data as individuals with a certain percentile in the polygenic risk score distribution and what absolute prevalence of coronary disease is that associated with in this case so you can see that the prevalence shown in this graph ranges from zero to single still single digits of prevalence of coronary disease. Thanks for audience participation but I will ask you sitting there in your chair to play the thought experiment along with me. How would you define or fill in the blanks to this question a polygenic risk score has clinical utility when. So a lot of people have thought about how to answer this question. What I read of the field is that consensus is emerging although it's not there's not a single right answer, but I think we are kind of agreeing on some of the concepts which I hope will illustrate some during this talk. I found the work, even though it's more than 15 years old now I have found the work of the e gap group to still really be helpful in thinking about putting clinical utility in its bigger context of kind of a hierarchy of evidence for genetic tests. So it's often called the ACCE model starts with the bare basics of first a test must have analytic validity. So if you think your test can tell whether a certain SNP is an a or a T can actually reliably tell an a when there's an a. If you can't you're kind of done, you know you need to find it for different tests. So assuming your lab test has analytic validity. What's the clinical validity. How do we know that if it's an a there that a is actually really associated with disease. The clinical utility and both of those are really required before you can begin to talk about the I'm sorry that those are that's those you have to be able to have one of both of those before you can go on to talk about the clinical utility of that information. The e gap defined clinical utility is the ability of test results to change patient management decisions and improve net outcomes and we can talk about how that might apply in PRS. So before we do let me illustrate those concepts with pharmacogenetics. So here was a in the e gap paper a way of illustrating these concepts so let's say you wanted to use someone's genotype at a certain side of Chrome p 450 locus to influence your genotype prescription of an antidepressant to treat their depression so the analytic validity is can you can your test actually measure the genotype you think you're measuring. Like I said if it can't, you have to start over you don't have a valid test an analytically valid test. So assuming it does, how do you know then that that particular set 450 genotype that you've measured is actually associated with the phenotype you think it is and that could be more molecular type phenotypes like the metabolizer status of a certain medication, or the kind of more downstream phenotypes such as whether a drug has efficacy or safety for the individual. And then once you determine the clinical validity, then clinical utility is alright using that information in patient care does it actually improve outcomes. Can it change treatment decisions and just the changing of that treatment decisions improve rates of remission of depression, improve safety profile medications. So when I think about that framework and applying it to PRS I still think it's very helpful although there's some unique things about PRS that maybe don't exactly fit to the e gap model. So the idea of analytic validity is a little unique you're not looking for Justin a or just a T at a certain location PRS is it kind of is a different construct in its measurement so that has implications for what kind of patient platform are you using what what is the call rate of the assay that that is not necessarily there for for other, you know, single snip tests you might perform or other other tests and laboratory medicine. And then I won't have time to do this topic justice but the clinical validity of PRS that does that association between the score and the disease or the trait of interest. And the frequency does vary between populations and that's kind of a critical area of health equity concern for the clinical application of PRS. Although I am I am heartened to see that there's there's a lot of improvement in this area as more diverse cohorts and methods are being recruited. You know, on the subject of different populations. Here's an example of how say a polygenic risk score is associated with in this case again, coronary artery disease in three different groups that were identified by racial and ethnic ethnic status so European Americans African Americans and Hispanic Americans in this example. So you can see that the absolute risk associated with a given quantile of polygenic risk score is slightly different between these populations, although overall the pattern holds so being in the lowest turtle is associated with lower absolute risk of coronary disease compared with the top turtle in these three different populations. So an example of the clinical validity. This associations are real. There is there is some lower accuracy in different populations, but these patterns that we see the association between PRS and the risk of a disease seem to be pretty robust and seem to hold up across populations. So then we're left with all right well is this going to make any difference in the way we take care of patients great the the associations are real, but can we demonstrate that using these tests can change patient management improve outcomes. So maybe when you were thinking about how you would answer that question PRS is clinically useful when maybe you thought about well it can tell who's low risk and high risk. Right, but then so what so we can in clinical practice, I might be able to identify some of my patients that I think are at low risk or high risk for a certain disease but what am I going to do with that information the the clinical management does not stop there can you do something about that risk. So potentially things that have been hypothesized or that for those at high risk you could implement more aggressive disease screening earlier a different modality that's more intensive made more frequently. So earlier, more aggressive prevention either through preventive medications or through more aggressive lifestyle modification, motivate patient behavior change we know that's hard, but that's often proposed as a hypothesis for help PRS might be used. So don't talk a lot about how being at low risk might help de implement potentially unnecessary preventive preventive medicine strategies such as less aggressive screening for low risk individuals, or less aggressive disease, other preventive measures. But that has also been proposed as one potential element of clinical utility actually de implementing unnecessary interventions for which the benefits don't clearly outweigh the risks. So when I'm thinking when I think about action ability, you know, a polygenic risk or as a continuum I showed you that bell curve but in a simplified way of thinking about it ultimately you have to view the clinician or you the clinician and the patient together have to make a decision are we doing something or are we getting a colonoscopy yes or no. Are we starting a baby aspirin yes or no. Now of course medical care is not that kind of single moment point in time it's okay am I getting a colonoscopy and am I getting it every five years or every 10 years you know like a lot of medical management is ongoing. But I do think it's helpful. When you think about the clinical utility is what would that action threshold be recognizing it might be different for different patients, certainly different for different diseases. But I think this this rubric that colleagues and I wrote about a couple of years ago is a helpful way to think about how you take the continuous measure of a PRS to the somewhat dichotomous action that needs to result from it. Before I move on I do want to circle back to that idea of de implementing unnecessary tests. We supported by this this award we actually did a national survey of primary care doctors in the across the US about polygenic risk scores and their perceived utility of it. And you can see for each of the possible uses of how they might use a polygenic risk score preventive medications screening lifestyle modifications you saw a bias towards doing more for high risk individuals and not doing less for the low risk individuals kind of a commission bias of wanting to do more but still not quite sure that they would want to use polygenic risk scores to stop screening individuals, which I think is not surprising. We've talked a little bit about action ability, you know that last part of the gap definition of clinical utility was an ability to test to improve health outcomes. So, I often think about that as a scale a trade off of risks and benefits so potential risks being using a polygenic risk score could lead to a cascade of harmful medical interventions that are unnecessary that don't result in that benefit. On the other hand the benefit would be the earlier detection and prevention of disease. Let me let me pivot now, as I lead into the Genova study which is the randomized trial we're doing now is to say alright. We've started to define a little bit about what clinical utility might mean do we need a randomized trial to demonstrate clinical utility. So this may be something that we discuss in the in the Q&A afterwards but you probably know that randomized trials are the gold standard for determining the effectiveness of interventions, and yet most of what we do in clinical medicine is not based on RCT level evidence. So I give the example of routine lab tests we don't have RCTs that say we should periodically check a patient's kidney function or complete blood counts. So I've often thought it's helpful to say well are we talking about PRS. Are we talking about it as a lab test by itself or is it really a part of an intervention that might be used a part of a preventive medicine intervention. And in the Genova study we see it as the latter. So Genova is stands for the genomic medicine at VA studies so that's where I practice in the VA system. So we wanted to implement polygenic risk force for six different diseases and put them into a primary care setting to see if we can improve patient outcomes by doing that. So the six diseases are those that are common in adult medicine and for which primary care doctors already have an approach to screening. So atrial fibrillation, coronary disease, type 2 diabetes, breast cancer for women, prostate cancer for men or colorectal cancer. We are enrolling just over a thousand of these individuals. We did want to, even though we're using a SNP chip, we did want to identify any valid actionable monogenic variants on the chip and report those out to participants and then make sure those individuals get connected to care. But otherwise for those individuals that do not have a monogenic finding, we generate polygenic risk scores for five diseases for each individual. And all of those patients that have at least one polygenic risk score consistent with a two-fold increase risk of the disease, as in the published literature, we call the high risk group and anyone, everyone else who does not have any high risk result is in the not high risk group. And that's about one-third, two-thirds. And within those two straight O, we randomize. So those individuals get their results either at baseline or they get it after the two-year follow-up period. And we follow all the individuals then for 24 months for new, for new diagnoses of disease, change in management, and some other patient-centered outcomes that I'll show you on a subsequent slide. So we described last year in Nature Medicine our approach to developing a clinical assay. So a lot has been written about what we call that phase one of PRS. So that's the first part. You know, a lot of the epidemiology, the statistical genetics, the innovation in methods. But it doesn't end there, right? You need to still develop a lab test. So we've described how we came up with a clinically valid lab test and what we used to make sure that was a clear, you know, up to clinical standards to be used for clinical decision-making. We also have developed a report. I'll show you on another slide that we think is very high level and yet transparent about the limitations of the results. And then number three is really what Genova is all about. What is this going to do in the patient care setting and what are patients and doctors going to do with the information? So as I mentioned, we do see polygenic risk scores as a part of an intervention, not a standalone lab test that you would order. So in addition to the report, and I'm showing you here, this individual has an increased risk of prostate cancer and subsequent pages of the report to go into that more. But we also, anyone with a high risk results discusses those results with a physician or a genetic counselor. The reports are sent to the primary care provider are put into the medical record. We have patient and PCP level information sheets with the recommendations based on these results. So here's our patient support of information for colorectal cancer on the left and for providers on the right. You know, we try to stay within what we think the evidence suggests. We are forthcoming and say, you know, guidelines currently don't speak to what you should do differently if anything about an individual with a high PRS for colorectal cancer. But as a reminder, here are what the current guidelines do say about an approach to colorectal cancer. And you might think about how to put that into your screening. Any of you, if you've practiced in the VA system, you'll recognize our EHR. So we do have these results as structured data in the EHR in addition to the PDF results saved to. And this is our conceptual model that when we think about how to measure the outcomes of doing this. So our primary, our primary outcome is the time to diagnosis of new disease. So we actually think in this population, using PRS and the clinical setting will actually accelerate the time to new diagnosis among the high risk individuals. So we do think that patients and providers will use this information, perhaps order appropriate diagnostic tests and diagnose. Undiagnosed previously prevalent disease such as a case of type two diabetes that that had not had never been checked and a one C had never been ordered. Or newly incident cases of the disease in the two year follow up period. We're also measuring process outcomes such as the change in management different tests and procedures the provider orders. We're also measuring outcomes including activation, medication adherence quality of life and we're, we're, we're checking in with PCPs on their perceived utility, in addition to health care costs. So with that, I want to just give you kind of a taste of where this line of research has gone or is going. I was very much informed by the work that I did in Genova. I started to think about prostate cancer as being really an exemplar condition where we don't have great approaches to screening currently, where the polygenic risk scores are starting to be pretty advanced and can actually discriminate a significant amount of disease among individuals and at least in the practice in the population that I take care of the veteran population is a huge is a is a is a significant problem. So this, the progress study the prostate cancer genetic risk and equitable screening study we hope to start enrolling next year, funded by a grant from the VA, which will enroll 5000 men nationally to a to a randomized trial of precision screening for prostate cancer versus usual care. And so I look forward to sharing those that that experience of the next few years with you. And it certainly has been an outgrowth of the work that was was supported by this, this innovator project. So with that I want to thank the Genova team. This really has been a team effort of course the patients and participants of the Genova study and any share I for funding this work and I'm looking forward to the conversation. Great. Thank you, Jason. So that was a really fantastic presentation and kind of description of what our polygenic risk scores and what does clinical utility mean and my role here today is really to try to put this into a larger context and I really appreciate the invitation from NHGRI to be here today and to present along with Jason and talk about some of the work that I've been doing over several years now and I really want to focus in on some of the thinking that we've been doing as part of the ClinGen program and this is another program that is funded by NHGRI. And we've really had the opportunity to think very carefully about action ability and what does that mean and how would we define it and assess it. Over the past decade we've really been thinking about this in the context of monogenic conditions. And more recently we've started thinking about what would this framework look like in the context of polygenic risk scores. And as Jason already mentioned, we're really building upon work that was funded through the Centers for Disease Control, the ACE framework or ACCE framework as well as the work that was done in the EGAP program to really think about carefully how do we evaluate genomic applications. And so to just put this into a little bit broader context of several of the different consortia that are funded by NHGRI, as Jason mentioned there are these different components, the statistical validity or that's part of the analytical validity of the polygenic risk score rather than the clinical validity is the polygenic risk score associated with the outcome that you're looking at, the clinical utility piece that we really focus on in our action ability working group. And then I want to touch a little bit upon implementation which Jason also already mentioned in some of the trials that they've been doing. And I really want to make a distinction here between evidence generation and evidence curation. So evidence generation is really the individuals and the research teams that are conducting the research that are producing the primary evidence and publishing that in the literature and conducting those clinical trials or observational studies that we use to assess the evidence for each of these pieces. And a couple of the consortia funded by NHGRI that are involved in evidence generation include the primed consortium and the eMERGE consortium in particular in the area of polygenic risk scores. The ClinGen consortium is really focused on evidence curation or evidence synthesis so taking the information that has been generated through other studies and synthesizing that putting that information together into a picture that we can see across all the studies that have been done to look at both validity and action ability. So in ClinGen we have two different work groups that focus on these two different pieces because they are both they are each quite large areas to focus on. And then I also want to mention the role of professional societies which are critically important because not only do they conduct their own evidence synthesis activities but they also make practice guidelines and recommendations based upon the evidence synthesis that has been conducted either through their own efforts or through evidence synthesis that is done elsewhere. So those are really critical components of figuring out how we use this information to care for patients. So the types of actions that we think about in the context of monogenic conditions were really primarily focused on the individual person. Things like whether a treatment would change whether we might change a method of screening or surveillance or maybe the screening interval might change. Circumstances to avoid might be something like avoiding certain dietary factors or other kinds of lifestyle factors that may put you at increased risk if you also have a genetic risk for that condition. And then referral to specialists would be where you see evidence that there is better health outcomes when patients with these conditions are actually treated within a specialist group that focuses on those conditions. However, in the context of polygenic risk scores we not only focus on those kinds of actions that are oriented towards the person, but we also think about actions that are related at a societal or a systems level. And so this might be including things like how you manage the health of an entire population as well as how you manage health care costs and de-implementation might be an example that's related to health care costs. So these are some additional considerations that we think about with polygenic risk scores. There are five domains that we really think about in the context of polygenic risk scores and these are adapted from the framework that we developed for monogenic conditions, but there are some nuances and differences. The first area or domain is the severity of the outcome. And this is really assessing things like the clinical features associated with the condition and natural history, what is the typical agent onset, how quickly does the disease typically progress, things like that. But in the context of polygenic risk scores we also think it's important to have that population and societal perspective and really thinking about what is the prevalence and the incidence of this condition in the whole population. In terms of the outcome likelihood, ideally we would have information on absolute risk. I'll show you in a moment that a lot of times we don't have that ideal information and then we need to rely upon measures of relative risk. As Jason really illustrated in his slides, identifying or defining thresholds for action to take a quantitative polygenic risk score and convert it into something where you would take the action among people who are above the threshold and not take action for those who are below the threshold, or you could take the polygenic risk score as a quantitative variable and think about the entire spectrum of risk. And then we also really want to think about risk at both ends of the distribution. As Jason also mentioned, those who are at lower risk, you may be able to de-implement some healthcare and because those individuals are known to be at lower risk. The intervention effectiveness is really very similar whether you're talking about a monogenic condition or polygenic risk score and really we're looking at whether there are professional societies or other groups who have recommended interventions based on evidence that they are effective actions to reduce morbidity and mortality. And in our context we are really focused on healthcare settings. So this is not really including things like personal utility as part of this definition but really focusing on interventions that would happen within a healthcare setting. The nature of intervention is looking at what are the potential harms or side effects or complications that could happen from using the intervention. And colonoscopy is a good example where, you know, this is a really important screening tool that we have but we do see a somewhat lower adherence than we would like to see in the population in terms of acceptance of this procedure. That's one example. And then level of evidence for polygenic risk scores where we are in the state of the science today is we really think that the level of evidence is likely to be fairly low. The emerging nature of polygenic risk scores and their use in clinical practice. However, we think defining a framework like this upfront is really helpful in terms of helping people think through and plan the kinds of studies that we would like to see in order to develop the evidence base to ultimately move this into clinical practice. So as we were thinking through what are the domains that we really want to care about. There were a few domains that we are interested in that we were concerned that we just wouldn't have sufficient data available to us to assess these domains. So we include them in our framework but not necessarily part of our assessment but more to highlight the gaps and future needs for the research direction. So one is looking at the chance to escape clinical detection and whether knowing the information about the polygenic risk score is really going to impact the time to intervention and really be able to make a difference for those patients. We also are very interested in evaluation of utility across ancestral populations. Right now I'll show you in a moment that just our evidence base is not quite ready to do this evaluation across all ancestral populations but this is somewhere we need to get to in the future. And so we want to capture this information in order to highlight the gaps that exist currently. And finally looking at cost effectiveness of the intervention. We are concerned that the field is just not quite ready yet to conduct these studies but we do think it is an important aspect to consider. So our goal is to develop three different kinds of assessments. One is an action ability summary report which is really a qualitative synthesis of the evidence and just lays out the information and makes that available to people. Then action ability scores are a semi quantitative metric that will score each of those five domains that I mentioned previously. And then the action ability assertions are based on those consensus scores using the semi quantitative metric. But really using categories that are descriptive of definitive action ability, strong action ability, moderate action ability and limited action ability to kind of help with the clinical interpretation. So I want to go through an example looking at breast cancer in terms of where we are at with curating the evidence. This is currently a work in progress so still more to be done but I think this can really highlight some of the challenges with synthesizing this evidence. And so this first slide is looking at what do we mean by a polygenic risk score for breast cancer? And in fact there are many different polygenic risk scores that have been published in the literature and that have been assessed in different studies and data sets. And overall in our search we found 112 different polygenic risk scores that have undergone a total of 366 different performance assessments and different study data sets. So that is a lot of variability. So we're trying to narrow this down and think about in what ways are these the same and in what ways are they different and how would we collapse the evidence across all of these different polygenic risk scores. One of the things we did was to exclude some of the polygenic risk scores that are focused on very specific questions like whether it's in males for instance or whether it's including only a particular subtype of breast cancer like ER positive only or whether you're predicting risk for afterness breast cancer or contralateral breast cancer. So we excluded those and that ended up with 95 polygenic risk scores with a total of 183 performance assessments. And so we were still thinking about how do we sort through all of these. And we decided initially that we would focus on those polygenic risk scores that exclusively used genetic or genomic information and did not include other risk factors like age at menarchy family history or BMI not to say that those other risk factors and risk models are not important. And we absolutely think they are important. And we absolutely intend to go back and look at those. But we thought it would be an easier starting point to just focus on the ones that included the genomic information only and so that left us with 91 polygenic risk scores with 140 performance assessment so that's still quite a lot of information. And this is just looking at some of those 140 risk assessments and here you can see that most of the assessments are using a relative risk as the measure of association versus an absolute risk measure. And you can also see in the table on the right that we're looking at what population was the polygenic risk score developed in what population was it validated and and what population has it been evaluated in. And you can see that there has been much more work that has been done in European populations than in populations that include other ancestry groups. And then in terms of looking at the evidence for actions that could be taken one of the primary ways that we do that is to look for practice guidelines. And so first we wanted to see are there any guidelines out there that are based on polygenic risk scores and the two guidelines that we found that actually indicate that we're we're not ready to make recommendations practice guidelines for polygenic risk scores so should not be used for clinical management at this time and uses recommended in the context of a clinical trial as an example. So the next thing we wanted to look at is whether there are any guidelines that are based on a threshold of risk, because if your polygenic risk score can predict an absolute risk for an individual that exceeds a threshold. Then these guidelines would apply to that polygenic risk score potentially. And so here's where we are able to find some guidelines that have differences in how to manage the patients based on the absolute thresholds of risk for those patients. As I showed you in the slide earlier, a lot of our polygenic risk scores are showing us relative risks rather than absolute risks. And so this is still a challenge for us to know whether the polygenic risk scores are actually exceeding these threats, these thresholds. I want to end with just a few additional implementation considerations that even once we have determined that it is analytically valid, it's clinically valid, and there is actionability, there are still some more challenges that we need to think about how to implement polygenic risk scores. And the first one is that the vast majority of our data that we have so far in order to develop these polygenic risk scores have been in individuals with European ancestry, which in the left hand graph is represented by the area in the red color. And you can see in the right hand graph that the prediction accuracy of the polygenic, the PRS compared to a European population is actually lower for populations aside from the European population to change this and give us more information on global populations. As I mentioned before, we kind of left out many of the polygenic risk scores that include clinical information, but I think one of the main questions that people have is whether the genomic information or the polygenic risk score is adding to the risk that we can predict based on clinical features for those patients. And this is just an example for breast cancer again, and the kind of pinkish color bar is using the clinical features alone. And then the blue color is the additional improvement in risk prediction that you get by adding the polygenic risk score to the clinical information. And so 0.5 is like a flip of a coin, you're doing no better than that, and then 1.0 would mean you have perfect prediction. So you can see there is some improvement on that scale with adding the genomic information to the clinical information, but it's not a huge difference. For the ones with the hash colored bars, those are predicting prognosis. So once you've already been diagnosed with the condition, what is your chance of survival and your prognostic outcome look like. And so you can see that there's a little bit stronger difference between just the clinical characteristics and adding the genomic information for those scenarios. As Jason already mentioned, de-implementation at lower risk is really something that is we should be thinking about, but that there are a lot of concerns. And this is a framework that when Norton and colleagues developed about the de-implementation process, and this was a generic framework, not just specifically in the context of polygenic risk scores. But you can see that there are several different kinds of issues that need to be addressed, including the magnitude of the problem, whether it's actually causing harm. Is it reducing equity in the population in terms of the various different kinds of actions that could be taken in terms of de-implementation? Are you just reducing exposure to that intervention? So for instance, by expanding the screening interval, or are you replacing it? Use drug A instead of drug B? Or are you completely removing or restricting that action for those who are at low risk so they no longer need this screening at all, for instance? There are a lot of barriers and facilitators at multiple levels. From the patient perspective, when we look at cancer screenings, for instance, where we've tried to increase the screening interval, one of the things that we see from the patient perspective is a concern about trust. Are they really, is it really safe for them? They've been getting the screening on an annual basis for a long time. Is it really safe to make that interval longer? From the provider perspective, they have concerns about legal risks, about communicating with the patients that it is safe, and making that clear with patients. And they may even still be getting reimbursed, whether they do the screening or the intervention on an annual basis at or at a lower interval. And so we need strategies at all of these levels to really address these barriers. And finally, we have a lot of workflow and delivery system integration issues that we will need to think about. These are, to some extent, listed within the ACE framework, and particularly within the context of clinical utility. But in the way that we've defined and thought about actionability, we really aren't thinking about the implementation pieces yet. And I think this is something that we will really need to pay attention to and address in terms of actually ensuring that these polygenic risk scores can be used in clinical practice. And it will include all of the people who are involved from the patient, the care team, to the regulator, all the processes, the tools that are needed to implement those interventions, as well as the information that's needed. So if we need to have some assessment of clinical features that goes along with the genomic information, how do we get that information? How is it transformed into an interpretable result? Jason showed you some nice reports that they've been working on. What is the output and how do people make decisions based on that output? So I'd really like to thank the team that has been thinking about this with me and all of these thoughts really reflect the thoughts of the whole group, and I'll end there and hand things over to Rob. Thanks. Great. Thanks, Katrina, and thank you, Jason. Wonderful talks on a great topic. So, you know, why we wait for some questions to come in. I have a few questions about polygenic risk, especially since we do think the biggest impact they're going to have is in primary care. So Jason really appreciate the work you're doing there. You know, looking at polygenic risk score, I mean, we've learned a lot from monogenic. And as you know, it's been slow to uptake monogenic testing even in primary care settings. And as you look at polygenic risk score, I mean, it seems to me so almost exponentially more complicated. And there's so many more variables. What kind of, from our experience with monogenics, how do you think we should start thinking about rolling out polygenic risk scores into clinical practice? Yeah, I mean, you're right. The easier question is how are they different? So it's the harder question about what can we learn from the monogenic diseases. I actually do think there's a pivotal moment in the field when we started to realize that PRS at a certain upper tail can actually start to approximate the kind of effect sizes we were seeing with monogenic disease. So the, you know, the, the increased risk of coronary disease from familial hypercholesterolemia has a polygenic equivalent. And I think that was a helpful anchoring. Because if nothing else, you could actually identify that upper tail of people that almost had a monogenic like risk from their PRS. So in that sense, some of the learning from monogenics could apply, but as you suggested, like there's so many other differences. Maybe some other takeaways from the monogenic world are, you know, an understanding that there are not enough genetic professionals to go around, not for, we already know that with monogenic disease. And now we're at a place where everybody could have a PRS result. It might be medium, low or high, but everyone has a result. Everyone's somewhere on the bell curve. So I think we have to learn about how we have tried to mainstream some of the more common genomic medicine scenarios in the monogenic space and take some of that knowledge about what kind of implementation strategies work or do not work to help mainstream and help kind of upscale the providers to handle some of that information. And that's a whole topic of itself, some of those strategies, but it's a challenge. Yeah, definitely, definitely. And then Katrina, kind of looking at some of the stuff, I mean, you talk about the curation of monogenics and just thinking, I mean, people from all over the world, you know, are contributing to that. You know, given the same complexity of polygenic resource from a kind of a curation standpoint. I mean, where do we get the time and resources to think about doing some of these curation for these more complicated things thinking that you have 99 different tests versus just this one. Yeah, that's a great question, Rob. And I think that we have an opportunity here. But we maybe didn't have with monogenic conditions because this is relatively new and worse, it's still emerging, and that we are trying to think about reporting standards and databases where people can contribute information that I think will make it a little bit easier for us to find the information that we need. We are able to work together as a research community to ensure that our publications include those reporting standards and our data makes it into the common databases so that we can find the information a little bit easier. And a lot of what is happening in the monogenic area is really needing to do a lot of literature search that is very manual and time consuming, because we didn't have these standards in place up front. Yeah, great point. Glad that you're doing that work. You know, as we all know in clinical practice that, you know, things make it to the clinic before the evidence is necessarily there and so patients showing up with polygenic risk for. I guess if there's, you know, clinicians on the call, I mean, where would they start to lean to kind of know what to do with that when a patient does bring these results to them. There are resources out there that they can lean on or individuals to recommend. So I can go I mean some of it really is could be the Wild West. So it's hard to give a single answer the first, the first step I would recommend is start with the test report itself. So what is this lab. Who are they where did this come from. Is it a clinical lab. Is it that the, is it that the patient access some they had their, maybe direct to consumer SNP chip data and upload it to a third party to generate these PRS you really have to look at that document to know where to start. So guys, there are some good publicly available resources, you know the CDC has some good information about genomics in general and polygenic risk score specifically NHGRI does too. But you know, like with many other things that say the PCP encounters, they, they started to develop kind of this trusted network of other colleagues they can curbside I'm happy to be that for any of you on the call, or on the meeting. So I would really start with the test report itself and ideally, if it's a reputable lab, they, the, the lab itself would have a 1-800 number and email address for more information for that specific test. And I'll just add to that to consider the professional societies in terms of what the practice guidelines are. And you have that as a resource, and I really think what Jason pointed to in terms of thinking of a systems level solution, rather than individual providers needing to each solve this on their own. So hopefully within the healthcare system that providers are working, there would be somewhere to go to get those questions answered. Yeah, definitely more resources there. So one of the question coming asking about, especially increasing diversity within the PRS and the question is, you know, is there any ongoing work to take into account non-European groups in order to make PRS more applicable to the whole population? So the primed consortium that NHGRI has funded, I think is really focused on that work as well as the eMERGE consortium. These are two large consortia which are not only including more diverse populations as part of the study populations that they're looking at, but they're also really focused on the methods. So how are we going to develop polygenic risk scores in a way that is applicable across all the different ancestral groups? So I think those are two very different but important questions, not only having the study populations in place, but also having the methods. Yeah, great questions and answers. Thanks. I mean, one of the other things that comes to mind, I mean, as you think through, this is more kind of a research question, there's always that difficulty in terms of knowing when to intervene and how to intervene. As we kind of think forward with randomized clinical trials, how should we go about getting that evidence-based, not necessarily to prove the utility, but just to understand, you know, do we implement these in children? Do we start at 20 years old? Where do you see us getting that type of information to drive those clinical trials? I can start and I'm interested in what Katrina says. It really depends on the disease for a lot of these conditions. You know, if it is a dementia, powering a trial that starts in children is going to be a big ask for a sponsor. So it really, you know, so the typical considerations of a clinical trial, effect size that you'd expect, what kind of observation period you need would come into play when talking about a trial. You know, already the wisdom trial is looking at this question in breast cancer, a more precision approach, and includes a polygenic risk score and other factors to breast cancer screening. And then, as I mentioned, our prostate cancer trial that we're launching next year also has looked at we, you know, doing it at the right age range that seems to be the clinically applicable use. The earlier in the life course questions are really important and a lot, I think a lot more difficult to answer with a trial. Yeah. And can I add to that, Rob? Oh, definitely. Not to make things more complicated, but I think I would also recommend to really think about the implementation questions early on. And think about designing hybrid studies that look at both effectiveness and implementation as a way to try to accelerate that research agenda and ensure that the work that we're doing can actually be disseminated and used in typical clinical practices outside of a clinical trial. Yeah. Thank you. Yeah. Kind of going on to, we talked a lot about the science and just wondering, I mean, what kind of ethical legal social type issues should we be thinking about or are unique to PRS that we really haven't faced before in clinical practice. That's a bold statement to say what's unique. I mean, so Katrina has already mentioned about the reduced accuracy in traditionally underrepresented populations, so I think that probably is the most pressing ethical concern. There are parts of biomedicine, huge parts of biomedicine that are based on unrepresentative data, so in that sense PRS are not unique, but there are really some unique statistical genetics considerations when it comes to PRS that fortunately groups like the Prime Consortium are working on. So I really think when it comes to kind of the ethical issues that really is top of mind. Maybe a second one is, I'm going to propose it and then say maybe it's overstated this idea that the use of polygenic risk scores will detract from other environmental lifestyle social determinants of health that we know are probably more important for many of the diseases we take care of. So that is a concern. I take it back though because at the end of the day I actually think patients and primary care providers have more common sense I think then to place all their eggs in the PRS or genetic basket they're very aware of those other risk factors so that it is worth at least naming as a potential concern and overreliance on genetics. It all talks about contextualizing these I mean your your first slide that you show I mean in 12 minutes you're supposed to go through all those and understand them and put them into context which creates a challenge was it's definitely there. Just for both of you I mean, as if there's like primary care physicians listening in. What would you like them to take away from this talk. There's one thing. I would, I would like them to take away from this talk that polygenic risk scores are probably coming their way and to get ready but also that they are not alone in this and that I think we can absolutely develop systems for delivering polygenic risk scores as part of clinical care that supports primary care clinicians and we really need PCP to be part of the study teams. So that we can design these in a way that will work for them. And I'll echo that I would tell the PCP these associations are real PRS are a real thing. The associations are the magnitude of how much it will impact your patients care and outcomes I think remains to be seen and that's an active area of research. Absolutely, like Katrina said, your, your healthcare system is doing you a disservice. If it has its, you know, by the time PRS emerge and become more prevalent, if systems aren't being put in place to support your use of that because it's not you couldn't be expected to go it alone on and take that technology. Yeah, really does create a significant challenge for primary care with the significant part of people not in a health system, and where they get those resources, so those are some great points. We did have another question that came in I think I think it's going to this one. One of the things is, as you know, as we develop stuff in research in that translation into clinical practice and this one kind of is in the lines with that is, do you envision ongoing evaluation of polygenic risk scores as they're reported. And so the example would be what will be a possible to evaluate new PRS in the existing Genova study. I guess that's a common question that we hear too is, as we improve these scores, at what time do you make a decision to exchange or add in addition to that study. Yeah, so maybe maybe there's a couple questions there one, do we, how often do we update our pipeline for new, you know, new prospectively tested patients. And I think maybe that's the question and not do we go back and reinterpret and, and, and re contact the patients and tell them they have an update report that that has a lot of implementation challenges as we even already know kind of from the monogenic space. We do, we do in terms of kind of prospectively our pipeline we do. We do have a little bit of a share of new GWAS and new PRS methods that are coming out near nearly constantly. So it's a little bit of a, if it's a little bit of a resource utilization to benefit ratio. So we have updated our materials we've updated our pipeline as it has gone. As you can imagine with a clinical test. That's, that can't be as reactive to every single publication that comes out, you know, we can't change it that quickly. Certainly clinical labs as they start to develop these assays and reports will need to have that kind of plan in place what will be their plan their policy for how often they'll update those. Now I'll just add that this is not a unique problem to genomic medicine that digital health, for instance, is another area where we see the technology evolving more quickly than we can conduct the research. And so by the time your research study is over, the technology has already advanced to a next generation and so I think we need to really think about how to design our research studies to be nimble to allow for this change over time that is happening. Definitely a challenge, especially with the pace of all these changes that happen so quick. And you know it also also makes you wonder in terms of hospital systems are there going to be some day people just sit watching these different, no version control for no better term, making sure that the latest version is as part of that health system and accurate. I don't have any other questions but I would like to just take this moment to thank both of you for taking the time out of your busy schedules to talk I know it was, I definitely enjoyed it. And we hope to see you in the future and we hope to let you continue your work and polygenic risk score. Well thank you this thank you for the invitation it was a pleasure to be able to talk about this Katrina thank you for the conversation. And thanks to NHGRI for putting this on and thanks to everyone for being here and and adding your questions and being part of the conversation. Thank you. Thank you both.