 If you could get your seat, we've had a really wonderful session already today, a series of three sessions. The good news is this is the final session and we're going to do our best to get you out of here. There are going to be three presentations and we're followed by a 30 minute panel discussion and Rex and I are going to do our best and we promise to get you out on time. The first presentation will be given by Vanessa Hiratsuka. She's from the Center for Human Development at the University of Alaska Anchorage. She has extensive experience in the ethical, social and legal implications of genomics, research and precision medicine, especially in indigenous populations who were really delighted when she agreed to give this first talk which is on American Indian, Alaska Native Community Engagement Preferences and Tribal Code Requirements. Vanessa, over to you. Thank you so much. Thank you for joining us here in the room in Bethesda as well as to those of you that are online in Zoomland and on what was it, Genome TV I think is what I heard. Sounds so cute that name. I'd like to say thank you so much for having me here and allowing me to amplify the voices of the Alaska Native American Indian peoples that I've been working with and what they've been saying about genomic medicine, about precision medicine, about genetic testing. I come to you from Anchorage, Alaska where I am now with the University of Alaska Anchorage, have been for the past three years. Before that I was community based in place. I was a senior researcher at South Central Foundation, a tribally owned and managed health care system and what I'll be talking with you about today is largely from that time as well as some of the work that I've been doing along with others at the Center for the Ethics of Indigenous Genomic Research. My pronouns are she, her, hers and just a visual description for those of you that may not be able to see as well as others. I'm an American Indian woman with dark brown skin, brown hair that now has some gray in it here and there and more there and more here than I would like. I'm plump, I'm wearing a flower shirt that is somewhat colorful on a black background I have glasses on and I'm standing in front of a podium and very nervous. I would also like to do a moment to acknowledge the lands of the indigenous peoples that we're on. I will massacre the names of the peoples because I'm not from these peoples and I don't think I've met anybody from these groups either which is a shame and also part of the historic trauma that indigenous peoples have as part of our history but I would like to acknowledge the Anacostan and the Piscataway people whose lands we're on here now. I am a grateful guest to be here as are we all I'm sure. One of the things that I had wished to do was to just quickly describe that when I'm speaking of indigenous research I'm speaking of the American Indian, Alaska Native peoples. As a Navajo and Dine, so Dine is our tribal name and when I'm a Wintoo person, when I fill out my demographics for a research study I will go ahead and mark down American Indian or indigenous or native and what that means is that I belong to a particular tribe. I am from the federally recognized tribe of the Navajo Nation and there are 574 federally recognized tribes. When I mark that down it's often in this race ethnicity category but it's also for American Indian, Alaska Native, Native Hawaiian people a political category. And I really want to state that outright because much of what I'll be talking about and much of what our participants in our empiric research have alluded to is coming from these 574 tribal nations as sovereign nations that have the ability to and are enacting research policy as it pertains to genetic research but also to other forms of data sovereignty. So I think that's an important context point and I do wish to belabor it because so much of the research and so much of the community engaged work that is occurring amongst Native peoples both here in the United States where we do have that political designation as well as indigenous peoples worldwide is with an idea that we are at one with other human beings but also with the spirit, the land, the cosmos, etc. And that brings us to a different set of values. Here in this slide I am showing a slide from the Alaska Native Knowledge Network that's describing some universal values of Alaska Native people. In our state we have over 200 tribes and like any group of 200 groups of people it's hard to find universal stuff but this group did and I think it's important to contextualize when we're talking about peoples that we talk about the values for which they hold. And some of what I'll be covering to use some of this language that I realize many of you are using other forms of language in the way of our technical terminology as it pertains to genetics, genomics and precision medicine and you may be a little bit less inclined to your humanities. And so let me just give you a reminder about a few of the things that you might have forgotten or just put aside since freshman year of undergrad. So I'll be speaking today about terms that are reflective of ontology, the nature of reality and of what really exists, epistemology, relationship between the knower and what is known, axiology, what we value and of course methodology, those strategies that we use to determine, to justify the construct of a specific type of knowledge. And I'll just say it outright, the methodologies, those methods that I use in my research are both post positive types of methods, the scientific method, but I also utilize an employee within my research because it's of and for indigenous people, indigenous research methods. And all of this is to say that these methods, these paradigms are grounded in the principles of relation, respect, reciprocity, responsibility, resistance, resilience, resurgence, restoration and reparation, all our words that you can use when you're doing categories, but also very important principles, particularly to indigenous peoples. And so I realize with 15 minutes to summarize a body of research is a little bit hard, but I did want to leave you with a few of those ideas that what I intend to speak about is how the people that I have come to serve in Alaska, the Alaska Native leadership of those tribes have sought to create our research codes and research teams and the questions that we ask in precision medicine and in genomics research in a way that where researchers and those being researched and those that are caring for those being researched are working respectfully, ethically, sympathetically and benevolently. And I say that because the way that those terms and what's elevated ethically is determined by the culture there. And that's why I wanted to show terminology of universal values there in English even though they come to us in other languages. So again, for those of you that are not living in the world of social engagement and community engagement, I had wished to put forward a broad definition of community engagement, the process of working collaboratively to proactively seek out community values, concerns and aspirations and I deliberately use those words proactive is what we've been asked from our tribal leaders in the Anchorage area to work on. So when there as a concept of a research project that that ought to be brought forward and under discussion and under discussion with whom I think is probably the question that many of us would ask next. In the case of the Alaska Native peoples and many American Indian tribes, we've entered into contracts and compacts with the federal government to engage in health care services on with and behalf of our peoples. And so tribal leaders are hiring and administering and managing health care systems now. The health care system that I had work for is one of those health care systems. So the stakeholders of interest are indeed those administrators that are tribal leaders, those providers within the health care system and the community members and many people that are those community members are both those enrolled tribal members from that particular tribe in that area or people like myself that are guest or moved there. And so I am not from the area that I'm currently residing. I've been grateful to be able to be accepted there. Some of the terms that we'll be discussing today are listed here and I just wanted to say that these terms are influenced by the community members and again when I say community members I'm thinking of not just the everyday person walking around but also the people that are wearing special badges when you go into the clinical health system as different forms of providers and support staff as well as those leaders that have hired those providers and support staff in the health system that I worked in we were called customer owners to further acknowledge that power relationship not just a patient but a customer owner or a person that owns the health care system and is also receiving services from it. So our community context is influenced by the hopes and expectations but also by financial resources by job and family expectations and by the broader social context of which we will hear more on and when we talk about participation I like to reflect on a on a little thing that's from 1969 but it holds true now and this is the latter of participation that Sherry Arniston had presented in a journal article back in the back in 69 in the Journal of the American Planning Association and I don't know about y'all that are in the room I think those of you that are on zoom and and in genome TV can see it a lot better but here on the the latter the bottom rung of the latter is has there's point one manipulation point two therapy and those two rungs are considered by Arniston non participation areas where it's not exactly participation in the engagement process and then moving towards number three informing number four consultation and number five placation all three of these are described as tokenism and then area six partnership seven delegated power and eight citizen control and those are referencing citizen power and I ask you to you know consider thinking in these different areas as you're thinking about community engagement if you define that community engagement as engagement of primary care providers or of another stakeholder group I tend to think of this across stakeholder groups and triangulate between those various groups that I had mentioned but this is just something I put forward to you if you haven't seen it and haven't engaged with it or haven't engaged with it in a while it's a handy way of thinking about participation and in aspects of engagement so in brief you know how do we do engagement and I think again this is about something that's been mentioned several times it's that forming of trust and for the work that I've been encouraged to do and required to do by our tribal leadership via tribal codes it's been to have active engagement of various stakeholder types in all phases of the project I know during lunch I was just talking and we were talking about you know who is it who it is that I work with and now I'm at the University and I don't work with students I work with community members now at a the work that I do is with individuals in the disabilities community broadly prior to that I would do work with various members of the indigenous community as co-researchers developing those research skills and training so when we talk about active participation in our tribal codes we speak of conceptualization of the project is an area for active participation the conduct of the project both in the way of how we manage the process of research how we manage resources themselves and by that I mean money and how we manage conflict so who decides what goes community engagement can look like a whole lot of things we've mentioned several of those things here depends on the project and I've listed here in engagement methods some of those more post-positivistic types of engagement methods and these are several of the things that we've done through Seager some of the work that I've been participating in in this century has been around biobanking that's owned and managed by Alaska native people through the Alaska area specimen bank and then that led to additional work that was through an R01 looking at community engaged research methods around precision medicine where we had conducted multiple pub public deliberations in a variety of different tribal settings and then some survey work as well as focus group work and interviews and all with indigenous researchers led by with the data owned by the various tribes and tribal consortia I've provided here a few different participatory research practices in empiric form and some discussion for you to look at in the future and just to note that I wanted to end on in the way of a scoping review so a literature review and this is something that my team had conducted looking broadly at participatory research in American Indian Alaska native communities and what we had found after this thorough combing of the literature was there was and these again are papers that were published things people wanted to talk about publicly right and in this we still see that there is a need for community engagement in the early stages of the research processes that there's an importance of guidelines for American Indian Alaska native communities more tribes and putting forward tribal code that is very specific to various issues and so could you wrap up on this please yes I can and then the last thing that I wanted to leave you with was just some things that we each could do in our own work regardless of who we are when you're working with tribes to respect sovereignty when you're working with any humans to respect self-determination to follow the lead of the community practice transparency humility to acknowledge the harms that maybe you have not caused but that exist for the people that you're working with and those that may come yet to build local capacity make long-term commitments and be flexible and creative oh thank you very much thank you very much Vanessa the next presentation will be given by crystal totzi and crystal is an indigenous geneticist and bioethicist from the school of life sciences at Arizona State University and she's going to talk about the opportunities for meaningful indigenous community engagement for population genomic screening crystal go ahead please yeah thank you so much for the kind invitation and welcome yet I shouldn't thank you hello to my people my genetics people I am going to do this rapid fire and I have a time timer to keep me at 15 minutes but I do want to start that even with the recently expanded ACMG panel only a handful of medically actionable genes had very information specific to those of indigenous backgrounds well how does this affect clinical utility what does this do and what does this mean for equity now we've seen variations of this diagram through the wonderful GWAS diversity monitor and this other which I've circled here has remained stagnant over the last couple of decades as it pertains to the inclusion of indigenous peoples in and GWAS studies or really any type writ large of genomics research however this is not a matter of engagement via recruitment nor is it selling indigenous peoples on the benefits of precision medicine and health it's a matter of overcoming mistrust thinking more proximally about health and empowering data decision equity this is a tells you about the structural inequities in terms of how healthcare is funded not only this is a pre-pandemic statistic in that before in 2018 the rate of funding funds to an indigenous patient was 2.4 times less compared to the dominant population so before we sell the next innovations and promises of precision medicine health we also have to reckon that there is a severe barrier in the amount of promises that the federal government has long promised indigenous peoples in exchange for resources and also we have to on this is a subject of a paper that when we keep telling indigenous peoples that they're going to miss out on precision health benefits if they don't engage without changing the power and balances that created research arms to begin with what we're doing is effectively engaging in a cycle of victim blaming and coercion I also want to say that simply dropping clinical genetic tests into our communities is not going to solve the health equity problem this is just a general clinical pathway of care for most rural travel patients that want to seek genetic testing in this case an indigenous patient has to referred out by their IHS provider often located 100 plus miles away which means that they're taking time off of work they are driving if they can find a car pay for gas get to a hotel pay for several nights in that hotel just to see a provider sometimes appointments several months down the road of question of course is whether or not there's a genetic counselor that's available content to contextualize clinical genetic test results at the end of the slide deck I have a very sobering map that shows you how very few genetic counselors are available in rural areas in rural states and states that have indigenous communities are these results interpreted against that relative lack of information on indigenous specific gene variation what is the potential impact in terms of the validity of the information that we're delivering what training is available to deliver cultural specific care to indigenous gene genetic test users our patients being fully informed about default data sharing and then here's an emerging question that we should all be considering what are the risks for indigenous patients who are not covered by federal privacy laws their Gina IHS patients and also many of our veterans a huge portion of whom are indigenous are not covered this is a catch 22 actually that it's a decision tree that many indigenous patients face what on asking the question of whether or not to contribute DNA for clinic clinical genetic testing so on the one hand some patients might decline because they can't afford our access testing or they're concerned about data usage on the other hand those that do agree implicitly broadly consent to any secondary data usage and data ownership by testing companies in any scenario this is why it's a catch 22 they derive little to no clinical utility due to lack of informative relative variants specific to their peoples this is outside the consideration of many clinical care providers who of course are focused on quality of care but by simply using a clinical genetic test that means that clinical the commercial gene testing companies can co-opt and clean ownership of indigenous peoples genomic data this is something we don't talk about enough that this data will be deposited into public data sets like clenvar even if patients know to ask to opt out of dating sharing do the tribal data sovereignty rules they don't have this option urban tertiary care centers that see indigenous cancer patients may also be bio banking samples and data for research under broad consent without tribal data nation approval what is this mean in terms of taking advantage of vulnerable populations that don't have any other alternatives what is this also should raise ethical concerns about the conflation of research consent versus the consent to care for minority minoritized communities there have been many studies that show that when you insert us a form that consents to having your data being used for other genomic studies amidst your other forms on on intake and at a point of care that many patients implicitly trust that white coat do they know what they're signing on that topic of equity this is something that for commercial companies have told me as a co-founder of an indigenous led biological and data repository called the native bio data consortium they asked us when we founded this nonprofit research organization how many indigenous patients and cross what phenotypes and we pushed back why don't you use our people's data and our DNA to study conditions that specifically impact us and of this we had a list of childhood conditions and metabolic conditions and what you all as a dominant population would call rare variants and we were told this by every single company that it's not profit generative to use our people's date DNA to create therapeutics that specifically impact us so recruiting more people into data sets is not going to solve the health equity issue dropping genetic tests in our communities is not going to solve this issue using the word democratize is a false synonym for equity when it's a system that's going to benefit most it's going to continue to disenfranchise us as indigenous peoples so this is something to consider when you're writing that next grant call there are issues in terms of using population descriptors like all processes of gene flow and drift indigenous peoples have had longstanding systems of kinship and relationality that are not mediated by blood such as marrying into neighboring tribes and also adoption and our clanship systems acknowledge this heterogeneous background under unified identity did not people to Navajo people before contact had over 400 clans many of these acknowledge that we acquired people from Pueblo individuals Acoma people Zuni peoples Mexican peoples Mexican people nation peoples yet it wasn't until 1934 that the Wheeler Howard Act was stamped upon us imposed these racially defined population groups meant to define us and dilute our rights to these resources but these ended up getting conflated under these eugenic notions of race and it's a dialogue that we are constantly in consideration and I'll bring that up in a moment but it's a very interesting because of a few data sets that are out there these are from indigenous groups with completely distinct genetic and cultural histories from our own from those disempowered groups that probably are not even recognized by their own colonial governments and in Mexico and further south and central and South America and yet these are the available data sets with sometimes as few as 30 indigenous individuals meant to impute something make some inferential statement about us indigenous peoples I don't have to tell you about some of the issues with using PCA is for principal components analysis for ad mixed populations for Native Americans we are often left out because our analyses on our small data sets necessitate a different type of small statistical sampling procedures that are incongruent with standard QA QC pipelines or worse for people that don't understand how unique our 574 plus of fellow recognized tribes in the US are we get lumped into the same category which doesn't benefit anyone it also reifies assumptions of biological purity when we are looking for to recruit least ad mix indigenous peoples we are also ignoring the real life to live experience of other indigenous peoples in regions like the southeast and east of individuals that have had more contact and more recent colonial settler history it also route ignores real contributions of inequities due to social and structural determinants of health we need to rethink one person one tribe because we often are categorized as one tribe and we do not acknowledge that people belong to multiple tribes and we do poor science this is something that I am a so grateful for and thank you for funding this recent NASA report one thing that I am loving is this move away from genetic ancestry to genetic similarity this is a very different I am loving is this move away from genetic ancestry to genetic similarity this type of typological move is something that I think should make transparent some of those statistical inferences that are irrelevant sometimes for our indigenous communities but in terms of like how community members feel about genomics research and data sharing this is actually something that was a topic of my my own research recently because we often default to tribal leader to leader rep our conversations forgetting of course that leaders are elected officials and we don't have the longevity of office or the the transmission of knowledge from those individuals from one administration to the next often time but those realities are consistent with community members and I simply ask them will my genetic data be used in other studies without knowledge or consent that was actually the top concern even more than will the research benefit my tribe that community members rated job and education opportunities created by health research higher than benefits of researching a disease or condition and this is because we think in terms of seven generations we take seven generations out we're not thinking about our own generations in terms of thinking about this pathway of innovation we also need to think about all the other dimensions of equity to conclude include data decision equity and this is a NASA article report on which I was a committee member so could you finish on this please yes thank you so we are now thinking about using AI machine learning approaches their blockchain federated learning to advance data consenting and data sharing creating our own portals that advance on group consent and broad consent through dynamic consent thinking about metadata labeling for the labels on repositories that we don't own but we're also thinking about building bio economies that are structured at all these different labels levels of policy and governance we also have to ask what happens to indigenous peoples outside of federally recognized territories and I just asked that when we're thinking about equity that we're also thinking about benefit equity as well as decision-making equity on top of engagement equity yeah thank you so much great thank you very much crystal the third presentation will be given by minky Lee minky is the deputy chief engagement officer for the NIH all of us research program and she's going to be speaking about advancing genomic research through community engagement on the all of us research program thank you thank you good afternoon everyone thank you for the opportunity to participate today I've been hearing a lot of different perspectives and I love the discussion the active discussion going on I'll try to address some of the points where all of us was mentioned today and where I think all of us could provide some insight into best practices and community engagement so the all of us research program launched in 2018 nationally with the mission to accelerate health research and medical breakthroughs enabling individualized prevention treatment and care for all of us and we aim to do this by nurturing partnerships with at least one million or more participants who reflect the true diversity of the United States which we've been hearing about a lot today ultimately with that with the data that comes from those participants that they've so generously donated we hope to deliver one of the largest richest biomedical data sets that is available to researchers so how are we doing as of last week we have reached over 500 thousand so half a million participants who have consented to the program consented to sharing their EHR and also completed initial steps of the program which includes a basic survey and also providing their physical measurements and biospecimen either blood or saliva over 728 thousand participants have created an account so enrolled and consented to the program and we have over 400 thousand EHR connected to to the database in our in our platform and then diversity wise we are also doing pretty well while the all of us research program is not solely a genomic study it with the numbers that we have currently it does make us one of the most diverse genomic studies so far we have over 80 percent of participants self identifying as coming from underrepresented communities in biomedical research our program defines those by both age race and ethnicity geography if your role or access to health care disability status sexual orientation and gender identity and educational and social income as well and then by race and ethnicity we are almost at 50 percent by the information provided by participants and the main contributor to how we reach this diversity really goes back to the program's intentionality from the very beginning I was lucky enough to have been a part of the program as part of the Division of Engagement before national launch and I can contest that we had funded and we had selected and funded inaugural community engagement partners even before we enrolled our very first participant into the program so we began with the Asian Health Coalition uh state pride net at Stanford University the National Alliance for Hispanic Health 54 in Nashville Tennessee and Delta Research and Educational Foundation each had a specific community that they worked with and since then we have not only continued those five partnerships but we've also expanded our engagement ecosystem to now include 18 nationally funded community partners if you count all of the sub-awardees through the different partners we have over 150 partners across the country and we know that the messenger matters especially when you are engaging these diverse communities so we have a grassroots local organizations as well as national and regional organizations again conducting various outreach and engagement activities across the country and as with the expansion we were very mindful of certain gaps and needs and priorities of the program when these partners go out and do their work we work with them within this framework that we've developed over the years and in our program we define engagement as bi-directional relationships the development of those relationships and also maintaining those trusted relationships often their work and our work begins with outreach and awareness again focusing on fostering trust with those participants potential participants and community members we build upon those activities and go into education and increasing access to the program which I'll speak a little bit to in a few slides we focus on engagement activities that can ultimately lead to enrollment and retention in the program but also again focusing on maintaining those trusted relationships with the community members and participants for participants who have enrolled into the program we really approach them as true partners we have several community members who have identified participant representatives into the program we have a panel of participant ambassadors and also participant partners who are serving in various program governance committees so I know earlier we had talked about co-creation prior to the inception of the program there were 17 community engagement studios across the country one of them was with urban American Indian Alaska Native communities and those community engagement studio sessions really fed into the development of the program protocol and we've had participant ambassadors serve on the very first steering committee meetings as well so we are very big on on co-creation and co-development as an ultimate outcome of all of this engagement we do focus on knowledge mobilization where either it's the data or the research outcomes that come from participants we want this to go back to either the participants directly or the communities that they have come from so that this benefits their communities in the end so from here I wanted to share some best practices that I think have worked great for for our program in hope that other studies can benefit as well an example that worked from the beginning is really having those partners at the table so these partners not only go out into the communities and conduct outreach and engagement as our trusted messengers and liaisons in those communities but they are also at the table at the program level where they are co-creating and co-developing with us pride net is a great example where even again before the launch of the program in developing those basics survey questions the questions that collected information about gender identity or a sexual orientation sex assigned at birth pride net worked with us as a co-developer and helped us revise those questions and really perfect them for the participant that we were hoping to engage our more recent partnership with American Association on health and disability we work with them to develop disability related questions in the basic survey last year which went out and those have been helping us collect disability UBR metrics and then also we worked with a HD on a communications toolkit for example of how fast our program works to really implement feedback from these community partners a couple weeks ago at our programs face to face one of the participants noted that we were missing closed captioning and they needed it for the for the meeting so within a few hours we were able to get closed captioning added and when they presented on their disability community toolkit they noted several best practices not just being 508 compliant and being 88 compliant when you develop assets that feature a person who is disabled that gives the that conveys the message that the event itself will be inclusive of people with disabilities and they can often tell if the person who is in the picture is really disabled or if you're just putting a person in a wheelchair so they said that true truly being inclusive really matters a lot at all different levels and those are the kind of values that we have been trying to really incorporate into our program at all levels not just participant facing but again at the co-development and program level as well and then selecting these partners we're also mindful of being culturally congruent in our engagement strategies Delta Research and Educational Foundation is one of our partners who was doing that kind of engagement they partner with HBCU's across the country and one of their programs Delta 2020 plus researchers provides educational activities for students and network members within the sorority so they present a speaker who comes and discusses their prospective research which is aimed at improving health care that is relevant to the people who are attending those sessions our other partner Asian Health Coalition is piloting a language working group which right now the program is available fully in English and Spanish and they have been advocating to add additional languages so they're working on a pilot where they will provide additional assets in the program in these five Asian languages and that's under works right now and they also partner with their community-based organizations so again partnering at different levels throughout the community is really important and it has been working really well one of the biggest barriers that I know I've heard today and I don't think we need to mention again is transportation when it comes to participation is a big barrier in times of time cost disability various issues might just be inconvenient so one of the models that our program noted that works well is to have this mobile engagement asset we started with one we realized that having physical measurement and biospecimen capability on board would help enroll participants from rural and other areas those who didn't really have access to an enrollment site it worked really well so now the program has two of those vehicles it's called the all of us journey if you go on the website you can see where it's traveling through the country and that journey partners with community partners again and they host them in these big parking lots the picture here is the one at Stanford University with pride net one event that was really successful was recently earlier this year in Brownsville Texas and four different partners the alliance and a HD and others partnered to host this vehicle and within a few days they were able to have a hundred and fifty accounts created and also almost a hundred physical measurements and biospecimens collected at that one visit the alliance calls it the one stop shop model which works well for a lot of our participants from our UBR community so we've been trying to implement that and expand access to people through that model and I'm gonna dwell a little bit longer on this slide then I had planned so I hope I'm doing okay on time because I think this is the most relevant to the different population screening and return of results that I've heard throughout the day today so the all of us research program began returning health related DNA results to our participants about a year ago now in December of 2022 prior to that we did launch return of genetic ancestry and traits results in 2020 so as of last month we now have offered genetic ancestry and traits hereditary disc disease risk results which test the 59 genes of the ACMG panel and then also pharmacogenomics results which test the results of seven genes we've offered each of these results to over a hundred eighty thousand participants and you can see how many have viewed their results here and as of now we are offering health related DNA results to about 5,000 participants per week and while we still have more work to do in expansion and also providing an equitable results experience to all eligible participants I do want to note that as we estimate based on the results that we are returning in terms of the hereditary disease risk reports that about three percent so I think two point nine percent of participants may receive pathogenic or likely pathogenic results and that when we reach a million that translates into almost 30,000 participants we receiving actionable results the program does provide clinical validation testing as a follow-up to those participants who do receive pathogenic or likely pathogenic results and that is actually an outcome of a series of listening sessions that we held with our participant ambassadors and community partners very early on in the planning stages of genetic related return of results where we heard from our community partners and participants and consortium members that you know people without health care or lack of resources the results can be very burdensome so clinical validation testing was added on in response to that feedback so that's something that we really are from at least from the division of engagement we're really proud of that we were able to achieve that outcome and then all all of us participants have access to our genetic counseling resource center so they can go into the participant portal and schedule a phone conversation with a genetic counselor it's that you can also request a interpreter into over 200 different languages and you don't need to have a specific result to request those sessions so that's also available as a resource thank you could you sum up in about a minute please yep I'm almost done thank you thank you and we also recognize the importance of diversity on both ends of research so when the data platform for the all of us research program launched in 2020 we recognize that we also wanted to have a community of diverse researchers using the data so we've put forth efforts to to help that happen you can see the number of researchers currently registered as users and we currently have set 97 minority serving institutions registered with a data use agreement our division also supports a lot of research engagement work to support that recognizing that there is capacity building efforts and things like that that are required for researchers to use our data platform pride net held a researcher boot base camp earlier this year we have a partner at Baylor College of Medicine who hosts annual faculty summits it's a three-day workshop essentially training people to use the workbench but also providing mentoring and support for like grant application writing things like that and they follow through for the whole year with these research teams to ensure that they are getting what they need in terms of support to use the workbench and then we have one other partner University of Utah who hosts is a high school student high school teachers every summer to come and use learn how to use the workbench and learn about the program and these are just highlights I'm happy to talk to you more about other partners as well and that's it thank you thank you very much pinky we're not going to move directly to our panel session and I'll call on my twin brother Rex to take it away thanks twin brother so we've heard three really interesting presentations and I want to make sure everybody writes down the questions that they have relative to these three presentations because we will it be including them in the discussion after we have have the panel and I did want to recognize actually that our the title is fairly uninspired for this panel and so I want to thank Caitlin for suggesting a potential alternative title too late to get into the booklet of course but beyond actionability discussing the who what when and why of expanding population screening to new conditions so maybe that's a better title for us to think about going forward so I would invite our four panelists who are Caitlyn Allen Ned Kalanj Jessica Hunter and Bob McNallis to go up to the front we'll actually use that panel chairs have been there all day long and what we're going to do is I we've asked each of the panelists to in two to three minutes and we're going to try to keep them to that to just make an opening comment about how they think about expanding population screening to new conditions and I should say also to new communities and we'll start with Caitlyn and just go in alphabetical order and then once we've had the initial comments then we'll have an opportunity for some discussion about the points that emerge from that and then we'll open it up to the floor and the folks online so Caitlyn thank you so we at medical University of South Carolina have recently as of 2021 November 2021 implemented a population wide genomic screening program and as I mentioned earlier this was a decision that we ultimately made was was that we would do this as a research protocol as opposed to clinical care and our goal is to enroll 100,000 individuals into the program and we're at 35,000 participants so far over the course of about two years we do whole exome sequencing and we are working with helix as our laboratory partner so they're cleo certified and all of the results for this program are returned into the EHR and we also provide free genetic counseling for all positive individuals so I'm saying that to just kind of lay the groundwork for where we are and how we I'll let you know kind of how we got there and to Rex's point about future expansion so currently we're we're looking at or we're screening all adults in South Carolina we are doing this we're returning tier one conditions and the hereditary breast and ovarian cancer Lynch-Genderman familial hypoclustrolemia and we did this made this decision as an institution because these were well vetted CDC tier one conditions they're considered actionable all the reasons we've sort of talked about today but that was something that leadership felt comfortable with moving forward with the CDC tier one conditions as our package and then considerations that we had early on are a lot of what we have been talking about today so ensuring that participants are understanding their findings both positive and negative results we're making really trying to make sure that we're be able to connect individuals to clinical care and being able to then ultimately track those downstream clinical outcomes so future expansion that we have in mind but have not done are actually thinking about pediatric populations and potentially expanding to provide this type of free screening service for for our pediatric population prioritizing more rare conditions we're also thinking about and actively expanding in the pharmacogenomics space and so returning results for genes that are associated with our priorities as as far as pharmacogenomics goes as well as returning secondary findings in particular apol one pkd one pkd two and part of the reasons for for focusing on those particular genes is because of our state's sociodemographic and and sort of health concerns with kidney disease morbidity as well as disparities in that space and then I think other considerations as as we're thinking about expanding are that we we have this under a research protocol which is great in a lot of ways that allows for us to re-contact individuals who've participated but we ultimately would need to re-consent them to provide them with additional findings and then we're also thinking about just what the right combination of considerations are for for for expanding and looking at ClinGen guidance to help us think through that. Thanks Caitlin. Next we can move to Ned. Do you want to give your introductory comment? Hi so I have kind of three things to talk about ebm old ebm new research and equity so there are evidence-based method evidence-based medicine methods that have resulted in recommendations for genetic screening on a population-based level so recognize that the USPSTF does that their recommendations are population-based recommendations to be offered to everyone in the groups that are identified for benefit versus harms and they have one recommendation around genetic screening which I participated in years ago and they're re-looking at it even as we talk today which was BRCA 1 and 2 so I think hereditary breast and ovarian cancer screening has a footprint with evidence-based behind it and there was a method that got the task force there then the eGap working group did its work and from those was there as well we came up with the other tier two tier one tests Lynch syndrome and familial hypercholesterolemia then I retired from genetics for 12 years 12 years later we have three conditions in tier one and I'm just a little amazed and that's so that tells me it's time to rethink about the methodology to kind of look at what Mike wants to do and look at what Les was talking about and saying we're not talking about lowering the bar we're talking about criteria that would help us get the evidence necessary to add additional tier one tests when we start to think about population-based screening so what I heard from Mike and I heard from Les we're kind of where the uncertainties are and can we identify those uncertainties and get a set of tests that are almost ready for primetime that is we can specifically identify what gaps would have to be filled in to get us closer to meeting those bars of either the eGap working group or the USPSDF and therefore provide additional tests that have that kind of sorry we used to say that at the clinical guide that kind of bulletproof recommendation that you know if you do this people are going to have better health so those are those issues how does that relate to research I'm glad North Carolina is doing that I would make a slightly different recommendation South Carolina oh sorry remember I'm from Colorado totally different grade and I know the Mississippi is between us and that's about it I think the other approach is to think about funding integrated healthcare systems to pilot these so we do this well we don't pay for it but we do it in newborn screening in order to get considerate consider is to be considered to be added to the routine uniform screening panel there has to be a pilot study that's identified at least one case and and CDC may be paced for that every now and then and NIH may be paced for it every now and then but thinking about taking on a guy singer or a Henry Ford or Kaiser Permanente and I don't mean to leave anybody on inner inner mountain there's three or four others I know loveless that are integrated healthcare systems who would be a great setting for saying we're going to have a pilot of a population based test for genetics so Terry and Eric's gone but if you're thinking about things that you could actually screen I'm sorry fund that might end in moving the bar a little forward towards new conditions being on the the tier one that's just a strategy I think about then finally we were reminded so well by the articulate and amazing speakers in this last session that there is this incredible opportunity to make recommendations that increase inequities through genetic testing and so all of the work we do needs to have the component of community engagement and the lens of how can we do this in a way that does not increase health inequities so those are my comments thanks very much just the hunter is up next yeah so I've been coming at population screening from the context of clinging so I lead the actionability work where we're doing evidence-based assessments of actionability generating reports that summarize actionability of gene condition pairs we score for factors associated with actionability how severe the outcome is how effective the intervention is the nature of the intervention to the patient and how likely the outcome is to occur aka punishments and all of the scores these semi-quantitative scores and our final actionability assertion are available on the website we've developed this framework and the context of secondary findings which is why most of the domains are based on the individual but we're recently shifting to more population-wide and we've been adapting this framework for the context of polygenic risk scores and now shifting as well to population screening and thinking about as part of that effort what factors need to be added to this framework in order to account for a population-wide impact you know severity at that point it's not just severity in the individual but you know how impactful is this condition to the at the population level and those kinds of things and so yeah we're beginning to think of what factors we need to include in that framework a lot of which we've we've learned through the polygenic risk score effort um but for oops that's a weird echo uh but for community engagement i will say that we have found this effort to be quite um sorry that's a little distracting somebody logged in someone in here please mute their computer or leave the audio component if you're online um so we have engaged in community engagement as part of our effort which we have found very valuable um you know there's a certain domains that are more subjective than others like the nature of the intervention you know the risk of burden to the patient you know is how we see it as clinicians and researchers different than the patient and is it different between patients who have had the intervention versus patients who haven't and so we've done a lot of that community engagement to essentially make sure that how we're measuring these domains is how the people that we want to benefit from what we're doing um would measure those domains and so that's been really um helpful as well as for particular topics we've done community engagement um and an example is uh prelingual deafness uh we engaged uh we did some community engagement with the deaf community um to think about how to phrase or terminology we found a lot of the evidence we found um use terms that weren't uh along the lines of what the deaf community would use and so we use that to essentially um make sure our reports were um inclusive um and included non-clinical interventions that were suggested to us through that community engagement so it's been a very important effort okay and the last but not least is Bob Nellis great thank you Rex good afternoon everybody and forgive me I'm just recovering from a cold so my voice is a little smokers cough it feels like um I'm from the office of disease prevention at the national institutes of health and unlike many of you I don't speak genomics as well as you do um but I do speak prevention and I was actually quite uh inspired by Les' talk this morning about that genomics is not an exceptionalism there's no exceptionalism there with regard to prevention the Bayesian theorem still apply um to all of the work that you do at ODP what we're interested in is sort of assessing and facilitating and stimulating prevention research across the 27 institutes and centers at NIH we do that partially through developing and coordinating and then implementing some of those prevention programs we work closely with the U.S. Preventive Services Task Force and the Community Preventive Services Task Force and Healthy People to really identify key evidence gaps in prevention and I think there's some other opportunities here to work with all of you to identify some of the key evidence gaps that might exist as you think about population screening um for us I know as we look across our prevention portfolio across NIH um of all the prevention research about a third of it actually looks at leading um risk factors about a ninth of that prevention research actually tests a randomized intervention for that um uh leading risk factor and about a 20th of that actually tests that randomized intervention for a leading risk factor in a health disparate population so really very small representation of health disparate populations uh even in traditional or should I say um just our run of the mill prevention prevention research I think that provides us some opportunities we've you know added um a whole area of our strategic work that involves advancing um research in health disparate populations um and really I think the important thing that we can offer is ways to understand and address research gaps and I heard a lot of research gaps today and I think I can offer a couple reflections on that a little bit later but just to let you know uh one other sort of interest that I have is implementation of this work and we heard a lot today about how it's going to fall on primary care clinicians um we've got a little bit of experience with that I'm reflecting back um not quite 12 years ago uh but 2000 2005 the American Academy of Family Physicians held its uh its educational focus on genomics to try to help family physicians embrace uh the era of genomics I think here we are 18 years later and I'm not quite sure double AFP has our family physicians in general have embraced it at quite the level that they might have hoped so I think we do have some work there but I think there's lots of opportunities too so with that I'll stop and look forward to the discussion okay so before we open it up to general questions I wonder if each of you could just take a minute or so and comment about what aspects of community engagement you've used in you starting up your projects and how you do that in an ongoing way to make sure that your work is continuing to be relevant to the communities that you're uh doing your research in I shouldn't say in partnership with so for our work at MUSC we developed our own community advisory board so there were existing patient family advisory councils that the you know that the hospital has in place and we felt it was we felt strongly that we needed to have our own community advisory board for the specifically for this project when trying to identify people that were going to be part of the community advisory board we were very intentional about representation from across the state of South Carolina so MUSC is in Charleston and mostly as a footprint there but our screening program is designed for the entire state so we really wanted representation from across the state we also wanted to find folks that would have differing perspectives about genomics and screening and not all of our community advisory board members agree or that they have varying opinions and that was intentional and so that's been that's been important for our work as well as the community the engagement of our clinicians early on and some of the co-creation and decision-making we had about how to just set up the program I guess I'll talk from my role with the state health department and say that as we put together programs now that are designed to improve delivery of care in historically marginalized communities we make sure first of all that our advisory committee includes a diverse group of individuals with lived experiences beyond that of the people that work in the health department or those that are members of the white supremacy majority that still makes up our state so I think that outreach and inclusion is important and then the next step is as you design a program even with that representation to take it into communities to a broader discussion with more people who have lived experiences differed from the ones on your communities so that you make sure you have enough input to tailor the delivery of a program that's going to be most successful for a given community well and I touched on this a little bit already and getting community engagement on kind of validation of our metric and I think that will be particularly important as we move forward towards population screening but for our individual reports that's been a bit more ad hoc on when a little red flag pops up then maybe this is an important report to get feedback on but I think perhaps going forward we need to be a bit more systematic about that and I think that would be a really important addition to our efforts. Great thanks and I'm not speaking for ODP because I'm not sure we engaged the public quite the same way but coming off a couple of experiences that I've had recently with in some research areas I know Colorado actually has University of Colorado has you think the boot camp yep really good results there's a team at University of Colorado that engages patients in development of educational materials and they found better uptake of whether whatever the intervention was whether it was blood pressure screening cholesterol whatever it was getting patients directly engaged and developing the educational materials that are used is an effective way to help engage patients and then help improve the educational process. The second one is from the organizational standpoint and I'm not quite sure what ACMG's approach is but there are other groups that actually at professional meetings bring together patient clinician or patient researcher pairs, dyads to help make sure that the patient perspective is represented in every aspect of the work not just as somebody to come in and just get their perspective but actually fully engaged in the work and that also seems to be a very effective way to keep people in line not in line but engaged in every along every step of the way. Thank you very much so we heard a really nice description from Minky about how all of us has reached out and engaged communities. I'd be very interested to hear from Vanessa and Crystal how that might look if we thought about engaging your communities. I was really struck by the fact Crystal that you said that only five ACMG medical action medical actionable genes were even relevant. How do we do a better job? There's two questions there first of all I want to highlight something really important and I'm actually on the Indigenous Research Working Group for the all of us research team that's where I stepped out about an hour ago and I had this exact conversation that there is a distinct difference between engagement versus research capacity and training. Now they can be part of the same spectrum but they are two polar opposite sides of the same of that spectrum you cannot use one versus the other. So when we're going into talking with community members and tribal leaders we cannot say oh well we're training students checkbox engagement that is a completely separate activity than what we're calling for which is the more difficult activities related to conversations that are difficult building trust. Good qualitative research and not just surveys and focus groups at democratic deliberations and Vanessa has given us a couple of great examples, several great examples that we have traveled in our communities as well. So I just want to have those two distinctions the other part about like the medically actionable genes. Yes so of those those are mostly centralized on colorectal cancer gene variants especially related and there's a recent report that stated that that is now the number one cancer in our communities and so there's obviously a point of intervention here but there's also a lot of concerns so in terms of when we refer an IHS patient out what tests are we giving them is it relevant how are those test results being returned back to the community are they validated in a research study versus a clear certified setting I know it's not usually a concern for most institutions but it is a concern for institutions that are smaller and that are community held. So there's a thing of scale here but then also what does that mean for meaning for one community versus another right and there's just a lot of concerns about Gina a lot of concerns about harmonization of data. The other definition that I wanted to just state is that we're talking about population screening and like research domains and then public health domains but those two spheres are separate in terms of their ethics and in terms of their regulations and in terms of how those are sanctioned so and especially the tribal IRB, IRB realm I mean that there are rules and like past practices for engaging with tribal IRBs when they're recognized by the NIH because under the common rule and single IRB mandate they're not always recognized there's only like 33 of them IRBs are not recognized other forms of a tribal research governance are not recognized but we're talking about public health now that's an entirely different animal we're talking about the different federal agencies we're talking about different illegal proceedings should something go wrong and then how the IHS and pathways of care especially even for you know we're just talking about rural tribal settings there's also separate considerations for urban individuals of which according to the last census now 80 percent of indigenous peoples that of AIAN communities now reside in urban areas there's a lot of issues here so sorry I just want to unpack that. Crystal can I I'm going to put you on on the spot Crystal is a great follow-on ex by the way formerly known as Twitter but you said you had a post earlier today not from this meeting but about from another meeting about Indigenous leadership and I think that was you didn't quite go there I thought you might but I'd be really interested in your thoughts about leadership of research as opposed to partnership and research. I'm going to point it back to you can you clarify a little bit more. So you basically said that for a particular proposal that it was discussing Indigenous leadership within the proposal and the proposals reflected engagement but not leadership and so that seems a a different level of engagement and you you were starting to get there but you didn't quite get there and I think you have some thoughts that would be important as we think about the research agenda. I have a lot of thoughts okay well right now we're presuming research from a very our common pathway of these are research proposals initiated by individuals with PhDs working in academic centers and then they're reaching out to to communities and even the work partnership is either thrown around really loosely and it's not really respected in the same way in terms of resource development. The post that I mentioned earlier or wrote earlier was something to the effect of we can no longer call things Indigenous led when institutions have 57% indirects or sorry Hopkins like near 100% indirects and those go to PIs when most of the grassroots activities are actually from community members themselves and they're the ones that are generating the data the protocols going through all the approval processes and they get a tiny fraction of that and this is something that we need to consider for the NIH models. Now NIH has been really great in the last two years you've started to really consider that you do not need to have a terminal PhD status which is important when you consider that the Seattle Indian Health Board Director who just stepped down recently is a master's level investigator so this meant that for individuals like her myself Joseph Irshada who's the Executive Director of the Native Biodata Consortium who were me recently master's level we couldn't even be PIs of our own research so and also what that meant is in terms of contracting with universities we were always having to have our lawyers step in which we have lawyers thank god but that took a long time and that's the first time and not the last time I'll say thank god for lawyers but with the single IRB mandate which has you know and common rule it has language about respecting tribal sovereignty that only means for tribes that have an IRB that has an FWA which is like a small small fraction of the 574 recognized tribes of those then you know you could still work with a you know large-name institutions that just happen to have institutional standards of refusing to see review to external IRBs to include tribal IRBs so like how do we win even if we wanted to partner with with universities we are constantly at the wrong end of that decision making equity and power and authority and there's something to be said that the distinctions between respecting versus operationalizing indigenous data sovereignty okay thank you I wanted to make sure Vanessa you had a chance oh just to ask about the question of what do we need to do a better job how do we need to think about different ways of thinking we've heard some great ideas already about helping address the inequity that we have inherently with indigenous people well first off I don't believe that the inequity that the american government has with indigenous people is alone to indigenous people I think we're talking about historic equities inequities with many many groups of people and and so there are some concepts that ought to be considered and which is exactly why I put together that I highlighted that ladder of of collaboration of engagement at the very least we should start attempting to climb such a ladder recognize that that ladder exists and then when we move forward on our work be it as an individual or as a system that we move up it I think one of the things that you know I was hearing from Dr. Lee was you know in in the example of an individual with disability that is being highlighted on something as simple as a flyer that indeed the community will recognize if a person is an able-bodied person that is a model that's sitting in a wheelchair versus a individual that has had that full lived experience as a for instance which is a tokenism right straight up and also the material wealth of being part of the the research is not being shared then with a member of the disability's community right as an actor or as a model for that group indeed it also precludes any individual or member of a group leader however we wish to say it from having their voice at that table and being able to describe and discuss what they see as being a the face and name and body for for such research so again I go back to these concepts of participation and really thinking long and hard what it means when we are indeed working with others and what our intentions are and being very clear and determined in those intentions right there's a question online that may be relevant to maybe if Vanessa and Crystal could briefly address it and the question I wanted to know all of the presentations have significance to oppressed communities that are not federally recognized so how can the nuances and the experiences that you've mentioned particularly for American Indian and Alaska Native research experts and the concern for respect how can all of these be extended to the other underrepresented minoritized groups and also the ones not federally recognized can I just confirm I we got to note that the folks online are not getting sound is that still correct okay thank you sorry church either of you I think what you just addressed relates to that and you could briefly summarize for us so oh you want me to rephrase the question okay it's really interesting we talk about biology in terms of distinctiveness across populations like between and within but then we don't even talk about relationality in terms of of how like in terms of respect of our kids and and I think this is really important because you know we have federally recognized tribal nations but we don't acknowledge their cousins their neighbors their their their and which have distinct and different types of historic harms but we we try to legislate across genomes and genomes don't care so this is where I think in terms of the principles that Vanessa brought up in terms of of thinking in terms of respect and humility but also extending that to how we are thinking and globally and ethically so I would just like to add that one of the things that I hold to with the work that I've been able to do is the role of sovereignty and how our tribal nations are able to force the issue of of the research codes of the meeting behind those research codes and other marginalized groups don't have that opportunity and so it doesn't mean that they don't have similar values that they don't have similar desires to be included in research to drive the research to have power sharing to be all of to do so many things all of those ours that I rattled off earlier tribal nations instead have that political authority to engage in in tribal consultation right one of the things that I had the blessing of being part of was developing the tribal consultation policy for the NIH that's not something that other marginalized groups get to have because there is no authority to do that it doesn't mean that there's not the moral imperative for all of us to follow through on such things and so that's where my mind goes is that we have a responsibility to our various marginalized communities to inquire in the same manner be it required or not when we are engaging and seek to engage in whatever type of work even in the absence of a legal no that's that's great thank you so I think we'd like to open it up for broader discussion so go ahead so I think one of the kind of underpinnings of dissemination and implementation research is that we wish to make the fruits of research as broadly disseminate as possible and so and that's a little bit different than thinking along the lines of basic science research right but it's engaging with communities and it's sort of finding ways to make something work right and so do you see a difference in the way that engagement should happen say let's say in my state of North Carolina we do have Native American populations and you know if we wanted to say let's let's have a statewide effort to do this genomic screening because we think it's useful so how do we go about doing that with the patchwork of constituencies that we would want to engage with it's the best way to do that yeah if I could go ahead and start the response one of the slides that I glossed over quite quickly as I frantically pushed the forward button was a slide that I had to deal with a newborn screening and in Alaska we one of the secondary additions to newborn screening was a in what was it 2003 2006 somewhere around there was a CPT-1A Arctic variant and the addition of that variant as well as materials that were developed for the public that those that had screen positive those were developed by between the state of Alaska as well as the Alaska tribal health system and they were directed to providers of care for Alaska Native people that were within the Alaska tribal health system Alaska tribal health system is again managed by Alaska Native people and is caring for about 40 percent of the state population when you look at managed care and clinical care there a kind of sticky thing there is that the materials are being mailed out when a person has a positive it's a cd like a dvd hasn't changed in all these years and it it's not something that was those materials as patient education materials hadn't been tested with everyday people that were receiving them but instead were to inform the providers of care for those individuals it was until a couple years ago that the health system had put together a research project that was NIH funded to to do some community-based inquiry both with the current providers as well as with the the patient population particularly those that had received such things in the mail and had received information that their child had this particular variant in a double whammy sort of way right and these focus groups had indicated that indeed the people had and the providers still had a lot of questions and the materials weren't quite as helpful as as we had all in public health had hoped they would be. I say that little story here because there are procedures that we can all think of now after the fact of what could have been done could have been done in the way of engaging with individuals with parents that are receiving this information ahead of time and developing it I mean goodness even the title of the materials is something around the lines of it's there's not only one in your energy crisis in regards to the metabolism issues yeah it's it's it's got it's a hot mess I think personally in the way of the materials but anyway I use as a little case study in the way of what could be done different in the for community engagement on something that is highly specific to an indigenous population an Inuit population not just here in the United States but other Inuit populations where that is there. So we've got four hands up and there's one hand on the line yeah we saw that one. So Mark, Pat, George and Bruce and if anybody else wants to see a line make sure you get your card up but Mark you're up. Thank you. This is to the panel though the handicap here is that my question is about somebody that's not represented at the meeting and that is we're talking about engagement and there's one entity that is has been specifically mandated to do patient-centered outcomes research that is PCORI. They also have a mandate related to rare disease which they've had a real challenge meeting their expenditure in the rare disease space and it seems like there's some opportunities there particularly since they've developed a whole set of methodologies for engagement research and so for any of you on the panel or Alana who I know has done a lot of work with PCORI that could comment on whether there could be some opportunities for engagement with PCORI in this space I'd be interested to hear your thoughts. Yes. Could you elaborate? Yeah. Sorry. I cough when I laugh. PCORI is great I mean and they have now years of experience doing this kind of engagement work I think they're a great target for us to work with. We've worked with them on a couple of different projects and they bring I think a unique perspective to it they build a you know a patient advisory group with each project as it comes along. I think there's some real models they might be able to tap into and use so I think that's a terrific idea and you're right I wish I thought of that earlier too to mention it. Alana. Yeah I just want to add to that too I think we have seen some I mean the fact that that we're talking about this here and as a grant reviewer both for PCORI and for NIH over the years I've seen some of a lot of the PCORI thinking move its way into the NIH grant review process we're like hey they didn't include there is no engagement at all in here in an NIH grant you know so um I think there that yes there there's definitely we can do more um but I think there is a little bit I think just what we've seen here and that we're talking about it is the right steps and I'll let the questions continue. So Pat I think you're next. Okay um yes my question's from Niki I had a question about the all of us research program so I thought you did a great job of giving us a really good overview of the diverse engagement efforts that all of us has pursued and then I was personally just interested in the the last slide that you showed that said for both hereditary cancer and pharmacogenomic variants then only about half of the participants viewed the results so my question is was that in line with your expectations and do you have any under have you explored the reasons that have only half have chosen to view the results? No it wasn't especially since it was a long time coming so a lot of our participants had voice that they were waiting for it it was taking too long so we definitely expected a higher percentage of participants to view their results especially with all the support that came with it in terms of genetic counselors and the reports themselves also look very nice they're visually appealing we had a lot of community partners review the language on them it comes with information on the gene and then just ways to understand the information better we the team is looking into how to increase that but yeah right now that's that's where it seems to be and for some of the participants if we anticipate a pathogenic or likely pathogenic sometimes we do they those are the cases where we do require them to set up a meeting with a counselor in order to receive their results so that might be an additional threshold in viewing their results but overall the once they do have a consultation with the counselor the satisfaction score is very high it's around 60 percent so it we don't think it's a something to do with the experience itself but something else must be um in action there and we we are having to look into it it has been less than a year so maybe it'll just take more time I'll just make a quick comment that that percentage is actually higher than people see with biobank return of results um we were just discussing like for both pan and geisinger that's 50 percent is actually much higher than we see with trying to return biobank results and whether people respond to that we do acknowledge it is high uh bruce yes thank you um this is actually it's not my technical question I guess for chris though and that is um you mentioned the use of blockchain and the very very very low when I expanded up blockchain there's always maybe wonder if it's a way for people to control their to moment information so that they can share it when they want and with whom they want and I'm wondering first of all if is that accurate is that how we were using it and then if so is that is it scalable and put in look to get us out of the quandary where we have to know the data frequently kind of lost in place like within that academic medical center for example where it isn't easy to report it to other settings yeah so blockchain is more at the infrastructure level and there's smarter people that work within data systems around the table but basically um first I would say that dynamic consenting is something that provides an individual level of of sort of decision as to to whom somebody might consent to have their data shared with types of data they are willing to share what like rules in terms of like data dissolution like do have are there cultural rules that they would prefer to enact that they want to remove their data or draw down their data and you can employ dynamic consenting models actually um at the the earlier design so I think that UK Biobank that employs this type of approach this is also an approach that we developed as a native bio data consortium as part of our infrastructure so that we can license it to other tribal nations worldwide for free um the the thing about blockchaining is and it's very important here is um you can sort of um define different types of user um access read rules um but those type of levers are usually done um by whomever is designing it right but and then again I'm seeking more from a community centered approach um you can actually have community members act as the authority nodes for deciding who gets access to certain types of data under what conditions what study types etc with in partnership with tribal nations so sometimes this is like uh positioned as an individual versus group consent speaking to an author of a great paper here that um uh interviewed tribal leaders and they say and one of the leaders and elders or was it a leader and elder both okay uh that stated it's not individual or group it's both and it's actually up to tribal nations to decide how they're going to make those decisions not outsiders but in terms of how tribal nations can govern you can have both an individual level of of of control and um as well as at tribal nation control blockchaining is one way another way is federated approaches where you can actually um like read and maybe to execute without having like full access to all of the data within independent nodes that are governed and housed by communities and this is something that um many other systems are i'm flowing as well uh i can't tell is it alana or kate that has their card up i made my comment earlier that okay sorry that like return of results was just like 50 pretty high uh christie yeah i had a follow-up question for mickey young um so i think you mentioned from your presentation that um genetic counseling was offered to all of us participants even if they did not have their results returned so wondering what kind of uptake you got on that and and what were their questions if you if you know that yeah um i'm thinking the only information i took away from that um in preparation for this talk was that the c-sat scores were high the satisfaction scores were about 60 percent the specific questions i would have to look into that in terms of what they're asking um yeah i don't have the the feedback off hand right now sorry uh is that a second question or no no great so i wanted to ask finessa you mentioned in your the list of things early on in your talk that should be considered you mentioned thinking proximally about health and i didn't quite understand what that was so could you explain what you mean by that honestly i don't know what i meant by that right now honest i i'm not reflecting um i'm not recalling where i had that particular point so i apologize it sounded very good it did it made it made great you know it's because to me i might interpret it as as the health needs of the community that are that are primary to the community but i could put any you know interpretation but maybe you you know or you have some thoughts on that i had that sentence on a couple of my slides um okay no often and unfortunately uh the human genomes diversity project the thousand genomes genographic projects are great great unfortunate examples of this in which researchers would enter into communities promising things like therapeutics we'll we'll discover the thing that is causing for instance the NIH NADTK made a similar promise to the often peoples like in 1955 we're going to treat diabetes and then leave not telling disclosing that maybe the it'll take a long time for these intervention vengeance um to actually be translated back to their health and meanwhile community members are wondering what happened where'd you go with our data so um that's what i meant by thinking more approximately um and in terms of the promises messages engagement etc. George thanks this has really been a terrific session and as i reflected on the values and the principles that you mentioned i was reminded of um a major principle that we learned as part of the NIH wide community engagement alliance particularly during the COVID era was the issue of trust and we we think we know when trust is enriched or when trust is eroded but from a research perspective and in your experiences do you have objective ways of assessing trustworthiness i mean we we definitely want to be trustworthy and we want to do everything possible to get there but how do you measure that and and what have you found useful from a research perspective in answering questions about trust if i might go first um within let's say the organization that i had worked for south central foundation there was a requirement a research requirement that researchers were to put forward their concept in a concept proposal of one pager um so they could submit let's say the specific games page as well as a cover page saying this is what my intention is and why we think that we'd like to partner with this agency and here are the outcomes of interest and the procedures that might be used you know again very broadly i would say presence or absence even of the request is an indicator of trust we had many researchers that once they discovered that that was a requirement pre-approval they were out not interested in participating not interested in partnering so just to get presence or absence to engage in a process um that's community-led would be a very low bar one might think but it's an indicator there's an emerging field of equity metrics we keep talking about equity how do we measure it that's actually a very interesting question some of this is rooted in the health economics i have a concern that health economics is really short term and also may substantiate systems of that are incongruent with actual community health but that is a starting point in terms of tracking whether or not researches are delivering on the promises that we have made one way to do this is tracking but research dollars into communities so for instance with indiged data which is an indigenous data science education workshop i started we actually shifted all of our research capacity and training outside of universities into tribal lands so we're funneling outside dollars from nsf and also philanthropic orgs and other than paying for or not paying for airlines everything else goes to tribal nations at a rate of 67 percent that is something that we we we is like above the standard of other types of training opportunities that have federal sponsorship in some form in some form we can do this with similar other types of grant mechanisms when we start tracking where are those indirects going the where those directs going who are we actually training etc and again also aligning that education is not the same as engagement thank you reps could i have a quick follow-up from Vanessa please because something she said actually just triggered a reflection you know very often when we put out a notice of funding opportunity we don't always do it but sometimes we ask for a letter of interest so did what you say apply also to letters of interest when before we even ask you to come in with an application yeah so let me just step back the process that i was engaged in with the tribal leadership of south central foundation which gets their tribal authority from the cook inlet region began with a concept proposal phase and then if approved the researcher and their team would in this was also true of those of us that were embedded within this health system so it wasn't a they do that we do something else no every single researcher student as well would complete the concept proposal phase that goes all the way to the board of directors if approved then the person was would move forward in partnership to develop a full proposal and then after that had been written and approved it would that full proposal would go back through a full process now once everything was in hand in the way of this is what the engagement would look like this is what our recruitment flyers and materials would look like these are our research team members on nauseam this you know on and on these are the actual majors then at that point then that's when there would be a suggestion of moving forward with funding application so and then this is a really detailed manner in which engagement would occur where there would be a hand and this required is how that group had chosen to have a hand in the development of the research question the processes the etc etc and embedding that information within the healthcare system all right well thank you everybody this has been a terrific session appreciate both the all of the panelists and the presenters as well as the questions and I feel like I need to summarize but at great peril here because there's so much to be talking about but first and foremost I think the one of the most important things I heard is the importance of building trust with the with the communities that you work with second I think I heard we use the word engagement a lot but that engagement is not enough we really need building on trust but we need to really think about true co-creation from the beginning of the project I also heard that I think we need to recognize the need to have a different view of what community leadership means because the community may not always have the same educational opportunities and the same socioeconomic opportunities that the traditional academic community does so I think that's something that was important I also heard the importance of recognizing marginalized communities that are not only marginalized but don't even have a voice at the table which some are fortunate enough to have and so I think we've heard the need in the context today that we really do have a lot of work to do in terms of thinking about making sure our databases include all participants all humans and make sure that we have a commitment to doing that and I think I will stop there but I'm sure George do you have anything else you'd like to add to that not a whole lot of rest I think you did a terrific job only to highlight the importance of the principles and the values and the terms that Vanessa mentioned I think trust is in there but the second thing was also many of the challenges that traditionally in our NIH funding we haven't paid attention to in working with community partners or what can I think it's a little different for sovereign entities but really rethink how we fund and how we support particularly small community-based organizations and small partners I think the final thing I wrote was paying attention to the power imbalances we may not think of them that way but we have our structures and we go through with our structures but they may really reflect serious power imbalances that go against meaningful community engagement but this has really been a true of rich dialogue so thank you all thank you all and and I will just say I think all of us that work with participants in general regardless of where they come from should put a copy of that ladder on in front of our desks great so so that thank you very much to everyone let's give them a round of applause and thank you very much for sticking it out to the bitter end here of today we have tomorrow and we'll start it at nine o'clock with the session 8 30 with breakfast no I'm so sorry so this is why I have my friends here yes so 8 for breakfast 8 30 for for the session thank you very much goodbye