 All right, awesome. Oh, no problem. Pleasure to be here with you all today. Sorry about that. To talk about some of the work that we are doing and the American Medical Association on operationalizing racial justice, and forgive me for my voice I've had. This is my third presentation this morning, so it's going in and out a little bit. So I think very core to this work and foundational to advancing equity and racial justice is how well and committed to we are the center of the voices, the experience, the ideas, the history of the folks who have been most marginalized in our country and society. And so first just offering a land and labor acknowledgement of the indigenous peoples rather, who have been present on these lands for thousands of years, as well as the extraction of life and energy for the sake of labor of those of African descent for over 400 years. We continue to celebrate the strength and resilience and absolutely carry our ancestors, I do personally on our shoulders and know that many doors are open because of their work. And this is not the beat to the exclusion of many folks who have contributed to the pushing of this country or the foundation of this country. So just to have a few grounding terms that I think is really important because what I've recognized in doing this work, I've been doing this kind of operationalizing at an institutional level in equity work for a little over 15 years now, 15 to 16 years. And they're still not clarity or consistency sometimes and how we're using the words, how we're defining them, or at least consistency and understanding the concepts behind them. And how different words mean different things and can have different impacts in terms of our solutions if we're not using specific words. And so oftentimes in this space, we hear lots around diversity and inclusion, which is, which are absolutely critical to get to the point of equity. But I just state that it's never sufficient enough to get to the point of equity diversity being that of which we have identities. And we all carry multiple identities within ourselves and amongst one another. And it's important and highly critical that we're evaluating all people and all identities. That is very foundational to diversity and equity. And then also critical is how well all of these varying identities actually feel included that they belong, that they have power in the space is really when we start to talk about equity. And I think that that is the part where sometimes I'm so sorry, is lost. That's the part that gets lost over time is that equity is about really this redistribution of power and resources to where there's a greatest need. And we need to have lots of intention and focus on that. Here's another way that it is pictured. Millennials will very much tell me in the younger generations that this is a great kind of foundational and starting visual to understand the distinctions between equality, equity, and justice, but also there are problems with this particular visual. It seems very kind of male dominant kind of in its picture and also very able and carrying a sense of ableism within this visual as well. So all these things we have to be very conscious of. But ultimately health equity meaning that we're assuring and having the conditions and resources, power, and opportunities to achieve optimal health. And I say it's critical that we have all of those. And Kamara Jones speaks much about and this is her framework that I think is really helpful is that we're valuing all individuals and populations equally, recognizing the historical context of how these injustices were produced and then we're providing resources according to need and we can eliminate disparities or health disparities will be eliminated when we actually do achieve equity. I am important to this frame and I think oftentimes has gotten left out of the conversation of how health is produced, how health is created within this country, is this broader framework? I come from a public health space for most of my career and now more deeply rooted in healthcare. And definitely even more so can appreciate the narrative being slightly off in terms of it really evolving and revolving around the clinical care system in the context of clinical care. And really lots of focus on the hospitals and the hospital walls, the doctor's offices, health insurance, all things that are really important that we know to take care of one's health, but really treatment focus. Not really looking to really the 80% or acknowledging the 80% of what really creates health, which is to the left of this graph, which is more extreme. And I think if we don't appreciate and we don't adopt and find ways to really integrate this into the work of any of our institutions around health in this country, we will still continue to kind of miss that mark on the ability to advance equity because most of the inequities are driven by these upstream factors and conditions that exist within our society and in our world overall. So the conditions of which we live in, the institutions and the power and the policies and the laws that set up those conditions and then the lenses, the root causes of inequities, those systems of power as well as oppression that provide and drive the lenses of how these laws are set up just based on our historical context. That avoids supremacy, racism, classism, sexism, homophobia, ableism, and xenophobia, just to say a few. But all of these are really critical to this whole framework, I think is really critical if we really want to address health equity, as well as even more specifically structural racism. From a definition point of view, just on the bottom right hand side, also pulling from Dr. Kamara Jones. And I think this is a brilliant definition because you really can interchange any ism within this. But the idea and the intention of structuring opportunity and assigning value based on race, unfairly disadvantaging people of color and unfairly advantaging whites, incurring at four different levels of internalized interpersonal, which is the one that people usually kind of gravitate towards what's happening with interactions through providers and institutions with communities and patients to the institutional level, which is the work that I've mostly focused on, that also speaks to the structural level. And at the structural level, this is a paper by Dr. Bassett who was my former boss and former health commissioner at the New York City Department of Health, which I've learned tremendous amount from. We know that structural racism is the ways that in which historical and contemporary racial inequities and outcomes are perpetuated by social, economic, and political systems. So they're not siloed, they're not isolated forms, but they're mutually reinforcing systems of healthcare, education, housing, and employment, media, and the criminal justice system. And so this is really the foundation of what many of us now know and believe of why our health equities exist in this country. And then another context and lens of how I come to this space before I get into the work of what we're actually doing at the AMA is the more kind of global context still focused here in the United States of colonization and the impacts of that and how that spurred systems and developed systems of supremacy, sustained systems of supremacy within this country as well and capitalism that ultimately has tremendous downstream effects on communities and individuals creating trauma and ultimately inflammation, ultimately bad health outcomes. And so there's a, and this is by Dr. Rupa, she's just released along with Raj Patel, the book In Flamed, which is amazing from my perspective. It's a fantastic book presented very differently and beautifully written. And I highly recommend folks to kind of check it out to just start developing an even broader lens about structural racism and the roots of, from the point of colonization and white supremacy and supremacy in this country. So we know during this time of COVID, that COVID propelled and accelerated several opportunities in this country. One, it was our digital transformation. I saw this slide and I said it was going to pull it because the context I could use. And so we saw the acceleration of telemedicine and where it was moving slow and now we're able to use it and do it and amplify more. Not that it's ultimately optimum and perfect, but we are in that space much more heavily. And then COVID comes around, exposes these inequities all across the country. As we all know, many of us who have been doing this work have been very well aware of it for generations. And COVID provided this public opportunity to really talk about it more and we saw more engagement amongst many institutions, governments, individual people, my mother talking about inequities and saying these words. And then racism even more explicitly as well. But I would say it really was the public murder of George Floyd that propelled our nation into talking about, sorry, talking about structural racism and making pledges across the country that we were going to do more, we're going to do better. And it's a monumental time that we absolutely have to take advantage of an opportunity because just from working in government and others who have worked in government knows that sometimes these doors and these windows of opportunity close. We're in a different time, I think now in this conversation around undoing structural racism. However, we know resistance is definitely strong inside and outside of our institutions. And so when COVID first came around, there were narratives about who was getting COVID and who was dying. The narrative predominantly was about these underlying conditions from these diseases, but really as many of us in health equity called out that just wasn't really sufficient enough. And that we had to look again at how was opportunity or how was health being structured? How were people being assigned value? And where were people able to live and not live? Did they have to go to work or not go to work? We had to really focus on the structures and the inequities that were built in our societies that really set people up to get and die from COVID more and still exist to this point. And so there was much more activity and action around reframing the narrative, especially around race. Wasn't the actual risk factor that racism is and that we needed to look at these systems? But what's really clear in the space of medicine is that we don't know enough yet about the impacts of racism on health. And there are absolutely more and more studies. We know certain journals are publishing way more on this, but the reality is that our leading medical journals have not published much. And so myself, Dr. Ria Boyd, Dr. Nancy Krieger, Dr. Fernando Demayo, did some investigation shortly after the incident with JAMA earlier this year on what was really happening around the publication of racism and the use of racism really as even just a word and including it, but also in publishing empirical data. And we realized or in studies and we realized the numbers were just extremely limited. And if that's the case, then we have a tremendous gap in what we know and what we understand and what we can act upon in the space of medicine. And I think this is a big area of which we all have an opportunity to have greater contribution to what is submitted. And this is not to say there are many people who have tried to submit papers that are inclusive of racism and papers that are empirical studies, but oftentimes they've been rejected. And so there's a method of accountability that's gonna be really important, but I think there's also opportunities in terms of contributions and what we're looking at and what we're putting forth to be published in peer review. And so as an AMA, we became other action and activities. We became really committed after the publication of George Floyd, especially. We issued a pledge, which is in the black box here, but then the House of Delegates actually passed policies that were really pretty forward-leaning from my perspective. And one was racism as a public health threat. Many folks did that one. Riding our healthcare system of racial essentialism, really just recognizing that race is a social and not a biological construct at all. And then supporting the elimination of race as a proxy for ancestry, genetics, biology, and medical education or biology overall, research and clinical practice. And this is not to say, because I hear this get conflated, that we're not recommending not using race as a category to collect data, race and ethnicity, just not to be used as a proxy. It is important that we understand what is happening across multiple identities and race and ethnicity being one of them. But more so to the previous point, we need to understand how are the isms and the systems of inequities and structures of inequities impacting health outcomes. So the question is, for all of us, how are we moving beyond our declarative statements and really moving towards anti-racist praxis? These are two other publications, one in health affairs and one in the Lancet this past year. One of the key areas and I think really frontline opportunities in this work is, how are we challenging dominant narratives and malignant narratives? There's a great resource by Nature, the National Association of City and County Health Officials, on looking at advancing health equity by looking at narrative. It's an excellent document. It's online. But if we don't get to this point of really unearthing and naming and having the courage to name where these dominant narratives show up, or if we don't develop the critical consciousness to see how these narratives show up in our day-to-day ways of doing work, whether it's around data or program or policy or advocacy or communications or publishing, we're going to still be in this space of really not unearthing the root causes, not unearthing who's responsible and who needs to be responsible or accountable and really obscuring power. And we know that these narratives, these malignant narratives can undermine health equity. And here's just a few of them. Kamara Jones also talks a lot about this. But I've seen this a lot in just my experience and I'm sure many who are listening have as well. We hear them and see them on media. But this context of inequities being unfortunate and unjust, and that folks, this context of individualism, they're self-determining individuals are making right and wrong lifestyle choices. That's been a lot of the narrative around healthcare is about lifestyle, lifestyle. And yes, lifestyle is important, but we have to contextualize lifestyle. How do people making choices? What do they have around them? Well, how much power do they have? What are the opportunities, the resources and the conditions? Again, all of these things are really important and our extreme focus on individualism versus collectivism inhibits us from really understanding and seeing that we're beyond ourselves as just individuals. The othering and really blaming other people and blaming people for their poor health outcomes. That is the narrative that I presented in the beginning in terms of blacks and Latinx and indigenous peoples having higher rates of disease, what's really coming across is that people are being blamed for that and not looking at this broader context of what sets people up to have to make the decisions that they have. And then our myth around meritocracy, also really critical and kind of undermining equity that this thought that we can just pick ourselves by our brew straps, work as hard as possible, and we're going to be able to achieve the same health and wealth that people who do have lots of wealth do, because we know there are folks who work multiple jobs, work really hard and are not able to acquire the same assets that other communities have that are able to pass down generational wealth and especially along the lines of racism. We know the wealth gaps between blacks and whites and just blacks, I mean whites and people of color are tremendous in this country. And then the last myth of the hierarchy of human value based on skin color, the false notion of that with white being at top. So I feel our roles in doing this work at the institutional level, and I've evolved over time in how I talk in the language that I use because when you start to really dig through the systems and the people there, I really believe we have to be very direct and very explicit and really name what is actually happening. And so we have to call out these systems of supremacy and oppression and we have to make them visible and not just mere descriptive disparities or inequities, but looking at again the systems and structures, the isms, figuring that out is really critical. And then also because at the institutional level, there's always and then at the kind of I guess societal level there are always a few that are in power. And how do we support them doing differently to act beyond their own self-interest, their own comfort and their own experiences to really care? And Drs. David Satcher talks a lot about what does it mean to really care? About doing this work of equity. And that's what this work of equity and undoing structural racism requires, people who deeply care and deeply value all lives. And so these are the lenses that we pulled together as we did the strategic plan at the AMA to embed racial justice and advance health equity. It's a long document of 80 pages, which is not typical for a strategic plan. However, what we were very clear on was that folks, especially in the healthcare sector, from very different points in understanding of these concepts and terms and having an analysis that's shared around equity and racism. And so we needed to provide a primer before we even got to one of our strategies to help support people to get us on the same page. And so our core strategic approaches in five are embedding equity into the practice process, innovation of what we're doing, building alliances and sharing power via meaningful engagement, ensuring equity and innovation, pushing upstream, address all determinants of health. And then lastly, again, understanding there's a historical context, how are we fostering and creating pathways for truth reconciliation and healing for our AMA past? Our theories of change are really rooted in these on the left-hand side, writing the injustice of our past, challenging those malignant and dominant narratives, centering the voices, ideas and experience of those marginalized, adopting a race and approach or anti-racist and intersectional approach. We lead with racism because we know what the data demonstrates and then also with the greatest gaps, some of the greatest gaps in this country. But also it's the area that people tend to be most uncomfortable to talk about. And if they had an option from the most part, they would put that race and racism to the side and talk about some of the other identities that we know are also oppressed. So to make sure it stays front and center and doesn't get kicked off of the agenda, it's a race and approach, not to the exclusion of other identities such as sexual orientation and gender and ability, immigration status. It's an and approach as I'm expressing. And then embracing public health frameworks, which I talked about at the top, and then implementing an inside out side strategy. We can't do this work of equity outside of an organization with other organizations unless we look internally for our own organization to do something differently and to do better and to challenge our own mental models, to challenge our staff mental models so that we change up how we do our work. We change up culture. We change up how well people feel like they belong and their voices and ideas are valued. And I think that's really critical on core to innovation, which is such an important word in the space of healthcare that if we truly want to be innovative, we need to be equitable. So embedding the first approach, just a little bit more specificity around that and what we're doing is building AMAs capacity to really understand and operational anti-racism work. So there are trainings in that training. Clearly it's not enough, but you have to have training. You have to have knowledge-based training and skill-based training. And you have to have tools to help folks with accountability and reminders. And it's important that you're working across all aspects of the institution and the organization. Important to have a trauma-informed lens. That's my biggest learning lesson at the New York City Department of Health is that we didn't do enough to really support this transition of culture to help support our staff and our employees through these conflicting times, potentially, in our institution. So you have to have the sports. You have to create spaces for psychological safety in your institution to do this well and to have some sustainability around it. We are using a model for the Government Alliance for Racial Equity. It's what we use at the New York City Department of Health. There are hundreds of government agencies that are using this model of normalizing. So building a shared analysis. A lot of the training aspects are underneath this. Organizing for really accountability within your institution and making sure that everyone is engaged. And then operationalizing, making sure that there are tools that are challenging folks' mental models again on a day-to-day basis, how we're collecting data and metrics, and then sharing that also. And then what we did as wrapped around was this trauma-informed and healing supports as well. So for us, this has been a very collective opportunity and effort with our staff. We've done assessments, which are usually done when you do organizational change work to get quantitative data and qualitative data on the experiences of employees. This is a collective vision statement that we made and put forward and we added equity to our values. This is from the Racial Equity Toolkit on an opportunity to operationalize equity. Highly recommend you all to check this out. Really a set of questions that you ask yourselves at any aspect of whether it's program, research, policy, advocacy, that provides a framework of how to structure and to ensure that you're considering equity. Equity and even more specifically racism. And then the reality is that there's a huge gap, especially amongst physicians because of how physicians have been trained in really just basic and clinical sciences, not those upstream contents and that we know the 80% of what creates health. And so we're working a lot to support their education. We just launched a health education center, Health Equity Education Center on our AMA educational hub. And we are partnering with many entities to put content in that we think is really quality content that may already be created. And we're also doing some new content that we're creating as well. So that's up and available if you all want to take a look at it. And then we're finding ways to foster the last part is fostering pathways for truth reconciliation and healing. I don't feel you can do this work without looking at your own institution's past and acknowledging that harm, acknowledging it publicly within our strategic plan. There's a list of harms, but that's only the beginning in terms of from AMA. And what our goal is and what we're going to try to get to is making sure we're quantifying the effects of our policies and decisions that excluded and harmed and finding ways to repair. This I think many folks would say is the front line of doing anti-racism work at this particular time. And just an example and start, one was an apology that we issued in 2008 for the exclusion of Black physicians from the AMA. And then this past year in February, our CEO Jim Medera read a few pieces and after reading those pieces and seeing that our father of the AMA Nathan Davis was really very much engaged in explicitly excluding Black women and Black physicians, removed the bust of Nathan Davis from public view at the AMA headquarters and put it in the archives. And so there's much work more that we're doing and engaged in. A lot of it's listed in our strategic plan. But I think the important point at this time is to continue to speak up, continue to kind of work collectively. We have an opportunity to learn from one another and I really look forward to listening to the others on the panel today. Thank you. I wanted to see if there were any questions for Dr. Maybank. What we've decided to do for this session is to have folks post their questions in the Q&A box and then the speakers will take their own questions. Okay, I guess there's no questions. I think there's a lot there for folks to digest. Dr. Maybank, I wanted to thank you for all of the work you've been doing to foreground a racial equity lens in your work as the senior vice president and chief health equity officer at the American Medical Association and for getting our symposium off to a terrific start because you've articulated so many of the reasons or so many of the motivations for having this particular session at all. Thank you. Can I ask... I mean, it's not a question but I just want to echo the comment that that was truly wonderful and I think you gave us a lot to think about and a lot of... We know we're symposium and a lot of the things that we're doing in age I think tie-on some of what you're talking about as well and so I look forward to further conversations and thinking about where different organizations can learn but I thought that was just... That was phenomenal. So thank you so much, especially with your tired voice. I know I'm sorry, I'm sorry. Thank you about that. Thanks, Dr. Gibbs. I really appreciate that and you and I will be talking more because I'm now helping support CDC as well so I think there's a lot of opportunity to again work together and to collaborate so I really wanted... I'm really excited that we're on the same presentation because I really was trying to reach to get to you to learn more about what you're doing. Okay, well hang out. After Dr. Lee's talk, we can thank... All right. Excellent. I'm so glad to see folks getting connected and people seeing ties between different sessions. I will go ahead and take my turn presenting here and see. Do you all see me? This is so awkward. I'll be talking about black, white disparities in NIH grant review scores and I'll be presenting a couple of different projects. The first is in collaboration with a UW team. Dr. Elena Orocheva, who's a professor at the Center for Statistics and the Social Sciences and Departments of Statistics and Social Work. Sheridan Grant, who just completed his PhD in the Department of Statistics. And then the other part of the team comes from the Center for Scientific Review at the National Institutes of Health. So Richard Nakamura, who's the now retired former director of CSR. Mark Lindner, who's director of the Office of Planning, Analysis and Evaluation there. And Meiqing Chen, who's the Informatics Team Lead and Data Scientist at CSR. To disclose my conflicts of interest on this project and some funding, Nakamura, Lindner and Chen either worked or worked at NIH. The UW team worked under NIH contracts. The material is also supported by an NSF grant from the CICIP program. And the views expressed in this paper are those of the authors and not those of CSR, NIH, HHS or NSF. So I'm going to go much more quickly than some would like. If you want more details, feel free to ask questions but you can also find more details here. Now, the symposium is about structural racism in biomedicine. It's about the policies, practices and systems that reinforce an unequal distribution of power and resources in social institutions. Ten years ago, Donna Kinther and her collaborators found that NIH R1 grant award rates for Black PIs was 55% that of White PIs. Since then, we've learned more about how policies, practices and systems contribute to these unequal funding rates. So part of the funding gap can be explained by differences in funding rates across centers and institutes. So what I have here is a figure from Mike Lauer's blog, Open Mic. And what you see here are the percentage rates, the funding rates at the different institutes and centers. They range from 9.1% all the way to 26.9%. Black PIs are disproportionately represented in some of these centers and institutes. So I've added black triangles to mark which ones they are disproportionately represented in. And the leftmost five of these belong to centers and institutes that are funded at lower levels than the average funding level at NIH. Strikingly, minority health and health disparities had the highest representation of black PIs of all the institutes and centers. It had the lowest funding rate of all the institutes and centers. And if you look at the total dollar numbers appropriated, they had the lowest amount. They pulled in about just under 1% of the total funding going to the institutes and centers. Now part of the funding gap can also be explained by disparities and inequities in other parts of science's reward system. So this work by Donoghinder and her colleagues shows us that when you compare bibliometric features of articles disclosed in biosketches submitted by PIs, the articles described there are published in papers, sorry, in journals that have lower journal impact factors. They have lower field adjusted citations and there are just fewer publications listed altogether. And this is the case when you compare new and experienced black PIs against new and experienced white PIs. Now the project I'll describe here focuses on disparities and scores and you might wonder why scores. In Ginther's original work, they found that applications with good scores were more likely to be funded regardless of race or ethnicity. You might also wonder which score should you look at? And to get a sense of this, I need to describe how the peer review process at NIH works. So in 2009, NIH introduced what they call the enhanced peer review process which requires that individual reviewers provide scores not only for overall impact but also for separate criteria for significance, investigators, innovation approach and environment. When reviewers are given these proposals in the preliminary round of scoring, they haven't yet spoken to folks on a panel. They're just providing their own independent assessment of a proposal. Now, in order to decide which proposals move on to the next phase, overall impact scores are averaged. About 55% of the applications move forward to that next stage of panel review or what NIH calls the scientific review group meeting. And at that stage, after discussion, reviewers are asked to provide an overall score. And if that is different from their preliminary score, they're asked to update their scores for significance, investigator, innovation, approach and environment. At that point, overall impact scores, the final overall impact scores are aggregated, percentiles are calculated, and this information is passed along to institutes and centers which decide whether to fund proposals or not fund them. They use information about the peer review feedback, but they also use information that they have about their own sort of programmatic priorities. Now, in previous work, Hopi et al. found that the decision point that makes the single largest contribution to the funding gap is the stage of deciding which proposals move from the initial phase of preliminary scoring to the second phase of panel discussion. And the preliminary scoring is where we focus our attention now. So we focused on three questions in particular. The first question was whether or not we still see funding disparities in our dataset. And the reason why this question makes sense is because, again, there's original paper. They studied proposals submitted before NIH switched to the enhanced peer review process. The second question is actually the hypothesis of our project. So do we, we hypothesize that we would see that Black-white disparities in preliminary overall impact scores could be explained by disparities and how reviewers weighed criterion scores when deriving their overall impact scores. This hypothesis was motivated by research and psychology demonstrating that evaluators prioritize whatever criterion gives preference to individuals belonging to preferred social groups. And NIH seems like a wonderful context to evaluate this question because in their instructions to reviewers, they tell reviewers explicitly to weigh the criteria as they see fit and that a proposal need not be strong on all criteria in order to be deemed as having high scientific impact. Now, to test this hypothesis, we used an interaction model and in statistics, it's common practice to run to fit a main effects model before doing the interaction model. And so along the way, we were able to evaluate the question of whether disparities in preliminary overall impact scores could be explained by differences in preliminary criterion scores. The data we used included applications submitted by Black and White PIs in fiscal years 2014 to 2016 and race in this case is self-reported and individuals had racial categories to choose from that were determined by the Office of Management and Budget. The Office of Management and Budget set standards for how racial and ethnic information is collected and presented in all federal reporting. Reviewers don't see the self-reported race. That information is set aside separately. However, they do see applicant names and they see other potentially racialized cues in the biosketches. We employed a special case of course and exact matching. We matched exactly on eight key variables, thought to be related to scores and award rates and had complete coarsening on the rest of the variables and matching improved balance on all matching variables and on most other covariates making the analysis more robust and less susceptible to model mis-specification. So here are the variables we matched on. We had some applications, applicant specific variables. We had gender, ethnicity, so Hispanic, Latino or not. Career stage, early stage, experienced or non-early stage new investigator. Degree type, so PhD, MD, MD, PhD or other, and NIH funding bin. This is something that's been used in other projects studying NIH grant review. Basically, it's a rough, very rough proxy for prestige. Basically what you do is take all the institutions that PIs are employed at. You look at the total amount of NIH funding that those institutions receive and you divide those institutions into five bins. We also matched on application characteristics. So in particular, whether the application was a new or application or a renewal. We looked at the amended status, so whether it's an original submission or revision. And we looked at IRG or the integrated review group. An integrated review group was meant to serve as a rough proxy for area of science. I think it's a little easier to see what the IRG is when you look at the multi-level review structure at NIH. So here what you have here are two applicants, applicant outlined in red and then applicant outlined in yellow. The red applicant has two applications that can go to different SRGs that go up and the SRGs themselves are nested within IRGs. IRGs are themselves nested under administering institutes or centers. And these thick blue lines are meant to demarcate structural relationships between different parts of the NIH system. You can also see at the bottom that reviewers can evaluate multiple applications and they can review applications in different SRGs, IRGs, and institutes and centers. So we matched applications from black PIs to white PIs at a ratio of one to three. In the matching process, we went from having 500 unique black PIs who submitted 1,015 applications to having 456 unique black PIs with 890 applications. Those applications received 2,578 reviews, which were written by 2,084 reviewers. The matched white population included 1,497 unique PIs who submitted 1,676 applications, which had 4,893 reviews reviewed by 3,866 unique reviewers. The funding rates that we discovered using this matching process, sorry, let me back up. The funding process, the funding rates we saw when you took all the black PIs and a population randomly selected white PIs was about 56%. So black PIs had a 56% funding for applications from black PIs is 56% that of applications from random white PIs. This is comparable to what has been found elsewhere. So Hopi and I published some work with similar funding rates and we ran our analysis before they published their work, not knowing that that piece of it would be done elsewhere. But what we were also able to do using our matching process was to compare matched black PIs to matched white PIs. And we discovered that the funding, there still remains a funding gap. Funding for applications from matched black PIs is 75% that of applications from matched white PIs. So 11.57% versus 15.39%. Now for the multi-level modeling of review scores, we relied on the NIH review structure here. You can see rectangles in the figure are specified as fixed effects. Ellipses are specified as random effects. In particular, we let Y be the preliminary overall impact score for the I-th review of the J application from the K-th PI, reviewed by the L-th reviewer and the M-th SRG. The key terms here are the race indicator variable, the vector of model specific control variable. So we had different models using different covariates and I'll walk you through those in a little bit. We also had random intercepts for PI, reviewer and SRG. And the within application reviewer, review error term, plus the usual normality and independence assumptions on the error terms. So we ran four different models and the main effects part of the project. This table shows racial disparity and preliminary overall impact scores as a function of different sets of covariates. Now to give you a sense of how to interpret these numbers, it's helpful to know that preliminary overall impact scores are scored from one to nine where lower is better. And to give you a sense of how big these numbers are, it's helpful to know that a difference of 0.3 is substantial near the funding cutoff and the sense that it could move an application from the 15th percentile to the 20th percentile or the 12th percentile. Now in the first model, what we did was use as our covariates what we take to be structural variables. Oh excuse me, I might just come right up. So included integrated review group, scientific review group, institute or center, application ID, applicant ID and reviewer ID. And you can see from this number here of 0.66 that this is a substantial difference. Black PIs have a higher preliminary overall impact score and that this is a substantial amount. The degree of statistical significance is set here in a way where P is less than 0.005. In the second model, we added applicant specific variables, so gender, ethnicity, career stage, terminal degree year, NIH funding history, geographic location, NIH funding bin, institution sector, graduate institution and minority serving institution. And we also added application specific variables. So application type, solicitation type, amended status, multiple PIs, costs requested, support years requested, council year, review group type, as well as human, animal, child, gender and minority subject codes. And you can see that when we add these variables to the model, the disparity and preliminary overall impact scores is still substantial practically speaking as well as statistically significant. Now, in the next model, we used the structural variables, but we also used, we focused on criterion scores. So scores for significance, investigator, innovation, approach and environment. And what's really interesting here is that the disparity and preliminary overall impact score decreases by quite a bit and is no longer statistically significant. And in the next model, we added, we had structural variables, criterion scores, as well as the application specific and applicant specific variables. And what you can see here is that when you add the application and applicant specific variables, the disparity doesn't change much from what we see in model three. Now, controlling for criterion scores in models three and four decreases the residual error significantly, but you'll see that it doesn't eliminate it entirely. And overall, what we see, the picture emerging that we see here is that disparities in preliminary criterion scores account for disparities in the preliminary overall impact scores. Okay. Now, to get a sense of what was going on here, we created frequency histograms for the five preliminary criterion scores. And what you can see here in purple are the scores for applications from matched black PIs. And you can see for every single criterion, as well as overall impact, that they are skewed to the right, which means that the scores are worse. The light yellow refers to scores for applications for matched white PIs. And you can see for each of these that they're skewed left, which means that they're scoring better. And this orange-ish color indicates the overlap in scores for matched black and matched white PIs. Okay. So onto the main hypothesis of the paper. Do we see differences in how reviewers weigh criterion scores? And does that difference account for disparities in the preliminary overall impact scores? What we did to measure this was use the main effects model and add this interaction term, which gives us this commensuration coefficient. And what we discovered, much to our surprise, was that we don't really see very much difference in how reviewers weigh criterion scores. There is a black PIs are slightly disadvantaged or penalized for their approach scores compared to white PIs. But you see that even though this result here is statistically significant, the total amount is quite small relative to the .3 number we described earlier being kind of a more substantial amount. Okay. So to recap, do we see funding disparities? We do continue to see funding disparities. This was surprising and interesting to us because when NIH moved to the enhanced peer review process and required that reviewers provide individual scores for each criteria, what they did was move to a system recommended by psychologists. They recommend scoring applications on pre-specified criteria as a way to focus attention on merit-related aspects of an application. And the hope is that that might decrease social bias, but we don't see the funding gap disappear. So what this suggests is that introducing criterion scoring by itself is not sufficient to close the funding gap. As for our main hypothesis, we found that there were practically negligible differences in how reviewers weighed criterion scores and deriving their overall impact scores. And instead, we found that the disparities and the preliminary overall impact scores could be explained by differences in preliminary criterion scores. These disparities in the preliminary criterion scores fully account for disparities in the preliminary overall impact scores. There are racial disparities and scores for Black PIs and all five criteria. And this suggests that the preliminary criterion scores absorb rather than mitigate racial disparities and preliminary overall impact scores. Now, given these disparities in funding and scoring at NIH, it's really natural to ask if there might be alternative ways of distributing grant funding that are more equitable. And in this piece, my colleagues and I just voiced some cautionary notes about a couple of these. So a really popular way of thinking about how to allocate funding is funding by partial lottery. In the system, applications undergo an initial round of review. The weakest applications are culled. And then the proposals that move forward get a lottery ticket. Some have hoped that this kind of approach would eliminate or at least alleviate bias. And what I want to be sure that folks keep in mind is that there may be disparities in that initial round of review. We find the racial disparities in all preliminary round scores. And I recall Hoppe and All's finding that that preliminary round of that preliminary round that filters out the weakest proposals is actually the decision point that makes the largest single contribution to the funding gap at NIH. So we just have to be careful about how we implement something like a partial lottery. Now, another approach that has been suggested is funding by PQRST. So fund researchers whose projects are productive, high quality, reproducible, shareable and translatable. You might wonder, how do we measure productivity and high quality? The suggestion is that we could take high publication productivity for previously funded projects as one possible metric. We could also look at top-sided papers as a metric for high quality. We have to remember Ginther and her colleagues finding that BlackPI is both new and experienced ones, are publishing papers with lower field adjusted citations and have fewer publications to report. So this shows us that tying funding to productivity and citation rates could just perpetuate inequities in other parts of science's reward system. So to close, I just want to say that when it comes to structural racism, this is going to be a continuous process. As we conduct more research that reveals new findings about social disparities in science's reward systems, we have to continue to reevaluate any process we use to allocate funding or other scarce resources in science. At the very least, this includes continuing to study grant review and funding, as well as journal review, publication and citations. But there are also so many other aspects of the scientific review process, the scientific sort of incentive process and reward system to look at, that this is just a tiny, tiny sampling of other areas. I see a number of questions and I'm really sorry. Oh, great. These were for Dr. Maybank. Perfect. Okay. I promised to hand over the session to Dr. Gibbs. I'm two minutes late. Sorry, Dr. Gibbs. If you have any questions, please find me in the remote in the 30 minutes that follows the symposium. Thank you, Dr. Gibbs. And thank you, Dr. Maybank. If you could stop sharing, that'd be good. And maybe while I'm getting my sharing together, I was wondering, is it right to think of it as maybe a death by a thousand cuts? Because you said all the criteria score have these issues. And so I was just trying to process your, is that an accurate way of thinking about how Black applicants fare in the peer review process? Yeah. I wish that we had more information to explain why Black applicants were receiving these lower scores for all criteria. When I listen to Dr. Maybank's talk and I look at her work, I can't help but wonder if certain kinds of work isn't considered as important. Considered as rigorous. And I think that these are areas that require a lot more research and study. Great. Can you confirm that y'all see my slides in the full view, or do you see the presenter view? That's neat to make sure. I see the presenter view. Okay. So we're going to swap. Do we all see the real view now? Anybody? Yes. Awesome. Thank you. So I'm so pleased to be invited to speak with my esteemed colleagues. My name is Kenny. I work at the National Institutes of Health in the United States, and particularly the National Institute of General Medical Sciences. We're the basic science part of NIH. And there I lead some of our training workforce diversity, our workforce development diversity programs within our division, particularly folks on undergrad and pre-doctoral training, but many other things as well that I will talk about. And so I think a lot of what I'm going to talk about builds on what the first two speakers talked about. And just to give you a bit of outline for the talk, I'll do a really quick bit of definitions and data. I think Dr. Maybank did a wonderful job there. I'd just like to make sure, especially if you came on later in the session, that we're all speaking the same language. I'm going to talk about some approaches that my own institute has taken, and then a new trans-NIH initiative to really tackle structural racism. And then finally, just some thoughts of my own as a black man in science, been doing this for a couple of decades on how we actually advance racial equity. And so as it relates to definitions and data, again, I think Dr. Maybank did a great job. When we talk about diversity, we're really talking about, in my mind, numerical representation in the biomedical sciences. Importantly, diversity is a property of groups and not of individuals. And so as I say regularly, I'm a black man, I'm not a diverse individual. I have diverse interests, but being black doesn't make me diverse. And importantly, diversity is not the same as underrepresentation, or it's not really the same as justice. So you can have a diverse group, but not really be doing things to move forward the cause of justice. Inclusion or belonging, we start thinking about the felt experience within the biomedical research enterprise and having environments where individuals from all backgrounds are welcome, are truly welcome, and feel that they are welcome and integrated and supported by the community. And then equity means fairness, right? And we talk about, let's see, equity and justice thinking about treatment, what are your access to resources and outcomes, right? And so fairness of opportunities and outcomes is not necessarily the same as equality. And sometimes we have to move differently to achieve fair outcomes. NIH has done that in the context of early stage investigators. And so those are investigators who are within their first 10 years of PhD. We've seen that there is a reproducible disparity no matter what. So NIH made a policy decision to remedy that by saying we're going to achieve equity, to achieve a fair opportunity for them. We're going to prioritize these applications differently. And so those are examples of how we think about some of these terms. When we think about broadly, this is some data from our data office and Hannah Valentine, the former chief officer for Scientific Workforce Diversity published this a few years back, looking at representation across the biomedical career pathway. And so at the top, we have Associate's Degree granting down to full professor at the bottom. Orange bars represent women. The dark orange are women from well represented, being wider Asian backgrounds. And the lighter are women from underrepresented background, Hispanic or Latina, Latino, Latinx, American, African American and Black, Native American, Alaska Native and Pacific Islander. And then the green bars are men, again, lighter being underrepresented, darker being lower represented. What you see is that as you go through the pathway, the green becomes more prominent and all the other bars become less prominent. But importantly, if you focus on the doctoral level, we actually do have more than 60% of PhDs or roughly 60% go to groups that are quote unquote underrepresented, be they women from all racial ethnic backgrounds, or about 10% from historically underrepresented racial and ethnic groups. I just want to call that out. Sometimes we so often problematize these issues. But we don't call forth the progress that we've made. So I'm going to speak a little bit about what we've done at NIGMS. Again, to reiterate NIGMS is one of 27 different institutes and centers at the National Institutes of Health. Our focus is on basic research that increases our understanding of biological processes, and that lays the foundation for advances in disease diagnosis, prevention, and treatment. Unlike many of the other institutes, we aren't focused necessarily on diseases, body parts, or life stages. And so for that reason, we're sometimes called the NSF of NIH. Additionally, though, two other components of our mission are to provide leadership in training the next generation of scientists, enhancing diversity in scientific workforce, and in developing research capacity throughout the country, focus on both institutions and regions that have less success at NIH funding. To put that into context, that means that we fund about 1,000 research, training, and diversity enhancing awards at about 300 institutions per year in all 50 states and territories, and about a clip of about $340 million per year. That operation lies into a bunch of different programs that I will not go over. The point is to say that there are a lot of efforts there. And so sometimes when we have conversations about diversity, which can sometimes be part of conversations about structural racism, we say, well, we're not doing a lot, and we actually are doing a lot at all stages here. I just want to call that out. One thing that I want to point to is that some of these programs focus on individuals, but a lot of them, for example, all the programs up in the blue bars at the top, URISE, Mark, GRISE, IMSD, bridges, and even these big things called T32 and RSAs, focus on institutions. And so what we're doing is giving money to institutions to then identify students so they can train, participate in various stages of the biomedical research creative development pathway. Recently, under the leadership of our director, Dr. Alison Gamme, vision director, Dr. Alison Gamme, we've worked to really think about how we can leverage the resources of these institutional training grants to drive institutional change, which echoes some of what Dr. Maybanks mentioned earlier. And so now any of our institutional programs, whether they have a quote, unquote, focus on enhancing diversity or their quote, unquote, general training programs are really expected to do the things that are important to help students gain skills, be they technical, operational, professional skills, and to enhance diversity. So any program, no matter where you are, whether you're a biophysics program at a major research institution, or undergrad training program at a historically black college at university, you're all able to contribute to diversity and skill building. So we asked them to do a number of different things. I'm not going to read all the bullets, but I highlighted in three, I highlighted three that are relevant to this talk. One, they're expected to promote diversity, equity, inclusion, and all aspects of the research training environment. That doesn't just mean having students from underrepresented groups, but also making sure they have access to role models and faculty from a number of different backgrounds. How do we encourage inclusive, safe, and supportive research environments? And how do we build cohorts and community where trainees can think of themselves as scientists and to be reinforces that? We also know the importance of mentoring, which again is individualized, not necessarily structural, but we are putting in a structure to oversee mentoring because we know that there can be also tour mentors in the lab, so to speak. And we also require all of our programs to include at least three pages on how they're going to a detailed plan on how they're going to have recruitment to enhance diversity, how they're going to focus on retention once the students are there, and then again mentor training, assessment, and accountability. And all of these are manners that we can use to drive institutional behavior change, which can make the context in which students from all backgrounds, especially those from underrepresented groups, are better to able thrive within the research environment. So we've had a number of these programs for a lot of years. I actually participate in them, some of them when I was a college student, but one thing that we have not seen a difference in this really piggybacks on what Dr. Maybank said about Black physicians, minority physicians, and Dr. Lee speaking about researchers is really what's happening at the faculty level. And so we focused on the last few years on really thinking about how we can make some additional focused approaches to enhance faculty diversity. I think most of the infrastructure, the diversity infrastructure that came to be in the United States was born out of this post 1960s and early 70s civil rights moment where there was clear legal exclusion. And the idea was that, hey, let's create opportunities early, because once we create opportunities early and people earn their credentials and the system will take, you know, then they'll be established in the system. We won't need any additional interventions. That clearly is not the case. And so this is an example of what that looks like. So these are some data that I published a few years ago in E-Life and that's been updated by our Division of Data Integration, Modeling Analytics. What you see again in gray are the number of PhD graduates from historically underrepresented groups, which I mentioned before, and Black are the number of assistant professors at Association of American Medical Colleges, basic science departments where most of them are PhDs. And so what you see is that, you know, in the 1980s, early 1980s, they actually were very few fewer than 100 per year graduating in the biomedical research sciences. And we're about 130 assistant professors. But you see basically an exponential growth over the last 30 years driven in large part by institutional investments as well as investments by NIH and other agencies. We don't see that same sort of rapid growth for the professorial positions that you go from 130 to 556. And if you normalize to where you are, again, you have 12-fold increase in PhD attainment, but about four-fold increase. So there's a disparity even there, right, even if you're trying to move, as I like to say, from PhD to JOB or job and academic job specifically. And so what we've done recently is develop a program to really target that goal. And I'll speak a little bit more about that. But that program is called Mosaic or the maximizing opportunities for scientific and academic independent careers program. The goal of that is really designed to facilitate the transition of promising postdoctoral researchers from diverse backgrounds, including those from underrepresented groups into independent research-intensive faculty positions, right? There's been a lot of conversation about career diversification, and we fully support that. And one of the bullets on the T32, on the institutional requirement side mentioned career diversity. At the same time, within career diversity, we know that scholars from underrepresented groups that have the skills and the desire to move to faculty positions and research intensive faculty positions are not making that transition. And so the overarching goal this program is to enhance the diversity of independent investigators conducting research within the NIH mission with the priority of dealing with document and underrepresentation at the faculty level. So this uses what's called a K99R00 mechanism. For those of you who aren't familiar, the K99 provides two years of mentor career development for postdocs. And then once they get an independent position, three years to fund their independent research, right? And so these are really successful awards at transitioning postdocs independent positions. Over 90% of them transition. At the same time, historically, very few scholars from underrepresented groups actually got these awards. And the ones that did transition at a rate about two-thirds of that, like black scholars, transitioned about two-thirds of the rate of white scholars. So even with all this, we were seeing disparity. What this program was designed was to say, hey, let's focus earlier. Let's broaden how we think about what a potential contributor is. And importantly, we said, hey, we know that many scholars of color, many underrepresented scholars, are doing a lot of service work at their institutions, even during their training stages, right? I can speak for myself. I was on a number of different university committees, even as a graduate student. And so we want to say, hey, that's part of your contribution to the scientific environment. And so we actually evaluate that as part of this. And so then we select them. And then importantly, what we say is, okay, we know that individual support is not enough. So what we're doing is that we put them, in addition, as they have their individual awards, into cohorts that are housed in the scientific societies, the American Society for Biochemistry and Molecular Biology, the American Society for Cell Biology and Association for American Medical Colleges to have additional peer groups, mentoring, skills development, grant writing coaching, just some additional people to have your ear and have your back, right? So part of this program, in addition to broadening the criteria, has focused on how do we actually make sure we reach out to people from these backgrounds to actually get them to apply? Because as we see, we see disparate funding rates, we also see low application rates. And so what we've seen in the last year is that 75% of the people who've applied have been from underrepresented groups, primarily Black and Latin. 75% have been women from all racial ethnic backgrounds. And this is the first group of scholars that have been funded through this program. And it might not mean much to you, but if you were to, for example, look at 31 random K99 scholars funded through traditional mechanisms, two to three would be from an underrepresented group. And maybe eight would be women whereas here, three quarters are women and nearly all of them from underrepresented groups. And so they go through the same rigorous peer review process, but by having additional mechanisms focus on both the needs and specific contributions of different populations, you're able to actually recruit and support a broader variety of scholars. So just a plug, if you are in the US or know a postdoc from an underrepresented group, have them send me an email actually, I managed this program. So beyond individual programmatic initiatives, though, I think our colleagues have made the point about structures matter, right? And so some structures are creating new programs for individuals and their environments, but also really tackling structural racism. And so NIH like many organizations was moved forward in both recognizing and needing to really deal with how structures reify racial hierarchies, right? And so that has been, since March, our, you know, our director, Dr. Francis Collins announced this initiative to end or, you know, attack, address structural racism in the sciences called Unite. And so they've started with really acknowledging because you cannot fix a problem that you don't acknowledge exists. And so I started with this apology from him. So to those individuals in the biomedical research enterprise who have endured disadvantages due to structural racism, I'm truly sorry. And I just committed to instituting new ways to support diversity, equity and inclusion and identifying and dismantling any policies and practices at our own agency that may harm our workforce or our science, right? And so I think, you know, senior leadership has said they've arrived at a kind of new shared commitment to address structural racism, recognizing that, you know, the impacts of history on people in the systems of those people set up that mediate those same impacts in the presence in the present need to be dealt with. And that NIH is in a position to really influence and contribute to positive change and breaking down these systems. And so I just want to call that out as an initiative. And so what does that look like? So that is called UNITE. And again, it's based on the work of five different committees of the NIH staff representing a lot of different levels from Institute and Center Directors to people who are administrative professionals, thought of as more support roles that aren't that are outside of science, outside of kind of the research space. But, you know, one committee focuses on understanding stakeholder experiences through listening and learning. And so you might have seen request for information that came out earlier this year that they're actively analyzing. Really, and this gets to Dr. Lee's comments, thinking about new research on health disparities, minority health and health equity. We know that research in these areas is poorly funded and scores less well during NIH peer review and the Institute that focus on that has had the lowest budget. Congress this year has requested there's been a request rather in the president's budget to increase the budget for the minority Institute, minority health and health disparities Institute. And we will see what is appropriated through Congress. One focus will focus on NIH as a workplace, right? So improving the internal NIH culture and the structure for equity, inclusion and excellence, right? I work at the NIH, it's a great place to work in because NIH is part of America, we have our problems too. And so really thinking about how to address those issues, how do we have transparency and communication and accountability for both those who work at NIH and those who are funded by and rely on the work that we do. And then finally, thinking about our extramural research ecosystem, those that we fund at institutions around the world around the country, thinking about how to be changed policy, culture and structure to promote workforce diversity and really, again, equity and opportunity, right? And so that's what this looks like. And I'll just give you a quick sense of what's been going on as we've been doing this for the last year. And so again, there are a lot of words here. I'll go through them a little by little. And so first, publicly comment, commit to identifying and correcting any policies and practices that perpetuate sexual racism. And that was done, that apology on in the, you know, formal rollout of this on March 1st. I'm the father of three small kids. I always think about Daniel Tiger, he says, saying, I'm sorry, it's the first step, but you got to do more, right? And so what have we done, right? So first is our leadership is thinking about aggressively implementing approaches to address this get their gap and to enhance portfolio diversity. This looks very clearly at builds off the work that Dr. Lee presented. And the doctor may make talked about in terms of how do we make sure that we are addressing it, not just studying it and saying, Oh, this is a problem was something that we have an our ability to change. Launching a multi phased and tiered integrated common fund initiative focused on transformative health disparities research initiatives to reduce health disparities and inequities. And so that is to say NIH is funded by primarily 27 different institutes and centers. The common fund is in the opposite of the director. And that says let's fund cross cutting trans NIH priorities. And what they've done is put out funding announcements for funding in this fiscal year on these minority health and health disparities issues, as well as setting Lynn the predicate for additional funding that will happen, I believe in fiscal year 2023. Additionally, what's happened this year is that we proud to request funding announcement NIH wide that was led by the minority health and health disparities Institute. And the FOA funding opportunity announcement focus on the effects of structural racism and discrimination on health disparities and health inequities. And so NIH puts out lots of funding announcements, but it's not common to see one that focuses again, calling defining and focusing explicitly on structural racism and discrimination. There are more than 25 institutes and centers who committed in offices or committed more than $30 million to fund awards through this mechanism. And again, that is an example of an action is leading out of this to say, okay, this is not the beginning and end, but we're going to it's not the end, but these are some ways that we can begin to really address some of the issues that we've seen. We're really thinking also about developing sustainable processes to systematically gather and make public the demographics of our internal and external workforce. Again, as a person who works at NIH, you can maybe see some of the differences that along lines of race that are reflected in broader society within our organization. I think it's also important for our external stakeholders because frequently those of us who are in program officer or review roles are the front lines of communicating with all you applicants. And so making sure that we know who is where and who is doing what and really understanding what are paths to advancing, what are paths to leadership, are those equitable. And so we're working with our officers to do that. Importantly, they've worked to start in the next fiscal year to develop, and again, fiscal year 22, a performance expectation of Institute and Center Directors that they're going to be accountable for equity, diversity, and inclusion efforts and to actively participate in NIH wide diversity efforts, either through having a chief officer in their institute or other appropriate means in coordination with various practices here. And so this is important. NIH is 27 different institutes and centers, Institute and Center Directors have significant influence on what happens. And so now part of the way they'll be evaluated is to really affirmatively address these issues. There are talks of having each institute come up with a racial equity plan to really understand what's happening and to address their local context. And so, and this is going to, because it comes from the director, and we focus on the director, it will have to necessarily cascade down to other parts of the organization. And then finally, and again, these are some of the initial actions, more is coming. So please stay tuned as expanding what's called our Distinguished Scholars Program. So that's an intramural, meaning NIH funds, three quarters of our money, actually 80% of our money goes extraordinarily, goes to institutions around the country to fund grants and training programs. About 10% of the money focuses on work that we do in-house. The most well-known of that, most recently is helping to develop the coronavirus vaccine by Dr. Kizmiki Okorbit and others at NIAID and Barney Graham at the National Institute for Allergy and Infectious Diseases. But we have an internal program that really focuses on building community with colleagues who have demonstrated and clear commitments to enhancing diversity in the research workforce. And those people disproportionately happen to be from underrepresented groups. And so that has focused and been deemed a success for those at the early stage. Now they're expanding it to make sure that senior investigators can also participate in that. These are some of our initial moves. Keep an eye out. Go to NIH Unite, put that in your favorite search engine and you will be able to see everything that's going on as a result. I want to, particularly in this audience, call out two opportunities funded through NIGMS where I work. We posted a notice of special interest back in March as well on, again, understanding and addressing the impact of structural racism and discrimination on both biomedical career progression and the broader biomedical research enterprise. So part of that is through a partnership that we have with the National Science Foundation called the Science of Science Policy Approach to Analyzing and Innovating in the Biomedical Research Enterprise, or SICIP Bio. And that release work was funded by SICIP. And so again, part of this collaboration as well as a research on interventions program. And that focuses really on understanding what are the interventions that are happening in training, mentoring, psychological factors, institutional factors, dealing with harassment, discrimination, boosting, training, confidence, and things like that. So those are two opportunities that NIGMS funds, and I put the name up there. I'm Kenny, and if you don't remember Dr. Blatch's name, just email me. I can forward your information to her. But we are interested in funding these applications. We're only developing a more actionable evidence base to move these issues forward. In the last few minutes, I'm going to just, I'll rift briefly as just sort of sharing a little more of my personal experience and views on promoting diversity and advancing racial equity in the biomedical sciences. And so this was born out of last summer's events. And after Dr. after, after George Floyd was killed, I received maybe 30 speaking invitations in the like the following five months. And I was not able to do that. And I also am the father of young children. And so I was then and am now managing work and COVID childcare. But I thought it would be helpful to sort of put some of this down. And so I encourage you if interested to really, you know, dig deep, there are lots of links there. And the first thought I had in terms of sharing was really thinking about, you know, what are we trying to do? Right? So so often we're talking about let's promote diversity, which is good and important, but promoting diversity isn't simply the matter of having different colored bodies coexisting in a laboratory, but really ensuring everybody can show up and contribute at their as their full cells. I think this reinforces the points that others have made. But you know, if you don't have a culture of equity and inclusion, diversity can actually represent another form of oppression and exclusion, right? Because if you commit to diversity, but not committing to racial justice, you're basically reifying reinforcing the social hierarchies that have marred our society and our enterprise for the last few centuries. And it directly harms scientists from underrepresented groups and other minority groups, as well as impairs our entire system, which loses out on our contributions. And so when we think about how do we move forward as a community, I think it's important, and especially if you have power in the community, how do we listen to those who've been affected, acknowledge Dr. Collins is an example of that earlier, where we aren't how we got there, including our own roles and perpetuating the inequity, and then importantly to act, right? And so listening means taking the time to understand what's happening, providing the space for your colleagues or trainees, if they choose to share their trauma, they don't need to. But really, and I think this conference is probably better than many that I present at, but really listening to social and behavioral scientists who have rigorous frameworks to understand what this is. And we listen not to necessarily defend ourselves or the perceived meritocracy of our systems and institutions, but really to learn. After we listen, we acknowledge, we tell the truth about where we are and how we got here. Again, Dr. Collins gave an example of that with the Unite. And I think about ourselves as scientists, we are committed to truth. The truth can set us free if we let it. And so if we truly acknowledge how we got here, including with the best of intentions, that helps us to then lay the predicate to act, right? And as I say, a fair amount, talking internal feelings alone don't bring about change. Resource allocation and policy do. And so again, I talked about the mosaic program as an example of action, different reviews, how we reassess peer review processes, admissions processes, who gets published in our journals. All these things are examples of when we structure them differently, we can have different outcomes. I think it's also important to let both local and specifically think about who is in your context and what do they need, right? And so generalized diversity that doesn't deal with the needs of the specific people there are unlikely to be super effective. And so I do remain confident that we can make real progress and we can, if we act to make positive change. As I say regularly, scientists, Black, Latin, indigenous scientists need what everybody else needs, opportunity, resources, and respect. And as we recreate our spaces to be more just and equitable and inclusive, we will have more diversity, we'll have better science and a better society. I know we sometimes say this is hard, but we are scientists, we do hard things, we've put people on the moon, we may as a regenerative medicine, CRISPR, and we had COVID vaccine in under a year. So we can do hard things, including working purposefully to undo the structures that hinder opportunity for so many. And I look forward to answering any questions and continuing to be a part of this work as we move forward. So these are my contact information, you can send me an email, you can follow us on Twitter at NIGMS training, I'm happy to answer any questions or just to converse with my colleagues in the remaining minutes that we have. I don't know other questions, I don't know if I'm seeing anything, which is fine if I'm missing it. Okay, so I don't see any questions, but I would love to, you know, converse or if there are any, answer any. Alrighty. Thank you. So, Carol, you are our moderator, and I will turn it back to you. Thank you for inviting me to participate in this great group with these just, you know, I love to hear both your talks and look forward to continuing the conversation going forward. I'm so grateful to you and to Dr. Maybank for doing all that hard work of figuring out how to change the policies and how to get the resources for they need to get. I think that kind of institutional change is so hard and so important. And thank you for taking the time to participate in this symposium. I'm looking forward to seeing both of you thrive in these endeavors. Thank you. Thank you. And I see one question, this is how, and maybe I think I would love to hear other thoughts on how do you deal with administration putting up roadblocks for grassroots DEI slash entire racist work. I'll just say, you know, I think about how do you go with there are a number of different ways, but one, think about who your who your allies are, right? Are there people who can be allies that that they listen to? There's also this idea called interest convergence. And so how do you how do you make sure that you can align your what you're doing with the interest that they have, right? And so sometimes people do things because the right thing to do is sometimes they do things because it's expedient. But either way, you know, if you ideally people do what's right, but if you get the resources that you need, that's what's important. I would invite doctors may bank or lead also share their experiences about if it was at the very last second. But I think you're but I think it came across. Yeah, I mean, I agree with that. And I'm somebody else put that in the chat that I'm very much a believer of organizing models of being able to create change. It works on the outside of institutions, it will work on the inside of institutions. And it's the same premise of finding those champions and folks willing to work with you accomplices, I call them or co-conspirators. I'm to come together and help create, you know, I think a sense of community and solidarity around the work. And then, you know, put forward some ideas for some actions and find those pathways of what you're able to elevate, because I'm very clear, you know, in organizations that aren't, you know, NIH who has top level leadership committed to this or AMA who has top level leadership. And it's really hard to really move this work forward. But as Dr. Gibbs said, not impossible. And it's possible. But you have to have really think through strategy around it. And I think it's finding the others as a start who are interested in moving forward with you. Well, on that optimistic note, I'll close our session. Thank you so much to Dr. Maybank and Dr. Gibbs for their work and for their contributions to this session. I'm so grateful. Thank you for having us. Take care. Have a great rest of the conference. Bye. Bye, everyone.