 Welcome to the April Health Policy and Bioethics Consortium. I'm Aaron Kesselheim, a faculty member here at the Center for Bioethics and a professor of medicine at Harvard Medical School. It is my pleasure with Leah Rand to be able to bring these monthly consortia to the community. These are an organizational effort of, from the program on regulation therapeutics and law, which is an interdisciplinary research group that Leah and I are a part of here in the Division of Pharmacohepidemiology, as well as the Center for Bioethics. And they run monthly during the academic year. This is our last one of the 2022-2023 academic season. It's been a great series. And we're glad that folks have joined us, been able to join us for the full series. And we're really excited about this final topic today, discriminating devices, the case of pulse oximetry. Next slide. So this is an hour and a half session. We are going to have some introductory comments from our moderator and esteemed panelists, and then have some moderated discussion, and then finally, some audience questions at the end. Please submit your questions using the Q&A feature. Don't use the chat function. The chat function is only for technical issues. And the Q&A feature is where you should submit your questions. And then our moderator, I'll direct the questions to the appropriate panelists, or both of them as need may be. If you feel the need to tweet about this, we use the hashtag HMSBioethics. And if you are interested in this series and want to hear about our next year's program or any other consortia sessions that the Center for Bioethics organizes, please check out the Bioethics website. Next slide. So the goals of this consortia are to try to identify key issues in the health care system that involve ethically challenging policies or practices, then try to articulate what are the key issues that these topics involve. Bring together experts who provide different perspectives on the issue, and to try to stimulate conversation and further academic study of the topic. Next slide. And so we have a couple of organizational ethics consortia remaining. But again, as I said, please check out the Bioethics website for further details. Next slide. So let me turn it over to Leah, who will bring us into our topic today. Thank you, Erin, and thank you everyone for joining us and to our two panelists and our moderator for taking the time to come and talk with us today. So I'm pleased to introduce our moderator to this conversation, Fraser Tesla, who is a researcher at Portal and a medical student at the Pritzter School of Medicine at the University of Chicago, where he was selected as a Bucksbaum student scholar. And at UChicago, Fraser researches the pathophysiology of microvascular dysfunction in heart failure and health policy initiatives related to sickle cell anemia. And he has published on improving access to pharmaceuticals, pharmaceutical innovation, reforming regulation, and improving health equity through new therapies. He graduated from Yale University in 2017 with a BA in history and conducted research in behavioral psychology at the Yale Decision Neuroscience Lab and Translational Medicine at the Yale Cardiovascular Research Center. He's also been a research assistant at Portal, where he has put out some important publications. And so we're really pleased to have him here today to moderate. Thank you so much, Leah. I'm also going to introduce our panelists for today. We're very excited to have two great panelists who did a lot of the early work during COVID on this topic. So I'm just going to read out a brief introduction for them. And then with no further ado, we'll start our introductions and talks. So our first speaker today is going to be Dr. Thomas Valley, who is an assistant professor in the Division of Pulmonary and Critical Care Medicine in the Department of Internal Medicine at the University of Michigan. He received his undergraduate degrees in History and Chemistry from Emory University and his medical degree from University of Miami. He completed his internal medicine residency and chief residency at the University of Texas Southwestern Parkland Memorial Hospital. And he joined the University of Michigan as a pulmonary and critical care fellow in 2013 and is the physician lead for the University of Michigan Swartz Rounds for Compassionate Care. Dr. Valley's research aims to understand and improve medical decision making in the intensive care unit. In particular, he seeks to improve ICU admission decision making to help clinicians decide which patients need and do not need ICU care. Dr. Valley also conducts research to understand the experience of family members of ICU patients and why many family members have symptoms of depression, anxiety, and post-traumatic stress months after leaving the ICU. Our second speaker today will be Dr. Amy Moran-Thomas, who is an associate professor of anthropology at MIT, interested in the human and material entanglements that shape health and practice. She received her PhD in anthropology from Princeton University in 2012 and held post-doctoral fellowships at Princeton and Brown before coming to MIT. Her writing often focuses on the social lives of medical objects. She also works on the cultural anthropology of intergenerational health, planetary change, and chronic conditions. As well as questions of equitable device design, technology, and kinship, and the afterlife of energy, carbohydrates, and hydrocarbons across scales. Professor Moran-Thomas has conducted ethnographic and historical research in Belize, Guatemala, Ghana, Brazil, and the US. Supported by the Mellon American Council of Learned Societies, the Wenin-Gren Foundation, the Rachel Carson Center for Environmental Environment and Society, the West African Research Association, and the American Philosophical Society. Her first book, Traveling with Sugar Chronicles of a Global Epidemic, published in 2019, examines the global rise of diabetes as part of the ongoing legacies of sweetness and power, including how unequal access to insulin varieties, oxygen chambers, glucose meters, dialysis devices, farming machines, coral restoration, and prosthetic limb technologies can shape how histories live on in the present, impacting lives and landscapes across generations. So before we have our speakers speak, I just wanted to briefly say what the format's going to be. We're actually, I think, going to bounce back and forth. So we're going to have Dr. Vali begin and talk about what care was like before COVID in pulse oximetry and pulmonary critical care. And then we're going to go to Dr. Moran-Thomas. He's going to talk about social histories of oximetry as COVID began. And then we'll return to Dr. Vali. We'll talk about his work that he published in December 2020 in a New England Journal of Medicine study that set the stage for a lot of what we're talking about now. And then Dr. Moran-Thomas will also talk about her work, which was also published in 2020 and the FDA's responses to it. So with no further ado, Dr. Vali, if you'd like to begin, we'd love to hear what you have to say. Wonderful. Thank you, Fraser. Let me share my screen. Can you guys see that okay? Yes. Great. Thank you. So I'm very excited to be here to not only talk with you about this incredibly important problem in medicine, but also to share the stage with Professor Amy Moran-Thomas, who truly broke this story before anyone and allowed us to build on her amazing work. I work clinically in the intensive care units at the University of Michigan and the Ann Arbor VA. And my research focuses on improving the quality of care that we deliver in ICUs across the U.S. And as a result, pulse oximetry is incredibly important. And so I wanted to start by just talking about how we measure auction and why that's important. Some have even termed auction saturation the fifth vital sign. In addition to your heart rate, your temperature, your blood pressure and your respiratory rate, every day when we're in the ICU and our medical students, our residents are presenting a patient to us, auction saturation is one of the key parts of that story. It tells us so much about how a patient is doing on a given day at a given moment. And maximizing auction delivery to our vital organs is really the cornerstone of ICU care. That's why being able to measure what someone's blood auction levels are is so critical. Right. We we use that all the time, minute to minute, second to second to know how a patient is doing. But it wasn't until about 50 years ago that we could reliably measure auction noninvasively. And we've made tremendous advances in measuring auction over the past hundred fifty years. I mean, we didn't even know until the 1860s that hemoglobin was the carrier of auction in the body. In fact, the pulse oximeter has been called the most important medical advance of the 20th century. And here's a picture of one of these early devices on the ear is a probe that detects this woman's auction saturation. But as you can imagine, these early devices weren't perfect. And so while we use them in practice in the 1980s and 1990s, there were still technical issues that prevented them from being considered the gold standard for knowing what someone's auction level was in their blood. That gold standard even today remains checking auction levels directly from someone's arterial blood. And this is a picture of an arterial catheter, which allows us to repeatedly sample someone's arterial blood. Without this arterial catheter, what we would have to do is stick a needle into someone's artery every time we wanted to get get a truly gold standard accurate measurement of someone's auction saturation. These arterial catheters can be left in and we can just repeatedly just take blood from these catheters. But over the past 20 years, there's been a push away from using these devices, mainly because conventional wisdom was that they were no longer necessary that pulse oximeters were so perfect and used so frequently that we no longer had a need for arterial blood sampling, that we had a new gold standard, in essence, for checking someone's auction levels. We could just go off someone's pulse oximeter measurement. Well, let's take a second just to talk about how a pulse oximeter works. Its goal is to measure the proportion of hemoglobin that's in your blood that's saturated with oxygen. And most often we use it on the tip of one's finger. On the top of the inside part of this device are a pair of light emitting diodes that send two wavelengths of light through the nail bed. On the other side of the device is a system to detect the light that's transmitted through the finger pad. So the pulse oximeter is detecting the difference between what light is sent out and what is actually transmitted across. And there are two key scientific facts that allow pulse oximeters to work. The first is that oxygenated and deoxygenated hemoglobin absorb light differently. So this is a graph of the different wavelengths that hemoglobin absorbs light. On the X axis, we have wavelengths of light. And we can see that with red light at a wavelength of close to 700 nanometers, deoxygenated hemoglobin absorbs more light than oxygenated hemoglobin. However, with near infrared light, oxygenated hemoglobin absorbs more light than deoxygenated hemoglobin. Therefore, oxygenated blood is more likely to transmit red light across the finger and deoxygenated blood is more likely to transmit infrared light across the finger. The second scientific fact that dramatically improved how pulse oximeters work is the use of the body's pulse. One of the reasons that I think we clinicians believe so much in pulse oximeters, we believe that the value that they're giving to us is the truth that we trust these values is because modern devices actually give you a value that says what the quality of the measurement is. Each device is different, but in some it might give three asterisks if it's a high quality measurement and one asterisk if it was a questionable value. Now this quality measurement is derived by how strong someone's pulse is, not by any other metric, but just how strong someone's pulse is. And if you were to use a pulse oximeter on someone without a pulse, it wouldn't work because the pulse oximeter makes use of differences in light absorption due to changes in blood volume. To reliably measure differences between arterial oxygenated and venous deoxygenated blood. For example, during the venous phase, you'll have a lot more deoxygenated blood in the finger than during the arterial phase when you'll have a lot more oxygenated blood. Each modern pulse oximeter has been calibrated during its testing phase to create proprietary formulas that can account for the different wavelengths that are absorbed based on blood volume. Oxygenated blood, deoxygenated blood, and skin tissue. And when you look at this figure, you and as I and many others have taken for granted that other more static factors that might affect absorption like skin tone, nail polish, the size of one's finger should be held constant in this area of absorption due to tissue and wouldn't ultimately affect the quality of the measurement. We now use pulse oximeters all the time in medicine from at home on our on our watches, or as we were basically telling everyone during the pandemic to go out and buy a pulse oximeter or to being continuously measured in the operating room or the intensive care unit. We use these devices all the time. These devices are also being used every day to make important life or death decisions. Whether to bring someone to the hospital when an ambulance comes to their house, whether to put someone on a breathing machine, whether someone should get a lung transplant or not. These are incredibly important devices in everyday medicine. And we were using them all the time, particularly during covid. We talked a little bit about how we were telling folks to get pulse oximeters at their houses. Well, in the ice use in particular, what we saw in covid was essentially a respiratory disease where patients were having trouble breathing having low oxygen levels. So pulse oximeters were incredibly important for covid. And in Southeast Michigan and particularly Detroit, we were we were one of the places in the country that was really hit hard by covid in the spring of 2020. And as a result, the University of Michigan was flooded with patients with covid. In fact, while we normally have 20 medical ICU beds within a matter of a few weeks, we had to create 160 additional ICU beds for patients with covid. And during covid, our group at Michigan started to notice something peculiar. The patients we were taking care of seemed to have different oxygen values, depending on whether we checked by pulse oximeter or whether we sampled by their arterial blood. And we were particularly noticing this because one of the many unusual things that happened to our clinical practice in the ICU at that time was that we were placing arterial catheters in every patient with covid in our ICU. Now, remember, we had increasingly moved away from the routine use of arterial catheters over the past 20 years. But because at that time during covid, especially early covid, we wanted to limit the amount of time that doctors and nurses were spending in the rooms of patients with covid. So to decrease their chance of contracting covid, we were placing these arterial catheters in every patient who came into into our ICU. And so as a result in kind of an unusual situation, we were getting a lot of simultaneous measurements of oxygen from both pulse oximeters and arterial blood for every patient in our ICU. The second thing that was out of the ordinary during that spring covid wave, I mean, a lot of things are out of the ordinary. But in particular, for this story, the second thing was the patients that we were taking care of. Based on local demographics, the University of Michigan's hospital traditionally tends to take care of about 80 to 85 percent white patients and about 10 percent black patients. But during that first wave of covid, we saw the demographics flip dramatically. We were taking care of about 80 percent black patients in our ICU at that time in that early spring covid first wave. And when we were seeing that mismatch between pulse oximeter values and arterial blood values at that time in the spring of 2020, we thought it must be a covid thing. I mean, like anything at that time, right? We thought everything was due to covid. And so there was something about the virus that caused covid might that might be causing this mismatch. And it wasn't until the fall of 2020 that we started to think of another possible reason. Professor Moran Thomas, if you'd like to jump in. And thanks so much. I'm so honored to be here in conversation with Dr. Valley and all of you. And Julie, if you don't mind queuing up my slides. Thank you. So since Dr. Valley just shared a bit about the clinical picture, I'll try to say something about social histories with a focus on cultural dimensions of the problem of racial biases in oxominers and related medical devices and kind of try to take up this invitation to use my own experiences trying to publish at the time covid began as a way to look out at larger infrastructures, kind of trying to reflect on how a problem can be known and not known at the same time. So next slide. You could actually flick ahead to the video after that. And if you tap once more, I will start to play. So this was the first pulse excimator that I interacted with when my family got covid early in the pandemic, sort of that very disorienting time. I bought this at our local pharmacy. And at the time, as Dr. Valley just mentioned, these numbers were being held as sort of a quote unquote biomarker for covid. So next slide. Many hospitals that even issued written guidelines to come in when it read 92. But our doctor said wait to go to the ER until 90. So I was spending many late nights staring at this thing, flickering on my husband's finger, kind of like, go to the ER, no, stay home, go to the ER. And I just started wondering not only as a concerned family member, but also as a professor who studies medical device design from social perspectives, sort of what is this thing? You know, how is it actually working? And in particular, I could see a red light glowing inside of it. So I wondered how this device had managed to avoid the issues of racial biases that other light sensing devices are famous for. And Julie, if you don't mind sort of moving ahead to the. Two more slides to the. So the one that, yeah, vision and justice. So so I Googled it. And that next slide and the studies about racial bias in the device popped up immediately, coming out of UCSF from 2005 and 2007. And the authors kind of explained that in their lab, they realized they were testing a color sensing device on almost all white test populations and noted historical reports of errors and did a systematic study to show that hospital devices at the time didn't meet FDA's safety thresholds actually for people of color. So reading this, I was thinking as a medical anthropologist, next slide. You know, we build social assumptions and theories into our material world all the time and sort of those questions of who is an imagined user versus an actual user of a device. There's this huge SCS literature and all that. And the ways attributions of troubled breathing, you know, have a long, very racialized history in particular. And I also work on how seemingly tiny device errors matter in my work on equitable device design for diabetes. So next slide. And it wasn't until Bernadine Healey's tenure as head of the NIH in the 90s that women and people of color started to be included in most FDA trials at all. So all that was on my mind at the end of May, 2020. So if you can double click to me. So I drove home from the Brigham ER after they kept my husband overnight with COVID. If you don't mind clicking once more to start the video. And Boston looked like this that night. There was, I ran into all this traffic on the way back. And I remember thinking, you know, what people are out protesting is actually built into this little device I, you know, was holding to. And I started to wonder, was that actually fixed in the 15 years since that data came out? And maybe that kind of quietly happened all the scenes but I thought like, this will be my way of being in this moment, just to try to ask for updates since 2005 and sit with this until I see data that this problem was fixed. And I started hearing back after reaching out to people but no one could show me that data. So MIT participated in shutdown STEM that year which was led by physicists on June 10th and sort of encouraged all the MIT faculty to spend a day thinking about how racial biases might be part of their own fields and tools. So that created a chance for me to bring these questions to engineers. So three or four of us got together. We were put in touch by the chair of MIT's social and ethical responsibilities and computing group. And I felt really lucky to be able to have a conversation with people who are some of the world's leading experts on the technical side of optical devices. And so we all looked at the data together. I shared what I had of working drafts kind of thinking through these questions at the time and they confirmed like, yes, this is still a thing and it's solvable. Like it's, you know, they could think of a range of technical ways to address it. So one engineering professor offered to share some cutting edge kind of photo acoustics work he thought would be relevant in this pre-competitive consortium model. They use it MIT when technology is considered sort of so foundational that someone chooses to open it to all companies in a sector. He thought it would be a great candidate for big NIH. And around that time, I got the exciting news that the New York Times accepted my draft and sent me a contract. So next slide. And our plan was once the New York Times piece came out the engineers would post it on Twitter and put up a flag and kind of invite any companies who wanted to come work on this to MIT and, you know, adopt some of these technologies. So I came away that day thinking, you know, that might actually work. Like this should be the lowest hanging fruit of systemic racism. Then the pushback on my op-ed draft started from places I had most expected allies actually like from doctors and from the FDA. So next slide. And I won't use individual names here. I think these individuals were articulating the common sense of their respective institutions. But since the FDA officially marked what I'm sharing here for attribution let's take them up on that and kind of the agency level to try to understand where things sit at the time. So I wrote to them kind of saying the only thing I could find in their latest guidance was this sort of underlined clause about including at least two people with darker skin in each sample which are only groups of 12 people. And that didn't seem sufficient to prevent the errors that, you know, the latest public-facing data showed. And so I was curious, is there anything in the FDA pullback guidance today about equal safety for both white and non-white groups of people? For example, et cetera. So the FDA wrote back asking for extra days to hold a group meeting about my question. And then they responded. Next slide. It looks as if you may have inadvertently misunderstood our recommendation. We suggest including two darkly pigmented subjects or basically they sent back the exact same sentence I had sent them but with bold fun. And they also sent a sort of diversity statement. So I honestly thought the times editor would find, would join me in finding this a non-answer. And I pointed out that it was the same sentence and asked the FDA what they thought I misunderstood but they never wrote back and the times pulled the piece. But I don't think they would have reacted like that either on an editorial level if it wasn't for the input of doctors. So in the past year, I've heard people say the issue of racial biases in axiometry wasn't in textbooks but that's not quite true because that's one of the places where I learned about it. So next slide. So this is the textbook where I read about the problem and the doctor gatekeepers I was speaking with at the time told me, we never teach that part of the textbook. And that space between what is known on paper and what's actually taught in medical school, that's culture. And I tried to press the doctors I was speaking with like had the data been debunked, had it been updated? And the answer was sort of, it couldn't be true this matters for practice or I would have learned it. But of course, looking back on the history of medicine all kinds of clinical practices that now seem and just were normalized in their time. And I think it's good to remember since people sometimes look back on this already as a long debate. If it was a debate, I should have been able to publish an op-ed about why I thought the errors could matter. A doctor could have published why they thought it didn't and a public conversation could have happened. But it didn't feel like a debate. It felt like a medical consensus that was wrong and that kind of like couldn't get space. So in the end, the New York Times said, kind of thanks anyway, unequal errors in this case don't seem like they could possibly be meaningful in practice, but we're really interested in publishing work about the issue of racial biases and technologies. So if you ever find an example of that, please let us know. And so thankfully at that point, someone helped me connect with the Boston Review. And Julie, if you don't mind clicking, it might actually be twice. And that lured sort of a phenomenal editor there. If you can click one more time, but those were some images shared by Nonan at the time I'll mention later. But he brought it to a reviewer who seemed to have a very strong background in medical STS. And next slide. And the night before my piece came out, I kept obsessively rewriting this paragraph. And I was thinking as I edited, maybe researchers somewhere will find this and think, we could actually collect that data. Let's look. And I didn't know it at the time, but one of those physicians ended up being Dr. Valley and his colleagues in Michigan. So I feel really honored to stop here and turn things back over to him. All right, just sharing my screen. Can you guys see that? Great. Yes. Thank you, Dr. Miranda Thomas. After reading her brilliant paper in the Boston Review, in the context of the things we were noticing clinically. Remember, we were seeing this in practice during COVID, but we really had no idea what the underlying problem was. We thought something about COVID was making this happen, but it was really Dr. Miranda Thomas's paper in the Boston Review that really turned on a proverbial light bulb. All of a sudden in our minds, we wondered, what if this wasn't a COVID thing? What if what we were seeing was actually inaccuracies in pulse symmetry among black patients? And I remember during those early days, there was a flurry of texts going back and forth between our research group, because first we thought like, this can't be real, right? Dr. Miranda Thomas talked about that textbook, right? Like, I feel pretty confident in saying that this information that we're presenting today was not routinely presented in medical schools. We had five different co-authors. We're all pulmonologists. We all work in critical care. We went to five different medical schools, four different residency programs, three different fellowship programs. Not one of us considered that this could be as big of a problem or even an actual problem in our daily practice. And so that was kind of the first thing that we thought like, this can't be right. And then the second thing was like, well, if it is a thing, it must have been published before. Right? And just like Dr. Miranda Thomas talked about, like we scanned through PubMed. We've found several studies from the 80s, 90s, early 2000s. Mostly from laboratory settings that did suggest that pulse oximeters might not be as accurate in black individuals as in white individuals, but generally the conclusion to all of those studies was like, there is a thing here, but it's probably not clinically meaningful. And we were stunned that there had been no clinical studies since 1990. And so very quickly, we realized that we were uniquely situated to look into this question as clinicians but also as researchers because much of our group's work is focused on the use of secondary data to answer clinically relevant questions in critical care. And so we essentially conducted two separate studies. First, we put together our own data from patients hospitalized within the University of Michigan. And I'll be honest, this was mainly to test whether this theory was actually a thing or not because we still were very skeptical that this could be that much of a problem. Pulse oximeters were just so embedded in our medical culture. And when we did this study in our University of Michigan data, we did see that on a large scale of all these patients over many years at the University of Michigan, pulse oximeters seemed to be working more accurately for white patients than they did for black patients. But we also quickly recognized that there was need to look outside the University of Michigan as well, just because if this is a pulse oximeter problem, then the first question that any reviewer was gonna ask us, well, how do you know this isn't just a University of Michigan problem? Maybe it's just the pulse oximeter you're using at the University of Michigan. But fortunately, our lead author, Mike Shoding, had a lot of experience using the MIMIC database from Beth Israel, which houses data from ICUs in 178 hospitals from around the country. And this is the main figure of our study, which again was led by Mike Shoding, co-authors Bob Dixon, Jack Awashina, Steve Gaye. On the Y axis, we have the gold standard of arterial blood. And on the X axis, we have pulse oximeter values. In red, we have values for black patients. In blue, we have values, excuse me, in red, we have values for black patients. In blue, we have values for white patients. And we can see the horizontal line within each box is a median value. We can see that on average, values for black patients were about 2% lower than for white patients. And on its face, this is actually kind of similar to what the prior laboratory studies that Dr. Moran Thomas showed from 2005 to 2007. Someone might just look at these values and say, 2%, that's probably not clinically meaningful. But I think what we did differently as clinicians was that we were interested in something that might be more meaningful for patients and for clinicians. And so in our study, we created a unique variable that we called a cult or others have called hidden hypoxemia. So in essence, it's the proportion of measurements where clinically meaningful low-oction levels might be missed, right? Where the pulse oximeter is essentially lying to a patient or clinician. And we chose values less than 88% as clinically meaningful low-oction levels because that's the value that we as clinicians would most definitely make a clinical change. We'd start a patient on auction or we'd increase the amount of auction that a patient's already receiving, right? So we're seeing how many times is a pulse oximeter missing when a clinical change should be made? And what we found was that black patients were three times more likely to have hidden hypoxemia, hidden low-oction levels by pulse oximeter than white patients. That's a major difference in accuracy between black and white patients. And that one out of every six pulse oximeter measurements in black patients missed low-oction levels, providing a false sense of reassurance to clinicians and patients. And when you think about the number of pulse oximeter measurements we take every day across the country, that becomes an absolutely staggering number. And finally, as we can see here, this finding has major clinical implications. At values below 92% on pulse oximetry, many black patients actually had auction levels below 88%. Again, a point when most clinicians would be making changes to the amount of auction that we're providing to patients. And we can see that at 92% of an auction saturation on pulse oximeter, about a third of a patient's values are actually the less than 88%. And once you get to 91%, we can see that more than half of black patients' measurements were below 88%. Since we published our study in 2020, there have been several other studies in different patient populations that have shown essentially the same thing. First, that in very sick patients who are receiving extracorporeal membrane auctionation, in children, Dr. Andrews showed this result, showed essentially the same thing in hospitalized children. And then in a large cohort of patients across the VA nationalities. We also saw what I actually never thought we'd see because, again, we get these isolated pulse oximeter measurements. I didn't know if we would actually be able to see differences in outcomes related to pulse oximeter measurement inaccuracies. And yet, over the past year or so, we've seen three incredible studies that have come out. On the top, we have a study that showed black patients who experienced hidden hypoxemia had delayed treatment for life-saving treatments for COVID, right? It's not surprising. Many of the treatments that we provided for patients with COVID depended on someone having low auction level. If a pulse oximeter missed someone having a low auction level, well, those individuals had delayed treatment. We also had studies that showed black patients received less supplemental auction. Again, not surprising because if a pulse oximeter is essentially lying and saying that a patient has elevated a higher auction level than they actually do, they're gonna get less oxygen. We also have two other studies that showed black patients were more likely to die and have more failing organs as a result of inaccuracies in pulse oximeters. So I just wanna hit on a couple areas of the importance of this problem clinically. First, I mean, just think about all the different places that we're using pulse oximeters. I touched on this before, but we've got home pulse oximeters. We've got pulse oximeters being used in ambulances, in clinics, in emergency departments, in the hospital when we're getting ready to discharge someone from the hospital. When we're testing about whether someone actually needs oxygen or not, whether we're considering whether they should get a lung transplant, right? Pulse oximeters are embedded within our everyday practice of medicine. And a question that I've gotten a lot since our study is how should we be using pulse oximeters now, right? How do we incorporate this evidence into our daily practice? Well, this is really tough because we use pulse oximeters so often primarily because the data that they provide are so valuable. And they still are valuable as long as we consider them within the context of any other test out there, right? We have lab tests that we do every day in medicine that we don't consider as 100% truth, right? It's just a piece of data that goes into a larger context within a patient's care. And for a long time, we didn't consider that for pulse oximeters, right? We just assumed that if a pulse oximeter is telling me that a patient's auction level is 100%, well, their auction level is 100%. So now what do we do? Well, some people say, well, we can rely on patient symptoms. And sure, we absolutely should be doing that. But at the same time, there's actually very little correlation between whether a patient feels shorter breath and their auction levels, right? Oftentimes people get shorter breath, but their auction levels are normal. Sometimes people don't feel shorter breath and their auction levels are abnormal. And so that isn't a fix by itself. Physical examination is difficult because by the time someone has low auction levels that result in changes in their physical examination, like cyanosis or turning blue, usually someone's auction level is already in the 70s. And to compound that, identifying cyanosis in an individual with dark skin tone is more difficult. We could consider using pulse oximeters in different locations on someone's skin, like on their ear or on their forehead, but that hasn't been well tested. You know, we could get more arterial blood, right? That is the gold standard. And I think ultimately, there are really two options that I recommend to folks who ask me about this. The first is, yeah, like if we have a suspicion that a pulse oximeter might not be working that well, particularly for black patients, we need to get more arterial blood or arterial blood gases or ABGs. That is definitely what we need to do at the same time. That's problematic, right? We're exposing black individuals to more invasive testing, painful testing simply because a pulse oximeter doesn't work as well for them as it does for other people. The other option is to just have a higher auction saturation, right? So that we maybe have a threshold that a black patient shouldn't have an auction level less than 94%. That is an option. And I think that is something that we're definitely doing, but at the same time, that again exposes black individuals to more treatments simply because we don't know whether we can trust a pulse oximeter's values in those individuals. Dr. Marin Thomas brought up medical education and research and how we disseminate that information. We were absolutely shocked to see essentially the same study that we did at a smaller scale, but essentially the same study with incredibly similar results that was performed in 1990. This was performed in Houston at a university in Houston and they again showed the exact same thing that we showed. This is their main figure. I've kind of twisted around to make it similar to our main figure, but we can see on the right side, we have measurements for black patients. On the left side, we have measurements for white patients. The shaded gray area is the area of occult or hidden hypoxemia. And we can see more black dots for black patients than for white patients. The same thing that we found 30 years before and yet we didn't disseminate this information and we certainly didn't do anything related to pulse oximeters as a result of this study. Since the publication of our study and Dr. Marin Thomas' work, we've seen some movement in terms of clinical societies, academic societies in intensive care, in pulmonology, trying to promote guidelines to at least help individuals recognize that pulse oximeters need to be considered in the context of potential inaccuracies. We've seen senators Warren, Wyden and Booker push the FDA to explain why this finding might exist. The UK has really taken the lead on evaluating equity in medical devices and the FDA has put out warnings about the limitations and the accuracy of pulse oximeters. But I just wanna conclude by talking about how we can make changes at scale. And I think ultimately that is, it needs to be pushed by the individuals, the consumers who use pulse oximeters. You and I every day, when we use a pulse oximeter in our home, by clinicians in hospitals and by large healthcare system. If we can push pulse oximeter companies to make changes, then we could have better pulse oximeters that work for everybody. And the VA in particular has had increasing interest in this problem. And my hope is, and I've been working with the VA as probably the largest purchaser of pulse oximeters nationally in pushing for change in making sure that pulse oximeter companies provide the data that demonstrates that their pulse oximeters do work as well for black individuals as it does for white individuals before we purchase those pulse oximeters. I think that is ultimately in my opinion, the only way we can make pulse oximeter manufacturers improve their devices. So with that, I will turn it over to Dr. Moran-Thomas. Thank you so much for that. And appreciation to Julie for some of these slides are a little more complicated than I would have made them if I knew it would be going on verbal cues. So thank you. So next slide, kind of picking up right where Dr. Valley left off. But it's been amazing to me to see the kind of cultural movements that his study together with Dr. Shoting and colleagues sort of catalyzed amongst doctors and all the, not just concern, but sort of new approaches to thinking about how to use data in hospitals as part of reexamining the tools that are in clinics and trying to use that data to protect patients. And this was so instrumental in kind of moving, you don't mind the next slide, Julie, what became the FDA alongside so much media attention. Next slide. But the FDA hearing that some of you might have followed in November is something I wanted to mention. So maybe you won't be surprised to hear since I was asked to comment on the sort of social scientific aspect of these questions that it was frustrating for the social scientists to help bring these questions to attention that no social scientists were included on the panel. And this became sort of apparent pretty quickly and Julie, maybe I'll actually be able to do my own screen share from here. Is that working? Can everyone see okay? Okay, great. So some of the kind of proposed solutions that were put right up front in the FDA hearing were this kind of, I would say almost over focus on quantifying skin color. And two of the three scales that you see here, the von Lutchen scale and the Montsell chart are from anthropology. And I don't know, I was really surprised when I saw this because in anthropology, we learn a lot about the history of von Lutchen scales. It's sort of an example of some of the most troubling parts of how anthropology has been complicit in the history of racism. So this came from the late 1800s and I can say more about them in a moment, but I just, all the anthropologists watching the FDA hearing thought like, if this attempt to tackle scientific racism in devices starts out the gate by adopting a skin color, quantification device that was such a big part of colonial genocides across Africa and later embraced during the Third Reich, justice just can't come from that. And beyond the historical baggage of the social history of this particular quantification chart, it's also just a technical mess. So when people try to use it today, they usually print this out and it prints out like whatever color ink you have in your printer. So we kind of thought the FDA must not realize and it seems like they're poised to write this into the revision of the ISO. So we, as anthropologists, this is part of the history of our discipline and we should say something. So, and just to briefly mention one cell charts which are even more commonly used in sort of the history of oxymeter bias studies. So they also, in the specific part of anthropology they're usually used in as archeology to quantify the color of soil. But if you look at the longer history of this color system in general, you can see how some of the values resonate all too closely with the social valuations of color and the time, this is the board in 1907. So a lot of troubling studies historically have kind of dismissed the risk of potential racial biases using charts like this, including like holding these charts purchased from the Forestry Supply Service in Jackson, Mississippi, up to people's foreheads and there's a launch. So anyway, so we went to the ISO meeting to kind of try to share some of this. This was led by Professor Diabetten who's a very well-known medical anthropologist and I contributed to the presentation. We worked with the executive board of the Society for Medical Anthropology who reviewed and endorsed it. Other professors kind of gave us notes from their own specialties. And we were kind of hoping to convey in the meeting not just that these particular scales shouldn't be adopted, but also that these deeper questions need to be thought through in a more careful way or for any skin color scale. So I'm leaving out the doctor's names here because I think the ones whose words were inadvertently published later seem to have been actually trying to keep in line some of the other physicians who were talking over Professor Betten's presentation, apparently, the kind of laughing emojis more obviously unkind, but this was kind of the side chat of people talking over her at the meeting. And being part of the professional association that brought this issue to doctor's attention, kind of trying to talk with the regulators that seem to have missed it for decades, trying to tell them that they're proposing a solution that could easily mask the problem we were there to talk about. I sort of read this all as an indication that these questions still aren't being taken as seriously as their gravity seems to warrant. And we were trying to be nice, maybe too nice because we didn't get into the ways that von Lüchen was the most progressive member of the German Board for Eugenics and Racial Hygiene, but it seems like some of how people were attempting to dismiss that was by noting that the Nazis disagreed with him on many issues such as like whether people, humans were all the same species because von Lüchen is interesting precisely because he was trying to argue against the overt racism of his time. But in that process, he created a tool that became really widely embraced in projects of perpetuating scientific racism. And that's exactly the kind of thing we were hoping that the mostly white doctors in the room deciding how to measure melanin for pollstock studies moving forward might kind of think through. And there's a much longer history of how the FDA grew out, not just of safety and consumer products, but also there's a Eugenics history that some have argued as part of why it's so difficult for the FDA to fully grapple with racial inequities, algorithms and some of the particular technoscientific features that these histories kind of ask us to think toward. So there's more I could say about some of the particular algorithms. This one's from the library for Chinese circuits, but overall I just, I left that meeting thinking that everyone's looking for an algorithmic fix. Maybe it will be possible, everyone's waiting to see, but there might be other kinds of more like hardware change that's necessary. And here I'm thinking toward what Dr. Valley kind of left us with those questions of how to think about creating market incentives. Because from the plane, like leaving this meeting in San Francisco, I found myself looking out and kind of trying to zoom out and thinking of the history that axiometry grew out of. These are some of the earliest attempts to make instruments in the history of hot air balloons. And there is a whole sort of, side history of what some call lost technology of a different approach to axiometry by Hewlett-Packard in conjunction with NASA that found a way to grapple with some of these questions back in the 60s and 70s actually before the FDA had started regulating medical devices at all. And anyway, I found myself kind of thinking from the plane, those kind of trying to look down on the bigger picture and imagine, what is the future we want and how do we get there? And I would love to see an innovation rush on equitable technology. And I do think for that to happen, we need structures that align equitable technology with market incentives. NONIN is a really interesting example because they invested in equity early on and those early UCSF studies, their devices are the only ones that do meet the FDA safety requirements for people of color as well. They invested extra money in research in that. There's rumors that maybe they spent extra money on certain rare earth metals for electrodes or whatever they did, they invested in it and their devices cost a bit more. But no one really bought that. Or they have a niche market, but they're by far the smallest of the three companies when we look back on the manufacturers from those studies three years ago. So I think it's not just a question for manufacturers at all. Like it's what's wrong with healthcare systems and societies that never sent a market signal that equity was something that people wanted and demanded. So those complacencies are all sides. My discipline certainly included as I hope some of this discussion gestures toward, but if we're expecting manufacturers to invest money in R&D that will lose their company's money, like that's not the world that we live in. And so I think there are a lot of things that capitalism can't solve. And as an anthropologist, I think it matters to attend to those too. But in this case, I just don't think that's one of them actually. I think the lack of clarity over the past half century has stifled innovation for equitable technology. And that part of addressing that is trying to catalyze a market somehow. So I hope that's something we can talk about in discussion. Like how do we make that system? One good thing about going to the ISO meeting and actually talking to people like always you realize things are more complicated. And I always just thought, well, why not add more than 12 people doesn't seem at all sufficient? Like these test groups are so small in the pre-market samples that these are errors that show up and say one of every 15 people of color. So that these groups need to be hundreds. And so one thing that I got a chance to learn is that there are only two labs in the country that do this kind of testing. They can accommodate about six patients a day. It costs like six figures for a couple of days of testing. So yeah, you can't just ask for that to be suddenly increased by many orders of magnitude. They already have a backlog of about eight months. And a lot of the projects to try to test new more equitable models would be held up for many years if we just like tried to make those pre-market groups bigger. On the other hand, post-market sort of once something's already in a hospital everyone's kind of already in defensive positions if there is an issue. And so I wonder if there's some way we could talk about something besides just pre and post-market but that idea of more clinical observatories for equitable devices like some kind of third space. Could there be something like a clinical equity certification that kind of helps signal to insurance companies and hospital systems what they should be buying once those objects come to market. And I think as public awareness grows purchasers will like start sending message that we want to pay more for equitable devices up front. In fact, we'll only buy those devices, hopefully one day with demonstrated clinical equity records because alongside the clinical imperative and legal responsibility not to provide discriminatory services it actually costs them a lot more money when people experience harms from prolonged care and the need for additional sometimes even lifelong services after delayed or missing treatment. So I think there is, I don't know some space for to better align equity imperatives with like actual cost saving measures for healthcare systems and those things are so often in opposition. I don't think this has to be one of those times. I'd also love to see a huge response on the federal level like a massive catalyst from NIH especially since federal guidelines were part of the problem historically and why this hasn't been addressed. What could accountability look like beyond the liability model alone? I wonder if mechanisms like a federal tax break for hospitals buying new machines you know how to kind of make new technology once it comes to market actually accessible to everyone which is so complicated globally as well. I would love to talk more about that in Q and A but one last note looking ahead my colleague recently sent some experiments to space with NASA and he also often works with a picture version of chat GPT in his medical device design. So he plugs in things like this kind of just thinking for how to get outside the box but if you ask chat GPT to create a pulse oximeter that solves for racial biases this is what it gives you back and which appear to be regular pulse oximeters covered in African flags and other kinds of marketing gestures. And it strikes me as a pretty accurate reflection of where responses stand on this right now and we actually need socio-technical architectures that don't exist yet I think and ideas that don't exist yet to change the optical structures inside devices and not just kind of the marketing optics. So while we're talking about AI I also think there needs to be a lot more conversation about what machines are learning from the patterns that doctors are bringing to recognition and from the data itself as the doctors are also bringing that into visibility but machine the models that are being trained on this data must be seeing these patterns too what do they make of that? It's wild to realize that all oxygen saturation data in existence that AI is being trained on is distorted by these disproportionate errors and so how do we think about addressing that moving forward? And my colleague at MIT, Leo Anthony and Shelley works on this as well but here's how the algorithm used by Epic makes those cutoffs. It really classifies oxygen in eight different categories. We also need to think about closed loop ventilators and all the different technologies that are about to come to market that use full-sock symmetry as a direct input. There's so much more to think about and just charting what are the ripple effects across the healthcare system and how do we actually address this before all these things are about to become much more part of the clinical space even more than they already are. This is only going to get more and more difficult if it isn't addressed as thoroughly as possible like right now. So addressing this actually might be the lowest hanging fruit of systemic racism which is sobering because it's still incredibly monumentally difficult and even though it's technically doable the sort of socio-technical worlds around any device can't be separated out. So I do think we really need people working together across lines of difference and in ways our systems aren't set up for right now and how to get beyond the existing barriers and actually incentivize work like that is something I hope we can brainstorm about together in the conversation now because I'd argue that if the question is how to uphold non-discriminatory healthcare when using bias clinical tools, the answer is you can't. Thank you so much for these talks. This has been awesome. I think we're going to transition now to the Q&A section. So maybe for format, we can type in questions if you have them audience questions in the Q&A format. There were a couple that were asked during the talk that were sort of answered by Dr. Valley but we'll try to answer the ones that come in now in real time. And so I'll just, a question just came in, a couple of questions coming in. So I'll just sort of ask them in order and we also have some questions on deck too that we can ask too if they don't come to the audience. So we'll begin with the first one, which is, and I guess we'll have both panelists answer if we can. Is this device bias also possible in non-contact thermometers? Yeah, I can start. There's evidence that it is present. I did post it in the Q&A, I can post it again, but there was a JAMA study published within the past year with infrared thermometers, temporal artery thermometers, non-contact thermometers at a large scale across hospitals that showed very similar patterns. So yeah, unfortunately, I think this should make us this should make us reevaluate many different medical devices that we use within healthcare that this story is not isolated just to pulse-lock cylinders. Okay, moving on to our next question. Besides the dire need for diversity among researchers, are there attempts to increase collaboration between basic scientists? For example, engineers or physicists who are experts in how such measurements function, clinical researchers and clinicians in the development of such devices and policy initiatives. How might patient and public involvement be beneficial in this context? So maybe I can start, although I unfortunately don't know of specific programs or initiatives that are doing that. I'm sure they must be out there, but I guess I would just like to voice my appreciation of what an important question that is, because I do think, for example, looking back on the history of the HP device, you really see that they're just the fact that there are chemists working alongside engineers part of how they were able to calibrate their device so that it works equally for any patient. There was spectrophotometry of a drop of blood to avoid a whole ABG, but you could calibrate the device with that. And nowadays there's been such a kind of siloing within engineering, of course, that the people who are kind of focused on creating better algorithms and optical sensing, it's a different expertise and they might not necessarily actually be in conversation with anyone who could add that chemistry component to an optical sensor. So I do think that the idea of having more dialogue between doctors and practitioners and designers of all kinds and between designers from different fields is something that I would love to see more of in the world. Yeah, I totally agree. And I just wanna highlight kind of one aspect to pulse oximetry where that could have been so useful earlier, right? I think those laboratory studies that Dr. Moran Thomas cited in her talk, many of the laboratory studies, they look at statistical bias, they look at statistical bias across the spectrum of pulse oximeter values and they essentially treat bias at a range of, 70% auction saturation, the same as a 100, a bias at like 98%, the same as 88, 89, 90%. But clinically, we really care about the margins, right? We care about whether someone's at 87% or 90%. And you're gonna get a lot more patients in that space as well. Seeing patients at 70% is actually incredibly rare, right? And so having better measurement within that tight range of let's say 85% to 100% where most patients are is way more important than seeing what happens when someone's down at 70%. And so being able to talk about that with engineers when they're creating these devices, when they're testing these devices about what do we care about when we're taking care of patients would have been incredibly useful. And so I think it really should be an aspiration to have systems in place like that where we can have such multidisciplinary expertise working together. Thank you so much. We're gonna move on to the next question, which is, should there be some publicized ranking of results based on equity outcomes as for example, with the gradeschools.org and should this be done for medical devices? I think it's a great question. And I, so this is a little off the cuff, but I do think that I wish there was an architecture for things like that. I wish that, I don't think the FDA, at least from my glimpses so far will is able to create that right now. But I do wonder if spaces like medical schools who wear IRBs and tons of informed consent are possible. Could there be spaces within hospitals where like between pre and post market where for example, these are as noninvasive medical devices, I think a lot of people would actually say yes, if the presentation was would you mind wearing two of these because we know there's a problem and if the new one works better, you have advanced access and if it doesn't, this will help us catch it before it's sort of more widely in circulation. And if it was a space where industry was sort of invested in getting that clinical equity certification because they could frankly charge more for devices because they would be worth more to help systems and saved help systems money in the long term. Maybe that could be like a way to, okay, let's do another iteration. I think it would lower the barrier of kind of that feeling of like if there's something wrong, then there's so many questions that immediately come into play if things are sort of post market but if there was some kind of middle space, I wish there could be some kind of middle space that could be created together for more equitable devices and for that information about how actual performance is working in a clinical space to be publicly and transparently available to purchasers. Yeah, I mean, how cool would that be to have something like that? Even still when we talk to Paul Sock Simmer manufacturers and I don't know if Dr. Marianne Thomas has had this these similar conversations and heard the same things but when we've talked to the largest Paul Sock Simmer manufacturers, they consistently say that it's not a problem in their devices even though this manufacturer has market share across the country even though we at the University of Michigan use their devices and even though all of these studies are repeatedly showing the same thing across the country in different patient populations, yet they keep saying, hey, when we do our testing, we don't see this problem. And so I think that highlights the need for what Dr. Marianne Thomas said about having some sort of neutral group that's able to do this and show that, hey, there is this problem here because we're still getting that same pushback about whether there actually is a problem despite such overwhelming evidence. So for our next question, I'm actually gonna broaden this question a little bit too to just speak more generally about the federal practices that you would suggest at the end of your talk, Dr. Marianne Thomas. So we have a question asking, are creating market incentives that drive development of just medical devices with the increasing marital diversity of the United States? Isn't there an argument to be made that these manufacturers to reach a larger consumer base will have to make devices more equitable? Yeah, I mean, obviously I hope that will happen in the fastest way possible. My observation of as a social observer on where some of the things that are slowing this down are currently happening. I think the FDA has a lot to answer for here because manufacturers were meeting the FDA requirements. The FDA requirements did not require devices to work equally for people of color even though the data on this issue was there. So I certainly hope that now that there's more public awareness that manufacturers will invest in this issue on their own. I still think that that question of how to bring more transparency to that process. So when manufacturers have something they're proud of to bring and kind of show that this has been fixed, it would be great to have something besides the spaces where they couldn't see this problem before to hold up as evidence that there's no problem in the new ones. And so, and I do think in general, having clinical data right now that this cultural movement of doctors kind of using retrospective data, it's amazing to see, but it does have this kind of feeling of like ketchup or something that harm has already happened and that's part of what doctors are struggling against but everyone is trying to reckon with and it would be great to create just more visibility in terms of how could these studies happen not just retrospectively, but prospectively and like in partnership with industry and kind of trying to create more alignment. Yeah, I wish that was happening organically already and there are some exciting like the Equiox model at UCSF I think is really exciting and it would be great to see more universities take up work like that and think about how it could be expanded. Great, so I guess for the next question, it's what would be your recommendations for developing a new generation or the current development of a new generation of pulse oximeters? And the writer has a suggestion of perhaps providing melanin presence as a percentage but what also be curious is here if you have other recommendations about how to change pulse oximeter design and regulation of design? You know, I think with, I can't get too much into the technical details and I'm sure Dr. Moran Thomas has a lot more experience and knowledge in that space but what I can say is that I think there's a clear need for testing any future device in, you know we've talked a bit about diversity but I think one of the issues that has been raised with prior testing is not just about racial and ethnic and skin tone diversity but also just the patient populations in which you test these devices, right? We traditionally don't use pulse oximeters in healthy patients, right? Like a pulse oximeter is intended for a patient who may have a low oxygen level and those individuals also tend to have a lot of other things going on at the same time. And so I think it highlights the need, right? Like if we were to test a drug in an FDA trial we wouldn't be testing that drug on healthy volunteers and then putting it out into practice, right? We would be testing it in the patient population who would need this drug and we would have three different phases of trials and then we'd put it out into practice. We don't have that same routine for devices, right? We don't routinely test these devices in the patient populations who would need them in critically ill patients in the ICU before we put them out into practice. And now on one hand, Dr. Rand Thomas brought up how that can slow the implementation of these devices into practice. But I think it does highlight the need for post product testing. And there is a model for that already. Pulse oximeters are already routinely tested as post product in NICUs for neonatal intensive care units. And so there is a model for that in small children that we don't use in adults. And so I think it just highlights the need to continue these, to test these devices clinically after their development, however we do develop them. Okay, I'm gonna combine the next two questions because it's sort of both their comments and there's a question at the end that's related to both. So the first comment was there seems to be some NIH funding and NIMHD funding to sponsor sort of the equity development guideline stuff. And so someone put that in the chat if researchers on the call are interested in that that's probably a resource worth looking into. We have another commenter who talks about his work in Rochester, New York on the subject, a home of former Codec Labs which did a lot of the pigment stuff that's involved with the history of this technology's development. And has talked about a clinician education program feedback to device manufacturers including an open letter and research that's ongoing. And so I guess the question for all of these comments is there seems to be a lot of global national interest in the subject, lots of researchers and collaborative groups who wanna work on the subject. But sometimes the nature of this is that people are working in silos and maybe you're not aware of the work of other people. Do you as researchers in the space in different fields and anthropology and as a clinician have recommendations on the best way to collaborate? And also maybe more generally I'll open this up to what is the best way for cross-disciplinary research in the subject? Like for example, with this call we're hoping to have a clinician and an anthropologist on the call are there other opportunities that you think can be done to kind of bridge those kind of communication gaps? These are such great questions because there are such gaps and I'm really grateful for this chance to have a conversation together with Dr. Valley. And so much of the collaborations around this have happened almost by happenstance or like who sees what on Twitter. And it would be great to try to think about ways to centralize those more cohesively. Certainly the open oximetry project out of UCSF is one kind of attempt toward that. Although there's so much happening that isn't yet centralized there either, but that's one platform I know of where people are trying to come together with sort of an open invitation to others working on this. I think about this question a lot though because I do wish, I mean, I wish this was part of an FDA approach actually and I am grateful for chances like this to try to have interdisciplinary conversations and I think so much more and the research opportunities and experiences that have been posted in the chat here are so great to hear more about and think more about. But there is, there's so much kind of cobbling together or people, one thing I felt at the ISO meeting was that the industry and FDA people represented there, that was kind of their job. They were being paid to be there and then the professors and social scientists who were there are kind of taking time out of whatever else they're doing to try to make a bridge and I think that having structures where people kind of come in on equal footing and have some kind of comparable resourcing and of course, it's not just thinking about who's in the test groups for these objects, it's who's at tables like that. It's like who are the experts who are gathered and seeing diversity in those conversations is like inseparable from thinking about diversity within test groups. And so I would really love to see more platforms for that that weren't just improvised. In the meantime, I'm glad for these chances to improvise. Yeah, I think some sort of central group that is interested in equity center design that's able to bring these diverse disciplines together, professionals together would be amazing. I think we're in a different, I hope, I feel that we're in a different place now three years after our publications, then we were in 2020. Dr. Brandon Thomas brought up, heard the challenges she faced in publishing her paper. We had similar challenges publishing our paper with very harsh reviews essentially saying like, this is bogus, right? Like, there's no way this can be true. And so it can feel incredibly, you can feel very isolated in that situation where you feel like this is huge. People need to know this and yet nobody believes you. And I hope that we're in a different place now where individuals are able to come forward and shine light on these immense problems. But at the same time, having larger groups, multidisciplinary groups to be able to share these things, to have other people be able to kind of poke holes but at the same time work together to address these holes and then come out at the end and say, wow, like, this is real would be immensely helpful. And it feels like we're in a space where we can create those things and work together. But we need a lot of support to be able to do it. Thanks so much. Our next question is estimate like practical solutions to the known kind of disparity in what pulse oximetry is able to find for darker, pigmented people. Dr. Valley, I think you actually spoke a little bit to this in your talk. You had mentioned maybe increased use of arterial blood gases or perhaps different cutoffs of pulse oximetry, perhaps even different designs for different pigmentations. Could you, I guess both speak to sort of what you imagine could be the best solution as like a work around until there is kind of equity in the device development itself? Yeah, absolutely. I think it's an interesting situation, right? Because much of the discussion these days is taking race out of medicine, right? Whether it comes to primary function testing, whether it comes to how we test kidney disease, it's all about taking race out of medicine. Pulse oximeter is a situation where we're actually talking about like putting skin tone into the equation when you're making clinical decisions, which is interesting when you think about it. I'll say this, there's only one true solution and that's a better pulse oximeter, right? Like everything else, like you said, Fraser is just a work around. And they're kind of crappy work arounds too, right? Like when we talk about getting more arterial blood, it hurts a lot. It requires technical expertise. It's not like going to the doctor and getting your blood drawn after your visit. We specifically have to sample blood from the artery. That requires someone with a little bit more technical expertise. Like I said, it hurts. If you're in the ICU and you need a lot of arterial blood, that means you either need to get poked a bunch of times or you need to have the arterial castor placed. So again, we're just trading one bias for another when we're doing that, because then we're exposing individuals with darker skin tones, black patients to needing more arterial blood drawn from them. But where we are right now, that's unfortunately one of the work arounds. I think the second work around is just saying, hey, we're not gonna tolerate a blood auction level of 88% in a patient with darker skin tone. Maybe we have a threshold of 93%, 94%. Again, it sounds good. We don't really know the harms of providing too much oxygen to someone. There are some theoretical harms there as well. And there are also the health services challenges there too. Like if you are saying someone with an auction level below 94% is abnormal, well, maybe they stay in the hospital longer. Maybe they have to go and get oxygen. There are a lot of logistical challenges there simply because the pulse oximeter doesn't work as well for those individuals. Maybe just to pick up on some of that. One of the things that I found most striking at the FDA hearing, there was a physician who presented on an incidental finding in the course of trying to look at ABG pulse ox discrepancies. They also found that ABGs were used significantly less in black populations. So doctors are using these assessments differently already. And I think that's an example where we really see how unconscious medical racism of practice interacts with the ways those inequities can be built into devices. And so, so it would be, yeah, so just to mention some of the approaches looking forward because some people have inquired about like, well, should there be like two different devices when you go into a hospital, for example? Personally, I would love to see other approaches first because I feel like our country is never done well with separate but equal. And I just don't see how, I can imagine a lot of ways that that could go wrong. And so many people identify in multiple ways. I would love to see a device for the pulse ox that is actually able to work for everyone. And from my conversations at MIT with engineers, it seems like there are multiple approaches that could be effective for that. So I mentioned photo acoustics. That's one kind of very new area of research, but if that could be applied to turning LEDs up and down, sort of like an algorithm before the other algorithms, that's one interesting possibility that I've heard discussed. Another one that from what I understand would increase cost more. So that makes it kind of a heavier lift to incentivize, but multiple sort of color wavelengths. So if there were like different filters on the LEDs, apparently according to people who specialize in optical sensing and make medical devices with industry, they say that that probably would actually like very significantly reduce, if not eliminate the errors, but super back of the envelope, it might increase manufacturing costs about 20%. So again, like how do you, you can't ask industry to invest a bunch of R and D that doesn't exist yet to make a product that's gonna lose the money. And so again, there's lots of things where market solutions don't work and especially with global access issues, I think there's lots of dimensions and nuances of all this that really need to be discussed, but just to say there are multiple approaches that people are thinking about that maybe, it would be great to get a chance to kind of weigh and see how they unfold before resorting to something like, different devices for different people. Okay, and the interest of time, we probably have time for one quick wrap up question, which actually there's like a ton of love to ask you guys, but like the loss like technologies and the press around Thomas's wired piece and kind of biometric data and how that relates and stuff. But I guess the quick question I'd ask you is what was most surprising just in the general process of the last three or four years, what has been most surprising in your research and what you've done? I could start. I mean, I'll be honest, just the fact that this existed. I mean, I can't stress how embedded pulse oximeters were into daily practice, right? Like every day, all day, we just trusted the values and we often use like the mind blown emoji, like that was us when we saw this. Like, I couldn't believe that this could actually be a thing and such a big thing. And to just know that like how many patients I could have, I probably did harm, right? Throughout my medical career, because I trusted this device, it just blows my mind. The reactions of doctors, it's been amazing for me to see kind of the full spectrum but how many people are really coming into advocacy roles through looking at this data. And maybe there I can link back to another question you'd mentioned, Frazier. It was a doctor of medicine at Yale who reached out to let me know about the Hewlett Packard approach back in the 60s. And that did really shock me. I had no idea when I first wrote the Boston Review piece that there had been a device that like really solved for this actually pretty like really well. And it was interesting to me in the FDA conversations that I've had recently, there was a moment where they kind of paused to debunk the myth of the Hewlett Packard machine. And but it's not a myth, I have one. And so that was interesting to me that the history still feels potent enough to be addressed and to like bring up a reaction like that on the part of officials. But it also just reminds me of like what is possible and something I just learned very recently someone reached out and emailed me and said Hewlett Packard didn't just test that device on black individuals. It was a black engineer who sort of ran the project who wasn't written up in the company newsletter. And so I think they were really talking about it. The diversity of people at the table and not just on the table as they say in STS, but thinking about how to bring different kinds of experts across all different kinds of lines of difference in conversation for addressing these issues. I think it's really, I don't know the sentence where I'd like to leave this conversation. Thank you. And with that, I think we're done for today. Thank you so much for our panelists. This has been, I thought a great discussion. Yeah. Yes, thank you. Thank you, Drs. Valley, Warren Thomas, Tessima for this good conversation and to everyone who joined us today and we look forward to resuming in the fall with the next series of health policy and ethics conversations. Thank you all.