 So I guess we can go ahead and get started or meeting here. So I'm going to introduce my friend and neighbor Brent James. His topic today is the best clinical result at the lowest necessary cost addressing unwarranted clinical variation. Brent's known internationally for work in clinical quality improvement, patient safety and infrastructure that underlies improvement efforts such as cultural change, data systems, payment methods and management roles. He has faculty positions at Stanford, Harvard Public School of Health and the University of Utah. So we'll let Brent go ahead and talk about clinical improvement. Thank you, Rogers. It's a delight to be with you. Thanks for the brave souls who actually show up in person. Presumably there are a few people listening in too. I want to make a particular argument for you today, an idea about the nature of clinical care delivery in the current era. The core of it sits around something called unwarranted clinical variation. I thought I'd illustrate the idea of clinical variation just to kick it off. Unfortunately, I don't have any good examples in ophthalmology. I rarely have interacted with ophthalmologists almost every other specialty in the world over the years, though it seems. I first came from the Harvard School of Public Health to Utah in 1986, took a job at Intermount Health Care. I was a fairly typical academic researcher. I was at the Dana-Farber Cancer Institute mostly doing big randomized controlled trials of cancer therapies. I didn't realize I'd joined administration until I'd been here about six months. In fact, it's fair to say I didn't know administration existed, all on us truth. Bumped into a guy named Steve Busk who was the head of finance at Intermountain, he built something called an activity-based costing system. That allowed you to track at a very granular level every little element of care that went into a case. He had become familiar, it was really Steve's idea, with the literature on variation in clinical care delivery, particularly Jack Wenberg, Elliot Fisher, Dartmouth Atlas sorts of things. They were comparing medical referral regions across the United States, Canada, and Europe and showing massive amounts of variation, the rates at which particular treatments were done. Population adjusted statistics, so you couldn't really attribute it to the patient populations, but truly significant variations, deeply troubling. I will come back to that idea in a minute. The trouble was as a care was so variant that it was pretty much physically impossible that all patients, even with gold card first dollar coverage access, could possibly be getting good care. The variation was just too big. Well, I show up in Utah, Bus Boom has this idea, okay, get it, there is a huge variation from place to place. He said, I wonder if there's variation in a single hospital. Now he had those financial data that were very granular, every lab test, every image and exam, acuity adjusted hours of nursing times, acuity adjusted six-minute blocks of physician time, especially anything from central supply. Of course, when he got me involved, I pointed out that he still has some limitations. Steve, I've looked at your data. It derives from financial data, yes, it reflects clinical decision making, but it derives from a financial model, you're missing major elements. I insisted that we form oversight teams who actually knew the care of physicians and nurses. We more than doubled the number of factors we were tracking. I insisted that we track every co-morbidity and individually stage it, severity of presenting disease for the primary illness, individually stage it. Every complication individually stage it and long term outcomes. Now, to my knowledge, this is the best that this class of research has ever been done. It's observational. Frankly, if it had been the VP for finance, I never could have funded it. It cost about $2,000 a pop, $250,000 maybe, $200,000. What he gave me was seven very experienced nurses that I could train as chart abstractors. With the full backup, you're doing an observational study, interoperative reliability, those sorts of things, but we were able to obtain very, very solid data. It was a three-step process. We call them quality utilization and efficiency studies. In the quality phase, we checked co-morbidities, severity of presenting illness and outcomes to say these patients are essentially identical coming in and going out. In the utilization phase, we said, all right, how many of what did you use? To perform this particular case type. And then the efficiency stages, we actually rolled it back for bus room in the cost, the cost of care. As it lists on the slide across a two-year time frame, we studied six clinical areas. I'm going to show you the data for transurethal prostatectomy, a very common procedure at the time. Gallbladder disease, total lipophthaloplasty, bypass graft surgery, permanent pacemakers, and finally community acquired pneumonia, very consistent findings across all of them. The short version, what we found was a massive variation. We showed that the amount of variation inside a single hospital was greater than the amount of variation that Dartmouth Atlas and others were showing across medical communities. We want to see real variation on the hoof just going to any single facility. And it will be there in spades. I thought to illustrate it to you, these are two things that our urologic surgeons said were essential when performing a transurethal prostatectomy. Neither were tracked in bus booms data, just in passing. Sixteen high volume surgeons. I've broken out the people who had, well, this was the mainstay of their practice that would have been doing one or two cases a week minimum. So they really experienced people. The little green square show true surgical procedure time from when they inserted their resecting neuroscope and started to cut prostate tissue to when they removed it and said they were done. It's very accurate, recorded in the anesthesiology notes, by the way. The little red circles are really the purpose of the procedure. How much tissue did you remove? That's the purpose of a terp to remove prostatic tissue. It grabs the prostatic tissue, removed. What I've done is array those 16 high volume surgeons in order of grams per minute. Just to give you a sense of the variability, procedure times the longest, the highest physicians, H and F, 90 minutes to get a good outcome on a standard case. At the other extreme physician K, 38 minutes. So that more than a 250% multiplicative difference, 150% additive difference. So fairly dramatic variation in surgical procedure time. Grams of tissue removed goes from a low of 13 grams for physician G up to a high of 42 grams for physician J. It's more than a threefold difference, 300% multiplicative difference. A crazy thing about it. You can see it in the slide. You can just look at it and see the effect. It's a very strong statistical effect. The longer you operate, the less tissue you remove. That was troubling. What in the world's going on here? Oh, the big one. The reason we laid them out this way, Hal Bourne was the chief of the urology teaching service at the time. He'd found a very obscure article in the urology literature. We were interested in a major class of treatment failure and tracking long term outcomes. Of course, you do this procedure for elderly men, benign prostatic hypertrophy, who have urinary output problems. They can't urinate because of obstruction resulting from prostate tissue squeezing in on the urethra where the urethra runs through the center of the prostate. Well, in that literature, a major clinical outcome tracker, some of these men will fail. They will require a repeat operation. And the research definition was within one year. So repeat operation within one year, which means an initial treatment failure. I found an obscure article that linked that to grams per minute. We validated that finding. You're actually looking at a clinical outcome indirectly. Those complications concentrated in these surgeons to the right, those three surgeons to the right. And was very strongly associated with grams per minute. So you're looking at failed outcomes associated with this. You can carry it a step further in the utilization part just to summarize it all. This is actually two costs of the hospital. Now remember, these are 1980s, 60s, and that's why it's so low. These days, it would be much higher needless to say. Physician M, $1164 to get a good outcome on a standard case. H was highest, $2,233. Twice as much money. By the design of the study, essentially identical patients, identical outcomes. That's why we really focused more than a two-fold difference in resource utilization to accomplish that shared aim. Along the way, you see the idea. I mentioned that we studied six different areas across the course of two years. That was a common pattern in all of them. In fact, some years later, I'm a member of the National Academy of Medicine. We had a then IOM Institute of Medicine committee funded through federal legislation. We were looking at variation in general. This turned out to be the single largest source of variation that you'll discover. And you'll discover it within any single facility. Oh, I need to extend it a bit. I've been picking on the physicians. We were able to track nursing practices, too. They're at least as variable as physician practices. The highest rate of variation I've personally measured in a career of measuring variation is physical therapy. Stunning variation. Amazing variation among physical therapists. This is not unique to the medical profession. Turns out this sort of variation spans all of the healing professions. Nursing, therapy, pharmacy, down the line, all right? It's not just the United States. You find the same sorts of variation in Canada, across countries in Europe, Australia, Singapore, around the modern world. You see that idea of variation in clinical practice, so I have a prediction I've never done it. But I think if I were to drop into the Moran IOM Institute and begin to measure common procedures, let's just say I would be stunned and amazed if this is not what I found. It's extremely rare to not find this. This is the common thing that you see. Do you react to this the same way that I do, others do, who make a career out of studying this stuff? What in the world is going on here? You see that idea? Now, one last thought before we leave the topic and start to go more general. What I was showing you were summary slides. One minute and I'll be right back to you. I was showing you summary slides. For example, that cost slide had more than 45 factors in it. Well, break them out. Look at the detail and you'll discover the most interesting phenomenon. There was no single instance where one surgeon was consistently higher, consistently low. Yeah, physician M on average was very, very efficient. He had two or three individual elements of care where he was the top of the group. Physician H on average was a really heavy utilizer. He had a couple of things where he was actually the lowest in the group. And if you looked at the detailed data carefully for any length of time, you were forced to conclude it wasn't a matter of choosing the best surgeon that fairly clearly best care was scattered across the group. The way that Maureen Bizagnano at the Institute for Healthcare Improvement in Boston says this, all teach, all learn. And what you discover is is that we as a profession, we as a group know more than we do as individuals. All right. And it raises an interesting question. How do I extract and share best practice across the group? Young lady, you had a question. Like one of the lowest grams per minute was also the most expensive, the H. Yeah, let's go back and look. So H was the most expensive, I and J are right up there. And H, I and J, I wish I had some, well, I can point at it to people in the room. So here's H right here. So he was one of the higher grams per minute. I ever read them in order of grams per minute left to right. All right, here's I, but look at J. So you're right. So what you're saying indirectly, it seems to me, is that association between picking the best surgeon doesn't necessarily play through, does it? And that's what I was trying to say right here. And that's what we discovered. Best care was scattered across the group. You want your patients to have best care? How can you learn from your colleagues experience so that you're not off in your own little world, doing your own little thing? If you say in my experience, by definition, you're saying, I don't believe in best care at one level, just in passing. Yeah, that's a really fun discussion to have. Well, turns out this is an example of a much larger category in the run up to the ACA, the Affordable Care Act, Obamacare, don't like that term. I testified four times to various congressional committees. Now the committee of record that really controls health policy in the United States at a federal level is Senate Finance Committee of all crazy places, testified to them twice. In one of those presentations, I was trying to figure out a way to take a truly extensive literature and explain it to a group of intelligent laypeople, if you call senators intelligent laypeople. If at the time, you did a medline search looking for articles in the peer-reviewed literature, documenting variation in clinical practice, the time you rolled out a little over 40,000 articles in a fairly well-focused medline search and amazingly extensive literature around the scene and surprisingly consistent too, across the board. I wanted to make it more functional though. What I did was break it out into four subcategories and I'm gonna give these to you and argue that it's a pretty good way to think about it at least at a policy level, health policy level or general level. The first, we've been talking about massive variation in clinical practice. The Dartmouth Atlas, for example, when you drive it down inside a single facility or even a single multi-physician practice, you see the same phenomenon amplified a little bit. The trouble is the variation is so high that it's pretty much physically impossible that all patients could be getting good care, even when they have full access to care. The second, well, it's even a little worse. If I had a poster child for that one to be Bob Brooks, Senior Professor of Internal Medicine at UCLA at the time, Chief Medical Officer at Rand, Bob and his colleagues said, well, maybe we can explain geographic variation by what's called inappropriate care. They came up with this idea of inappropriate care. Inappropriate care, what they really meant, that's where the risk inherent in a treatment outweighs any potential clinical benefit to the patient. All right. Remember, our primary maximum as physicians is first to no harm. It's the areas where you put patients actively in harm's way. You've done more damage than good through your treatment, at least on average, statistically across a population. That's the idea. They develop formal instruments for assessing that. They applied them. Two major findings from that body of research, of course, that expanded well beyond Rand. Number one, finding inappropriate care does not explain geographic variation. You go into a community that has a very high utilization rate, population-adjusted utilization rate for some particular treatment or procedure. You compare that community to a community that has a very low population-adjusted rate for the same procedure. On average, the proportion of care judged clinically inappropriate on a careful review by one's peers was about the same. No association. But there was a secondary finding, much more troubling. In that initial set of studies, the high-water mark was carotid endarterectomy, 32% of all cases performed judged to be clinically inappropriate. The risk inherent in the procedure outweighed any potential benefit it should never have been undertaken for a third of the cases performed, but it was. It gets worse. It's now 14 years old. The courage trial in cardiovascular medicine estimated that over half of all cardiac stenting is clinically inappropriate. Lest you think it's out of date, two years ago now, two studies in JAMA show that that's still persisted today. Over half of all cardiac stenting, risk outweighs benefit shouldn't happen. You see the same thing in a number of other common procedures and treatments. Cancer is particularly bad around these sorts of things. Surgery for mechanical low back pain shows about a 50% inappropriate rate. If you think number one was bad, number two, it just completely overwhelms number one. Nobody's done a careful synthesis, so this is a educated guess. I'd say a minimum of 20% of all care delivered in the United States, Europe, Canada, Australia across the free world, clinically inappropriate should never have been done. Does more harm than good. Deeply, deeply professionally troubling. Number three on the list, November 30th, 1999 Institute of Medicine's Committee on Quality of Health Care in America publishes a report called to err is human, you've probably heard of it. We'd done an evidence review on that committee. We'd found about 60 major articles documenting an unfortunate fact. Well, you remember our headline. We said a minimum of about 44,000, up to about 98,000 American sites here in hospitals where the cause of death was not their underlying disease. It was the treatments we used to address those diseases in a way that for each case to separate independent physician reviewers judged was avoidable. Interestingly, when we did that, we knew we were being conservative. A good thing to be in an IOM report. Yeah, I participated in research some years later published in 2011. The real care associated injury rate resulting in death, a minimum of about 210,000 preventable deaths each year, directly attributable to us. 210,000 kind of puts COVID in perspective, doesn't it? That's been going on every year, year over year for decades. You see, not a relatively short-term pandemic. This is a long-term endemic. Directly attributable to health care. It means that hospitals are somewhere between the second and fourth most common cause of preventable death in the United States. And that one really has common balance. You have to love that one. The idea of a hospital as a major public health problem because technically we are. Now, frankly, if we measured not number of mortalities, if we measured instead life years lost, you get a different picture. So let's just be fair about this one. On average, health care delivery ads about three and a half to seven years of life expectancy to every American. On average, some much higher, many of us not so much. That's on average net of this. The upside is dramatically better than the downside. You'd be an idiot to stay away from health care because of fear of care associated injuries. You know, that's the craziest thing though. I kind of get it. Nobody ever reads a whole IOM report. The whole second half of that report, the main point we were trying to make, we had valid science, strong science. You can drop those injury rates by 40 to 60%, typically without losing any of the benefit. That was the point. This was a price we didn't have to pay, but we did. In fact, just published a little piece in NGM Catalyst over the pandemic, 20 years of gains in patient safety have disappeared, which is another very fun discussion associated with the stresses of the pandemic. Number four is a twist on number three. We called number three injuries of commission where the care actively harmed. Number four, we labeled in a follow on IOM, committee patient safety, achieving new standard for care. We call them injuries of omission. Turns out there are a series of things that we know for a fact work that do work and work consistently and well, usually supported by very strong randomized controlled trial evidence, level one evidence. Okay, for things we know work, how well do we execute? This is sometimes called high reliability, if you've heard that term. How well do we execute for things that we know work? This was Beth McGlenn initially at RAND, she's now at Kaiser Permanente based out of Oakland, California. She identified 90 such treatments that we knew work. They tend to be non-controversial. Everybody agrees, yep, we ought to do that. Then she just measured for patients who had a clear need, clear benefit. How well did we execute? Two famous articles in New England Journal of Medicine, the first for adults, we managed to do it correctly, 54.9% of the time for children. The second article, 46% of the time. So the way to think about number four, yeah, we routinely achieve miracles. That's actually a really easy case to make. People alive today get better healthcare and better health outcomes, better lives in any previous generation of people living on this planet, and we make real contributions. But if we could perform not at a roughly 50% level, but something close to 100%, what kind of outcomes would we achieve? See the idea? It's really been a lot of lives and pain and suffering on the table is what that implies. Now, I want you to think about these in a particular way. In one sense, it's a pretty damning list, isn't it? The fact is, is we are the best who will deserve a scene that's set on the top line. What I say is, is care falls short of its theoretic potential. Realize it's our first job is to deliver the best possible care to every patient who seeks our help. That's our fiduciary trust, our fiduciary commitment, our ethical commitment to patients that drives all of our behaviors. There's another one behind that, though in its particularly appropriate in an academic teaching institution, we try to pass along our knowledge to the next generation. And as part of that, we want to pass along something better than we ourselves received. So the full mission of an academic institution, number one, that patient care obligation are top goal, number two, teaching, but number three, research. Well, the first step in figuring out better is to figure out where you fall short. That's how I want you to think about this. It's a list of opportunities, a list of where we fall short, a list of where we could be better. Here's my question though. Have I convinced you in a fairly fast, actually superficial way, we could be much better? When you carefully review this literature, you cannot escape the conclusion that we could be dramatically better. And if nothing else from my words this morning, and I hope you'll take that away, we fall far short of our theoretic potential. And it's within our reach. These are things that we could do that we could accomplish. It redefines what it means to be a physician, what it means to be a healing profession. It's part of our ethical commitment to our patients, to our profession, in some sense, to ourselves in terms of what we accomplish. That's the idea behind it. That's what you learn from studying this carefully. Now, it extends in a really funny way that you might find interesting. Some of you probably don't even know what. It relates back to a guy named Deming. W. Edwards Deming is a father of modern quality improvement theory. It turns out that unwarranted clinical variation causes waste of resources. The term that we apply to it today is quality associated waste. The definition as your clinical outcomes improve, it causes your cost of operations to drop. All right? So one of the ways I look at that list I was just showing you, what I'm actually seeing is waste, not just waste of human lives, but waste of financial resources at the same time. This idea of quality associated waste ties the pieces together in a very interesting way. Well, okay. So yeah, it's easy to find examples where this happens. For example, I was just with common spirit. The true injury rate they've found in their patients matches the current best literature. About one in four patients have at least one significant care associated injury as part of their clinical experience during a particular care encounter in a hospital or outpatient procedure. All right? One in four, 25, 26%. David Bates just published an article in New England Journal 23.6 is what he got in Massachusetts. The study that I helped publish back in 2011, 26%. You see that idea. Two big OIG studies, 25% apiece for Medicare patients. Well, that's what common spirit showed. They estimated that the additional healthcare costs to treat those care associated injuries was a minimum of $4,600 per patient. It roughly doubled their length of stay on average. For example, you get that idea of waste? Okay, how big is it really? We were trying to get a handle on that in 2010. We convened a panel of experts, international experts at the IOM, a classic evidence review body. Well, the summary line says it all. That's actually from the press conference where we released the report. What we said was a minimum of 30%, probably over 50% of all spending in healthcare delivery is quality associated waste. It turns out to be a dominant category for the cost of healthcare. Truth and advertising, I'm the guy who's built general models for quality associated waste when I apply the models, I get about 65% waste. What does that mean today? On 2022, outside of COVID, it represents about $2 trillion with the T dollars in financial opportunity. Do any of you perceive that the cost of healthcare is a growing problem for our society? Access to care, insurance expense is a problem for people needing healthcare. That over a third of all families who are suffering financially, a major part of their bankruptcy or medical expenses. For example, us burdening them with medical expenses. What would it mean to the people of the United States if we could reduce the cost of healthcare by half? It turns out that it's financially dominant. It kind of pulls together the administrative view of trying to make a financial goal of a practice versus the clinical view of best care for patients. And the really strong thing about it, the way that you solve the money is by getting better clinical outcomes. Put some money where it belongs behind better clinical outcomes. So the next thing we need to discuss, we know why it happens. The really short version, we could spend a few hours on this and it'd be great fun. The primary cause of clinical variation is complexity. What's called in that literature, clinical uncertainty, at well, complexity, increasing levels of complexity. It's increasing exponentially by the way. This is science and medicine continues to advance. But in a care delivery environment that relies on human memory as its primary mechanism for execution. Of course that happens in an unhelpful care delivery environment associated with perverse financial incentives and relatively low frontline transparency. Now, I'm not gonna go into this in detail in the interest of time. I just don't have time to do it. It's a great fun topic. Here's the thing you need to know. If you decide that you want to get better clinical outcomes and take money out, get financial returns out of the thing. If you fail to address this problem, you will fail. You have to get the diagnosis right to get the treatment right. There's a short version of this. Well, in that setting, yeah, just one more comment on it. David Eddy at Stanford back in 1990s, the first person to ever use the term evidence-based medicine in the public scientific literature. He was a thoracic surgeon by the way originally who branched off into really the foundations of evidence-based medicine, developed almost all of our formal methods for doing it. Other people popularized it. People like Sackett, Cochran popularized it broadly. But Eddy did the theory behind it. This is how he set up. The complexity of modern medicine exceeds the capacity of the unaided expert mind. Sure thing was never said. He proceeded to demonstrate that scientifically. Excuse me. Or said another way, if you rely on what are called craft-based approaches on in my experience, on functioning from memory, turns out that approach is not scientifically tenable. All right, it's the key idea behind it. Well, in that setting, we found proven solutions just to wrap this up a bit. Clinical management method. At a theory level, it comes back to an interesting idea. So we've got a number of different ways that we as human beings have come up with to deal with complex situations. The most common we use in medicine is subspecialization. You know, I've got a medical license in my pocket right at the moment that says it actually does say that I'm qualified to practice medicine and surgery in all its branches. Stupider thing was never said. As if I could keep track of all of that stuff. Well, yeah, so I focused just on internal medicine, let's say, still pretty broad. Well, internal medicine, just endocrinology, still pretty broad. Within endocrinology, just diabetes. Of course, I've got some friends up here at this institution at the University of Utah, specialized just in cystic fibrosis diabetes. And I can make a pretty good argument that that's still too complex. What's the old joke? You know more and more about less and less until you know everything about nothing. It's just subspecialized down. What I really wanna call your attention though is the second thing on the slide. In lean theory, it's called mass customization. And it's a fun idea. We've been using it in the house of medicine for at least 70 years. We didn't put a fancy name on it and try to sell it as consulting services. But we've been using it for a very long period of time. Here's the idea. Spilled around the semenoxymoron, the key to effective variation, effective variation is standardization. So in quality theory, we call it standard work. Standard work where your aim is to make it really easy to do it right. So it's not relying on human memory. But then you vary on the basis of individual patient needs. Spilled around the idea that every patient is different. We all know that. Anybody who's treated any patients ever understands that every individual's unique and different. Well, what if you took the common stuff that was the same, standardized it? So you didn't have to think about it. You can put it on automatic. And then that allows your most important resource to train the expert mind, that physician, that nurse, that pharmacist to focus on that relatively small range that really makes a difference for this individual. See that idea? We were the first group here in Utah to stumble on that, really. It's a big randomized controlled trial back in 1991. It's led by Dr. Alan Morris, who's still active in the community now in his 80s. Yeah, controlled trial, the treatment arm was a new artificial lung for acute respiratory distress syndrome. Kind of topical timely today. That's the mechanism by which most people who die from COVID die is ARDS. So we've really seen a hit on it. But the control arm was standard ventilator management. You can put these people on a mechanical ventilator. High end expiratory pressures. High oxygen concentrations to try to get enough oxygen into their blood through their portion of their lung that's not fluid compromised to keep them alive, basically. Well, you know, tell me an artificial lung, I won't go into its details. Basically, it was extra-cabrial membrane oxygenation. We thought to examine the control arm of the trial. Everybody to that point had just assumed that academic intensivists, well-regarded academic intensivists delivered ideal care. But we started to measure ventilator settings across the eight academic intensivists who were in that ICU as part of the trial. This was the first time that we demonstrated significant variation, same physician, same patient, morning to night rounds, significant variation, a unique model with ventilator settings, very strong physiologic data which gave you a chance to really see in to that particular box. We knew the answer to this. You do understand in trial design, if you don't deliver the care in a consistent way, you cannot causally link treatment to outcome. That's why we use protocols in the arms of trials. It's a key principle of trial design. We realized we needed a protocol for the control arm for ventilator management, a standard approach. We set out to build it. Today we might call it a guideline. I'll call it a best care protocol, a guideline. And wow, did we quickly hit some significant problems. If I were building out this idea of complexity, these are some of the elements that would be part of it. Turns out we have evidence for best practice, actual evidence, only about 20% of the time on average. Oh, it varies. Yeah, pediatric oncology is the best I've ever found. It's up around 50%. Where I used to work general surgery, it's really close to zero. Frankly, ophthalmology, up until a few years ago, you guys were really close to zero too. You didn't have an awful lot of randomized controlled trial evidence to inform your practices in many areas. You see the idea, but about 20% of the time. Have any of you ever built a guideline? I built well over a hundred. And you live this. Our best literature on it today are major specialty groups who are building guidelines. Saying how well is the evidence informed. But it's about 20% of the time. Well, you have a fallback, expert consensus, technically level three evidence. Eddie's the guy who studied that. If you review Eddie's evidence around expert consensus opinion, if your experience is the same as mine, it will completely destroy any faith you may have ever had an expert consensus opinion's ability to identify best practice. All right, it's basically a random function. We know that guidelines don't guide practice. The reason is we try to load them into people's minds and then expect them to apply them correctly from memory. That's been well studied. It doesn't work as a short version. You hit the old complexity problem. You get about 50% execution. When you do it that way and finally the last one on the list. You know this, I know this. No two patient server exactly the same. It has a profound implication. That means you can't write a guideline that perfectly fits any patient if that's really true. And it turns out empirically it is. Well, Alan therefore chose to use this idea of lean mass customization for the control arm of his trial. They developed, well, still to this day probably the finest evidence-based best practice guideline I've seen in my life. It was a flow chart 80 pages long. It averaged over 40 decision nodes per page. Well over something like 1200 explicit instructions on how you set a ventilator based upon a patient's physiologic presentation divided it out just like a good and intensivist will and expiratory pressures, respiratory rates, oxygen concentrations, so on and on the line. Right? The nurses and respiratory technicians have tracked through it the pencil in the physiologic data on a copy of the flow chart next to the patient's bed next to the ventilator in the ICU and the yellow highlight, the pass said followed and so you set the next expiratory pressure here. When you're doing mass customization though you realize your protocol is not gonna perfectly fit. The way that Dr. Moros said to the intensivist, he said, look, he said, if it recommends a set and you just don't think it's correct you can override it on a whim. You don't have to ask permission. You don't have to ask justification. Just do what's right for the patient. Your primary fiduciary obligation is to the patient. Give them best care based on your expertise but they didn't stop there. That's the key phrase. Anytime somebody varied they followed up for it. We met every Thursday afternoon for an hour and what we did was go over areas where people varied. It tended to be very normative across the group as you started to learn from each other. Argue with each other about very tightly focused specific questions and events. You see the idea. A funny thing happened. The first patient they tried that on patient number 29 in the series and ventilated the RDS patients they were tracking here to the left. They followed the guideline recommendations 41% of the time they varied more than half the time. But then they started that little feedback loop running took them eight patients across four months before they went over 90% compliance. Oh, here's the thing you need to know. In that four months they put more than 125 changes into the guideline. That is an interesting statement. That best of class evidence-based best practice guideline changed wholesale when it contacted reality at the front line coming up. It improved survival by about 450% just to anticipate. It did improve outcomes. We went from 9.5% survival in that moment to criteria of patients to 44% survival. First time since the RDS was defined as a syndrome back in the sixties that anybody had shown an improvement in clinical outcomes. Now interesting in the structure of the trial we couldn't derive that we applied it across ARDS and that started in a Martin Luther King hospital in Los Angeles and validated those findings. So a dramatic improvement in clinical outcomes as measured in lives. All right, I said that I built about 125 of these. This happened every time. We'd always start with an evidence-based best practice guideline. What it means is if you're gonna use these sorts of methods you have to have a built-in method, a formal consistent method to somehow tune your theory of the guideline to reality. But what you end up with is a validated guideline based on data, performance data. That's what you're shooting for here. So you see how data fits into the whole thing to help you tune the guideline and then measure its performance. How many of the guidelines that you use actually have validation data with them at this level? I have to tell you in my entire career of doing this at the National Academy of Medicine and places around the world, it only happens if somebody set up this kind of a learning healthcare system. It's the only place that happens. But you have to think of your practice that way integrating the science into routine practice. It's a basic idea. We called them shared baseline protocols. They're a strong form of lean or care process models. And here's just a quick review of the theory with them. Number one, you identify a high priority clinical process, right? Like ventilator management of ARDS. You build an initial evidence-based best practice guideline and they're rife with problems, poor evidence, unreliable consensus, yada, yada, yada. Do it anyway, it's just to prime the pump. The big heavies are steps three and four. You blend it into clinical workflow so it doesn't rely on memory. We had about 20 little tools we use, standing order sets, clinical flow sheets, patient worksheets, what we call the action list. So that if you just let it happen on full automatic, what you got was the guideline evidence-based best care. If you just didn't touch it, all right? But you blend it in so you're not relying on memory. Technically what it is is clinical decision support deployed into frontline care as a clinician interacts with a patient, not just the physicians. This is very important for the nurses, for the pharmacists, for the therapists. Just as important for them as it is for you along the way. If you do this well, it will make the practice smoother and easier. It will reduce burden. Yeah. Side by side with that, you have to build a data system. It's gonna track two classes of data. Number one, anytime somebody varies, you wanna know that they've varied. So you have that standing order set. That's a representation of a guideline, by the way. You decide that order just doesn't fit. You uncheck it, you cross it out. You're free to do that without subvariance. We flag it. Yeah, the clinical flow says next, we should go here and do this. You say, yeah, not for this patient. We need to go do this other thing instead. You're free to do that, but that's a variance. So you flag it. Then side by side with that, you embed a data system to track technically intermediate and final clinical and cost outcomes. Intermediate outcomes or process steps. Final outcomes, obviously end outcomes of interest around a particular condition. All right? It turns out there's a method. It's the same method you use to design randomized controlled trials just in passing. That's where we derived it, that you can use to design those data systems. And when you do, they tend to be very parsimonious and they tend to be the things you must have to deliver care. So it doesn't need to be a burden. Tends not to be a burden along the way. Step five is the funnel in the way I used to say it to my colleagues here in Utah. Ladies and gentlemen, it's not just that we allow or even that we encourage, we demand that you vary from protocol based on individual patient need. I've got strong evidence that I can't write a guideline that perfectly fits any patient doctor. That's why we have you. You will get as much scrutiny for complying with one of our guidelines too much compared to your peers as you will for complying too little. That's literally true. I can show you specific cases around that. If you're different from your colleagues, we have an opportunity to learn. You'd be surprised how often it's what they call positive deviance that somebody in your practice teaches you something better. So it's not negative necessarily just in passing. See the idea? And then the last step, you have to build that feedback loop. A formal structure, part of your operations, people's protected time to respond to variation data, look at outcomes data, best current science to keep the thing moving ahead. So to come back to what I mentioned earlier, yeah, survival improved from 9.5% to 44%. Interestingly, by this point, remember that idea of quality associated waste or costs fell by about 25%. We expected that by that point. Here's the funny thing, physician time fell by about 50%. This turned out to be really popular among the intensivists. Major practice, more efficient. It meant that you got to spend your time on things that mattered. And the way you do it is you take the scout work off the plate so you can focus on the stuff that really makes a difference and get it to work consistently correctly every time. Reliable support for your clinical teams is the idea. Now, just a couple of things I'm gonna wrap up Roger, hopefully not too far over. Number one, we count our successes and lines. At its high point within Intermountain, I could document rigorously document more than 2,000 fewer deaths per year. People who would have died who didn't with strong evidence by the way, randomized controlled trial evidence in many instances. I just wanted to show you, it works. It makes a real difference and you understand that mortality is kind of the tip of the iceberg, a dramatically bigger impact in terms of function restored, suffered, and diverted. All right? We could be much better. And this gives the truth to that statement. But we literally count our successes and lives. We could be dramatically better for our patients. Annoying that, I would argue that you have a professional obligation. All right? To achieve that level of performance on behalf of the people who seek your help. This was just a quick example. I'll skip through it really quickly. We started to deploy standardized care. This idea of mass customization into primary care around chronic diseases. This is a trial that we published in JAMA. Clinical outcomes got better. I just wanna show you the cost data. Primary care costs went up a bit when we started to tighten it up. Hospitalization rates fell by 22%. That's actually a pretty good surrogate measure for quality. That diabetic patient, that heart failure patient didn't need hospitalization for acute exacerbations of their disease because of dramatically better outpatient management. Is this the specialist outpatient procedures associated with their chronic diseases fell by 21%. Roll it up. Our costs, remember, for deploying that, this was about 160,000 patients in that trial. It's $22 per person per year, about $3 million per year. But our total medical expense fell by $115 per person per year. We took out about 19 million, netted $16 million in financial benefit from this little trial. We called it a team-based care at Third Generation Patient Center Medical Home. The takeaway, nearly always better care is cheaper care. I say nearly, it's not always, but nearly always. You can take out the money if you get your clinical care right. You can solve the financial problem. You can make our services much more widely available. For me, this is diversity, equity, and inclusion on the hoof right here. This is what gives it some meaning, some weight, some reality. But it depends on our performance to get that job done. I have to tell you, I didn't run these numbers the finance guys did. This was across Intermountain for four years. It's magnified because I didn't put the zero on the Y-axis scale, in terms of the slope of the line. This was our expected growth with population growth, demographic shifts, new technologies, conservatively. We decided we wanted to limit the impact of healthcare costs on the community, the consumer price index inflation plus 1%, less than GDP growth. So that healthcare year over year took a smaller slice of our total wealth as opposed to a larger slice as it continues to do. We needed to hit that black line. It means we needed to take out 13% of our total cost of operations as a system. This is the first four years. We actually did it within four years, not the five that we'd initially targeted. We took out the 13%, just under $700 million in cost of operations. Five big improvement projects. I can track every dollar of that back to a clinical improvement. Every dollar, all right? Well, why do I show you that? I want you to know that it's real. I'm gonna conclude at that point. I just wanted to share the idea that our future depends on our ability to execute together. Medicine shifted into teams, team-based care. The average number of physicians seen by a patient while hospitalized is nine for physicians. For other team members, nurses, therapists, pharmacists, well over 40. It's no longer a solo enterprise. How do we practice together? More important, how do we learn together? How do we achieve our academic goals of the best possible patient care, training the next generation, but at the same time, creating a new dramatically better world for the next generation? So that's what this story is all about. I think there's some useful ideas in this even for a department of ophthalmology. You'll be the judge of that. But I thought I'd just leave you with those thoughts. Do we have any time for more questions and answers? Not really. We do good. It's not depressing. It's invigorating, it's energizing. So I have a question. Yeah. So the second half of the Institute of Medicine report back 15 years ago was improving diagnosis, which wasn't really sort of grabbed onto by the medical community, but now seems to be having a bit of a surge of interest because all of your cost-saving measures are not terribly helpful if you don't have the diagnosis, correct? And perhaps maybe a explanation for some of the unnecessary care. What process or procedures are you suggesting to improve getting the right diagnosis early in the course of somebody's disease? So the best way I know to describe that is to tell a story. It happened at Intermount Medical Center. It was led by a fellow named Don LaPace. I mean, no Don, he was overall cardiovascular services. He's a cardiologist. The ACC, the American College of Cardiology, prepared a series of what are called indications guidelines for cath lab. And a cath lab, now Don kind of oversaw those for the system, four big cath labs. You do five main things, diagnostic cath, a stent placement in the coronary vascular tree, pacemakers, defibrillators, system maybe, nuclear stress tests. Well, it turns out that ACC had developed guidelines. They're fairly complex. What Don managed to do is reduce each one-to-one sheet of paper. And what they worry, put it to the form of a check-off of indications for when it was appropriate to do them. They arranged in the simplest had about 40 items on it. The most complex had about 90. It took both sides of the sheet. Now, there's evidence, by the way, about how the expert mind works. Even at 40, it's more than the carrying capacity of the unaided mind. All right. What he did, he did two things. He wanted to make the burden less. He went to the insurers in the area and he said, look, people hate pre-auth. If the cardiologist just checks off an indication, we should be able to use that as pre-auth. And they agreed. And it reduced the pre-auth procedure to one or two minutes. Functionally, what he was doing is he was getting the evidence in front of the expert cardiologists. I mean, this was a very respected cardiology group. These weren't slackers. There's a major research effort that they participate in to Brant Muellstein, for example. Major research effort associated with it. Big part of the university's training programs are associated with it. A lot of fellows and residents up there. Well, yeah, just check off an indication. What he really said is, you can't use my cath lab unless you do. He put some teeth into it. There was a certain amount of whining. Now, when they started, we had a couple of things in place. We already had a long-term outcomes tracking system that we built to support cardiovascular medicine. So we had an easy way to track outcomes because we'd invested in it, it was already in place. The second thing, it turns out that we were in the bottom quintile, the bottom 20th percent of the United States of America in terms of population adjusted use rates according to Dartmouth Alice. So we were already had some of the lowest use rates in the country. All right. When Don did that, utilization fell by 24%. About a quarter of the cases that we were taking to the cath lab were clinically inappropriate, saying it the other way. Our clinical outcomes improved in that long-term outcomes tracking system. We couldn't identify a single case where failing to do the procedure had resulted in harm to a patient or less an optimal outcome for a patient. Even the cardiologist had to agree that, yeah, we were over treating. So what was the mechanism? The mechanism, it all relates back to complexity. Okay, I've got this patient with a problem. What are the evidence-based indications? But by the way, I harass LePay heartlessly. It's part of my job as a surgeon. Don, holy cow, the ACC is kind of cardiologist in terms of the indications that they lay out there. Let's just say, if I'd been building them, I think it would have been a lot tighter. All right, they were early on the side of let's do it if there was a question, okay? At least the way Don had set it up, he could generate the data to answer those questions. So down the road as part of routine practice, you see the idea? So you're right, there was, I believe it or not in everything I've discussed, I did not include misdiagnosis. We believe that 10 to 20% of initial diagnoses are incorrect across the entire house of medicine. Of course, if you get the diagnosis wrong, it's not just that you fail to treat the real problem, you're gonna probably use treatments that harm because they're not aligned to a real need. It's not just the expense, almost anything we use has secondary consequences. You see the idea? The reason I don't list it yet is because I haven't seen strong solutions. On the other hand, for inappropriate care, the key factor is getting the evidence in front of the decision maker, fairly complex evidence in a usable form at the moment they're making the decision. If you're relying on, in my experience, relying on your mind, most of this stuff is so complex. I'm sorry, I don't care how smart you are. This was Eddie arguing that it's probably beyond the capacity of the expert mind. Well, we've seen that play through, not just for that cath lab. We've seen it play through in a number of other areas as well. Roger? It's gonna make an impact on all this, you think? Just the artificial intelligence. Artificial intelligence is gonna make an impact on all of this. So I have an opinion, so don't give it too much credence, just an opinion without evidence. This is my third pass of machine learning. It's gone through the Gartner hype cycle. This is pass number three. It's a particular analytic technique if I put on my statistical hat. And it has some very attractive features. Is it a useful tool for helping us deploy? Yes, it is. You know, at Stanford, at the Clinical Excellence Research Center, where I spend most of my professional time, I have a number of colleagues that this is their specialty. The reason they come and talk to me is they can't figure out how to deploy it. They wanna figure out how to get it out into actual practice along the way. Is it a useful tool? Yes. Will it replace your mind? No, it will not. It will still need wisdom, intelligence, guidance from a thinking mind, all right? It will not be some sort of a magic bullet. It will be like the next breakthrough in antibiotics that takes us a step up a little bit, but not the magic solution. So give me enough time and I can make a fairly compelling argument around that, by the way, with specific examples, but so I don't think, oh, I don't know. Interesting area for research, but it still is just that. An interesting area for research that's finding difficulty in application as it did in the first two passes, just in passing. Well, it may contributions, yes, it will. Is it the answer? I don't think so, probably not. Other comments or questions? Anybody online? I'd have to fire up chat somehow, but I can't see it, okay? We're probably out of our time, but I just want to thank you very much for a fascinating discussion of stuff, really stimulating and good at thinking, so our ophthalmology specialty. I hope not people thinking, Dr. Harry, should get you thinking.