 Hello, my name is Monica Wahee and I'm going to present for you this presentation which I also presented at the Fourth World Heart and Brain Conference on August 27th, 2021. The title of the presentation is, Data Collection for Healthcare Quality Improvement for Diabetes and Heart Patients. So here's what I'm going to talk about. First, I'm going to talk about trends that we've had in healthcare quality, like how do we figure out what is healthcare quality, especially for chronic disease patients. Next, I'm going to talk about patient-centered key performance indicators or KPIs, what those are and how we should probably think about them. After that, I'm going to talk about implementing the Plan-Do-Study Act or PDSA model for quality assurance and quality improvement or QAQI in healthcare. And I'm just going to quickly review. I'm not going to get detailed into the model because I know some of you are really into that model. I'm just going to skim the surface and talk about the implementation of that model and what it's done for healthcare, at least in the U.S. And then finally, I'm going to propose my new approach, which you've probably read about on my web page, which is a different way of looking at quality insurance and quality improvement in healthcare from the point of view of the healthcare QAQI department. So I have this new model called SPRAB. I guess that spells SPRAB. I'm trying to like make it make a word, but it's not really a word. But anyway, it helps you. At least you can pronounce it like PDSA you can't even pronounce. So I'm going to start the talk by talking about what we were thinking about in the 1990s. So I'm 50. Okay, I'm just going to say it. And when I was in my 20s is the 1990s. So what was I doing? I was a research secretary in a cardiovascular unit at the University of Minnesota. We're studying cardiovascular disease and hypertension. And so we were wondering like, how do we know if our hypertension patients are doing well? Well, obviously, their hypertension is controlled, right? But is there more to it? Because we were giving them like beta blockers, which if you ever take beta blockers, my mom took them, they make you feel like crap, like a lot of them do. A lot of hypertension meds make you feel like crap, let's just face it. And so if your hypertension is controlled, but you feel like crap, you don't feel like exercising, you don't feel like going out, then it's not just good to look at the metric of adherence to medication and that you're on that. So what other metrics can we look at? And that's what we were talking about quality of life. So the same thing with diabetes, right? We can look at blood sugar control, but at what cost? Are these people able to move around and do stuff? And so in the 1990s is when we were talking about evidence based medicine, like, you know, what is the actual measured evidence behind these treatments? But really what counts as evidence was the other thing we had to struggle with. Next, while we were struggling with those ideas in the 1990s, as the 2000s came on, we had these ideas of patient centered medicine, which is a great idea. Like who's going to argue with that? I'm a patient. I want medicine centered around me. So it's a nice idea. But like, how do you apply it? I mean, I'm different from other patients. All the patients are different. They're not uniform. And also research shows that patient satisfaction is not always consistent with health care quality. I've seen research where parents bring their kids a pediatrician, and they're like, my kid's sick, give me drugs. But the kid has a virus and the antibiotics are not going to work. So the parent is dissatisfied. But this is high health care quality to not give out antibiotics. So the metrics we collect must be important to patients. And they may come from the patients themselves, but they may not be a measure of health care quality. And also patients all feel differently. So how do we do patient centered medicine? So we were still stuck. So now I'm going to give you a little quiz. Okay, this is a big picture quiz about what the whole literature says about health care quality and cost. Okay, I said it's a quiz. It's kind of like a game. Okay, so for each row, you have to state whether the relationship is true, possibly true, false, or it's complicated. All my relationships are is complicated, right? But these relationships can be A, B, C, or D. So the first relationship is if you improve this, you drive up this. So if you improve quality, you drive up cost. Is that true, possibly true, false, or it's complicated. Then let's look at the next one. If you improve quality, you reduce cost. Okay, so is that true, possibly true, false, it's complicated. How about the third one? If you ignore quality, you drive up cost. Is that true, or possibly true, or false, or it's complicated. And then the last one is if you ignore quality, you reduce cost. Is that true, possibly true, false, or it's complicated. Well, let's just see. Okay, here's the answers. And this is again, the big picture of the literature. If you improve quality, you drive up cost. Well, this is not always true. It is usually true. This is because it is work to improve health care quality. If you've ever tried to do it, it takes like hours. And there's tons of paperwork. It's just work. It's good. It's good work. But it takes money and time. So it's going to drive up cost. Now the second line, if you improve quality, reduce cost. In some special cases, that's true. And that's when your money is literally flying out the window. Okay, so for example, in the US, we have Medicare fraud. If you reduce Medicare fraud, which is people billing Medicare for not health care for lies, okay, if you reduce this, you improve health care quality and reduce cost. Okay, but that's because Medicare fraud is really costly. So most of the time, if you're improving quality, you're on the first row. You're not on the second row. But if money is bleeding out the window, I love saying that about health care money bleeding out the window. It doesn't even make sense. But if money is bleeding out the window on the topic you're studying, then you can probably reduce cost. Now looking at the third row, if you ignore quality, well, this is what's ironic, you almost always drive up cost if you ignore quality. Because health care is kind of like a clockwork orange, like if anything goes wrong, it breaks, right? So if something goes wrong, like your ER doesn't work right or your ICU doesn't work right, something goes wrong in there, everybody sues you in the US. And everybody sues each other. So that drives up cost. Then finally, if you ignore quality, can you reduce cost? And the answer is, yeah, you can. What you do is reduce health care access. So that's how we do things in the US is that if anything costs too much, we just make it so very few people can have it. So if health care costs, as it starts to cost too much, we're going to just reduce access. And then if it is any good quality health care, then only like maybe the elite can access it. And so that way you can get to it. But pretty much, if you ignore quality, you're going to drive up cost. So then the question is what side are you on, right? Okay, so I mentioned KPIs, like key performance indicators. So let's say you're like, okay, I want to improve quality and reduce cost. But if you focus on reducing costs, I'll tell you what the problem is. Okay, so look at this slide. Let's think about somebody who gets in a car accident, but they actually get good health care, and they get put back together and discharged. Okay, so we start out with a car accident, and then they get in the ambulance, medical transport, they come to the emergency department, and then they go into surgery. Then they go to the intensive care unit to recover after surgery, and then they're discharged. So this is a happy story. It's a story with a happy ending. Okay, now let's look at where things are costly. Okay, things are costly in surgery, and in the ICU. And what I mean by costly is that's where if health care goes wrong, we all end up paying. So in surgery, if there's a mistake and there's a surgical infection, that's a problem. And also if they're catheterized in the ICU, and there's a nosocomial infection associated with that, that's a problem. So where you see the labels A and B is where health care quality can go wrong and really drive up costs. So how do you reduce that? How do you prevent that those things going wrong and the costs going up? Well, the first thing you do is make it so nobody is burnt out in surgery in the ICU. And you might be like, well, then I have to hire just tons of surgeons and nurses. And I would say, no, all you have to do is go back and reduce car accidents, which is at letter C. And that's where I am in public health. And, you know, hint hint Saudi Arabia, you know, see if you can do this. And then secondly, in the emergency department, if you have more efficient handling or somehow people don't have to go to surgery as much, you can do ambulatory surgery there or something, then you're going to reduce the caseload at D, where it's labeled D at the emergency department, you'll reduce the caseload at A, which is surgery. And again, people won't be burned out there. So it's about managing resources at C and D and doing things better at C and D in order to save money at A and B. So if you make KPIs to reduce cost, they're probably going to be on A and B's budget. But A and B's budget is probably going to go up. In fact, everybody's budget's going to go up if you try to reduce costs at C and D. So the problem is that if you're looking to save money from doing QA, QI and health care, the whole system saves money. But the KPIs can't be played. It's really hard to know where to place them. So you can place KPIs on the whole system. And if you don't like how the system's running, it's really not clear where to add money to improve that KPI, if that makes sense. Philosophically, improved health care quality reduces overall costs, like I just demonstrated on the last slide. But it's across the system. So we need to not use these KPIs about costs. It's just going to cost money. If your department's going to do something to improve quality, it's going to cost money. So now that you believe that, let's just focus on quality improvement. Because if we focus on cost reduction, the approach may cause health care quality concerns that in the end end up wiping out any cost savings, right? Let's just face it. We need to prioritize what is wrong in health care and improve it, okay? And all kinds of things are wrong in health care. So we need to prioritize the ones that are the most wrong and try to reduce those. And if you try to reduce the ones that are the most costly, often those are the most wrong, but maybe not the same. So it's better to just focus on what are your problems and make them go away. Because I assure you that will reduce some costs somewhere in health care if you make priority problems go away. Okay, now that I've convinced you that we need to focus on health care quality and not cost, how do you improve health care quality, right? That's actually a really hard question. So like I said, you can set system level policy, you know, like medical record to use, but how do you improve it? Like what KPIs do you use? And each component ranks slightly differently in health care, like you just saw on the slide, the ER and the ICU. So what exactly are you trying to measure? And you know, I talked about my experience with hypertensive patients, you know, increase the percentage of our hypertension patients on medication, right? Like let's say we're at clinic A and we're going to set a KPI that we're going to increase our hypertensive patients on medication to X percent. But what should X percent be? And what is X percent now? Like how many people are already on medication, right? And then why did we even pick hypertension in the first place? Because I was talking about hypertension from my past, but is that really the priority? And if you're like, oh, well, there's hypertension and there's diabetes and there's there's asthma and there's arthritis. Well, you know, each one, each metric you're following, each KPI has to be measured and it drives a cost. It doesn't cost nothing. Like people are going to have to do data entry. Somebody's going to have to do data analysis. Somebody's have to write a report. So even if you can figure out what KPIs you want, like how do you even do them? Well, whatever you do in health care, you're going to have to use two sources of data. Okay. And these are just philosophical sources. The first one is one you all are familiar with. It's administrative data. It's data created as part of a business process, such as health care billing, right? Because we have billing data, but also like medical records data. Whenever you're scheduling people for like an MRI or an appointment, any appointment, you know, medical transport data, you can tell I'm a data scientist, right? Like this is where I live is in this administrative data. Okay. But the problem is, you can't use this data for health care quality unless you apply a study design to it. So now we're going to move over to the right side of the slide, which is called research data. You can convert administrative data into research data. You can also just collect your own. So if you are collecting like a cross-sectional survey about maybe patient satisfaction, then that's research data because you have a research question. Or maybe you do a provider survey. You know, whenever you have a research question, you're collecting research data. And so if you have a research question about what's the percentage of your hypertension patients adhering to medication, I'm sorry, honey, that is a research question. And so you're going to have to dig around in your administrative data and put together research data. So that's called data abstraction. And I have actually a course online you can take in how to do data abstraction that I'll link you to. But you have to do that. You can't just take that raw data from the administrative data, from whatever database it is, and do something with it. You have to create a research protocol operationalizing. How are you going to take data from that and turn it into research data so you can do health care quality metrics on it? Okay. So pretty much your administrative data is just sitting there. And your research data is what you make to answer your questions out of that data. But also, remember, you can just ask questions of people. You can just do qualitative studies where you interview like clinicians and find out what's going on. So you can add to your research data from outside that's not administrative data. But you pretty much have to turn everything into research data if you want to answer a QAQI question. So the dominant approach in health care quality assurance quality improvement. I call it an approach. I guess it's a model. We hear a lot about agile as being a management approach or module. That's dominant in management. But for some reason, in health care management like QAQI, we have this other model called the PDSA. I did research on this. The PDSA is promoted by the Institute for Health Care Improvement, or IHI, which is from Harvard. And Harvard's really invested in this model. But the model is not based on agile or any sort of management model that is current. The PDSA is based on the scientific management principles model, which was popular in the 1930s through the 1950s. And I actually met an engineer who's really intelligent. He taught me a lot. And I said to him, what is this PDSA? And he explained that in automotive, you know, obviously, if you drive cars and you get in an accident because a car is not designed right, this is a big problem, or the car is not manufactured right in the factory. There's a factory problem. And so he said that when you are managing a factory, which is like a big thing that's got a lot of stratification, like you've got people at the top planning the whole factory, and you've got people working in the factory. So in that situation, you can make subtle changes in like workflow and stuff and prevent systematic problems with the car. But this is not the same as what I just showed you on that other slide where somebody gets in a car accident and pretty soon they're in the medical transfer then into the ED, that's like all these little factories right next to each other, these little business processes that are separate. It's not like one big factory where somebody makes those wheel and somebody makes this car door and eventually this is going to go on a car. That's not how it is. Okay, we don't know what's going to happen to this person. They're going into surgery. They just had a car accident. Okay. And so the PDSA model is based on a model that works in a factory setting, but why would it work in healthcare, which is a totally different setting. I just explained to you the conceptual problem with coming up with a management model based on something that works in a factory and trying to apply it for healthcare. Okay, so there's a conceptual problem with that. But aside from that, I just wanted to jump ahead and say, what does the PDSA actually do? Well, the first thing I have to tell you is what it does not do. Now, you'd think IHI, if they're going around and teaching all these QAQI departments how to apply the PDSA, that they would be thinking about all the things the QAQI department has to get done in that healthcare system. But the PDSA does not address a whole bunch of things that QAQI department has to do. It just is very focused on just a certain part. So here's what the PDSA does not cover. It doesn't tell you what surveillance systems you should set up and how you should run them in your QAQI department. And that's not trivial. Like it depends on what your issues are in your healthcare system, what you want to track. And the PDSA is not going to help you figure that out. What issues in healthcare should you prioritize? The PDSA does not tell you that. You have to do other things to figure out that answer. And then how do you set your KPI numbers? And how do you benchmark the change? So remember that thing about percentage of people who have hypertension on medication? Like should you choose that KPI? If you do, what is the number now? And what should the number be? And how do you figure that out? The PDSA does not explain any of that. And then also, you know, those of you out there into evidence-based medicine might be familiar with the levels of evidence, right? Like case control study, cohort study, and clinical trial, systematic review, meta-analysis. So those of you who are familiar with that will notice that the PDSA doesn't have any of that in it. And those study designs are about causal inference, like what's causing stuff. So if you can't use those, how can you figure out what's causing your healthcare problems or solutions? And how do you polish it? Like you can't. And so how do you do all the functions of a healthcare quality assurance quality improvement department and use the PDSA when it doesn't even answer these questions? So I'm going to tell you what the PDSA apparently does from the basis of evidence, from what I found in the literature, okay? So I'm highlighting this article called The Problem with Plan-Do Study Ad Cycles. And it's from 2016. And so if you remember correctly, since the 1990s, the IHI has been promoting PDSA and everybody's been trying to implement it. So we have a lot of projects that have happened all over the world, not just in the U.S., but all over the world. So we should be able to tell, like, what the PDSA is good at and not good at. And this article kind of summarizes what I have seen as well. And that is that the PDSA looks deceptively simple, like you've seen that diagram, that circle. It looks really attractive. But it's not an evidence-based approach. And so we don't really have any scientific evidence against it or even for it. Like, we just don't have any scientific evidence about it, which is a problem for me. Because I don't know how to implement it if we don't have that. I don't know how to replicate anything if we don't have anything that's been evaluated. So what this paper says is we know the PDSA efforts are expensive. And we do not know if the PDSA efforts actually help people meet KPIs or help healthcare organizations meet KPIs. We don't know this because this information is not in the literature. So the four things that they seem to be able to feel comfortable stating, the four findings, are this, which I put on the slide, is that PDSAs require significant time resources and staff investment. And if you don't make the huge investment needed, all of your work will be a total waste. So it's kind of like when I tell people you can't buy half a car. If you budget for a car and you only budget like half as much as the cheapest car, you're not going to be able to buy anything. And so the PDSA cycles are kind of like you better have a big budget when you start or you won't get anything at the end. And unfortunately, the PDSA isn't appropriate for most problems in healthcare. The ones that cross subsystems, like the one I showed you with the person coming in to the ER and going through surgery and being discharged, maybe if you tried to improve your emergency department function, like within the emergency department, you could do a PDSA, but you probably couldn't do it to cost all of those subsystems. And here's the big kicker is that even if you do the PDSA right, it doesn't always work, meaning it doesn't always get you to the answer. What was the cause and how do you make this rate you don't like either go up or down? This comes to the new proposed alternative model approach that I am proposing that I guess you would pronounce it SPRAB. And that is a different way of looking at QAQI in healthcare from a department perspective, where you just consider the functions of a healthcare QAQI department. So these are the five functions I have decided you all do, even though I don't run your department. I decided that you do S for surveillance and monitoring, P for prioritizing issues, R for research studies, A for action to introduce change and B to benchmark change. Now explain how all these functions work together. I know that if I were to go to anybody's department right now, they would have certain surveillance. Maybe you have like an incident database where you keep track of incidents. I know in radiology, they'll have a database where they keep track of errors, like if you inject the wrong dye, those kind of errors. I know that there can be databases of falls where they keep track of falls. And in any case, if you're keeping track of anything in some sort of database or on a spreadsheet, that's surveillance and monitoring. And that's work, right? So you have to decide what things get monitored and what are the rules, who does the data entry, all that like nosocomial infections, you're probably tracking that. Now P is for prioritize issues. So one of the things is that those things you are tracking are obviously prioritized because you're putting a lot of effort towards them. You do need to continue to prioritize things. Like if you're tracking something and it's not really worth tracking, maybe you should take that down. Also, when you're prioritizing issues, they don't just come from inside. If your patients start complaining, you can prioritize those issues. You can start tracking things, for example. So this is really a different function of the same QAQI department. On one hand, you're doing surveillance and monitoring. And on the other hand, you're prioritizing issues. And you might be doing that based on what you see in surveillance and monitoring. But you might also do it just based on other things like maybe your political leaders have come to you and said that this is an issue and they want to give you funding for it or whatever. The third function is research studies. And I want to emphasize the word research. A lot of QAQI departments tell me that their research is exempt from the IRB or from ethics boards. Just ignore that. I will tell you your research if it's research is probably not exempt because research is about systematically understanding what's going wrong. And the only way it actually gets exempt from an ethics board is if the ethics board thinks what you're finding is not generalizable. And most of what we find in healthcare quality is generalizable. If you have a slippery floor, people are going to fall. If you don't train people out to put catheters in, they're going to do it wrong. Like that's unfortunately systematic knowledge and you're probably doing it. So whenever you do a research study in QAQI, you need to read a research protocol. You need to do that because that's evidence-based medicine. And you need to put it through the IRB and maybe it'll be exempt and maybe it won't. But if you don't do this and you give it to me, I'm an epidemiologist in a biosatistician. I cannot make sense of your data. I cannot make sense of your study if we don't do this together. Like write this protocol, explain what we're trying to research, and then do it. Because I do causal inference. I'll tell you what's wrong. I'll tell you what's causing it. I'll even tell you what to do about it. Right? And we can even publish it because it's evidence-based medicine. So this is the big message from my model, is research is research. And if you need to do it, like your falls are too high or your nosocomial infections are too high or whatever, you know, your surgical site infections are too high, you need to do a research study and you need to answer these questions. Because why? You need to move on to A, which is action to introduce change. We need to make evidence-based changes in your system. And I cannot even tell you how to do that unless we get research from evidence-based protocols. Right? So let's say we do a research study. One research study we did on nosocomial infections, I was like talking to the healthcare system and I said, why are your infections so bad at your cancer center? And they said, well, let me go talk to them. And found out they weren't following the antibiotic protocol. So there, like all they have to do is fix that at the cancer center and they'll have better metrics. And so that's what I'm saying is action to introduce change literally becomes obvious if you do research studies. And in fact, you can write like another research protocol to introduce the change like a field study and then do B, which is benchmark change at the end. So what I have helped people do is go from, we know it's a problem, but we don't know how big it is and we don't know how to fix it, to actually studying it, fixing it and showing that it's fixed, like doing a research protocol and studying where it is and why it's that bad. And then figuring out what needs to be done to fix it and then restudying it, like fixing it and studying it again. And then you can like think about how many publications that is and think about how many people don't get sepsis. You know what I'm saying? And this was the failure of the PDSA. You really just couldn't get through all that and have any numbers and know that you did anything. And so my model is not deceptively simple. It's as complex as a PDSA, if not more. But it actually reflects what you're supposed to be doing in your healthcare QAQI department. It takes into account all the functions. And at the end, you do evidence-based research so you can publish, maybe even get promoted. So there are advantages of my model to the PDSA. As I just summarized, issues are tracked in surveillance and are prioritized, okay? But you can get prioritization from patients or whoever is complaining. Actual research takes place when necessary. We use protocols, the IRB, the ethics boards, evidence-based medicine. And we can do system-wide improvements. We actually have numbers we actually can publish. And it's just as expensive and complex as the PDSA. But we get more useful outcomes and we can move you along and make your healthcare system so much better. Data collected from records or patients must be part of a specific evidence-based effort when you use my model. So one of the things that grew out of the PDSA was this need to just collect tons of administrative data. Remember that? That's not research data. That's raw. And so it needed like people to apply research designs to it. But there was all this extra data lying around. And this is just not efficient. It's much better to just make a research protocol and just pull in the data you need from the administrative data. And also, if you're going to work with patients and use patient data, it is the most respectful to just take the least you need. And if you have questions for patients, it's the most respectful to really think about those questions. Have a real research protocol before you sit down and interview them or give them a survey. In the PDSA, there was so much data maintained from these administrative systems. And there was just this constant data collection, as was described in that article I was showing you. It was just not patient-centered. So I may not be able to tell you what is patient-centered, but I know collecting a ton of data about chronic disease patients is not patient-centered. That's not cool. Okay? And most of the time, if you're dealing with patients or providers or anybody, you're doing research, so you really should go to the IRB. You should just write a nice protocol and go to the IRB. It might not be exempt. It might be exempt, but just don't care about that. Just care about the quality of your research. So in my approach, the SPRAB or SPRAB, any data collected is part of a formalized protocol. Either it's a surveillance effort, which is ongoing, or it's a research effort where we're studying the original level of some KPI, or we're trying to benchmark a KPI, or we're trying to do action to change the KPI. Everything is written in a formalized protocol, and we just have to prioritize the issues for data collection. And again, that comes from your department. Your department sits down with me, and I can help you decide. But this way you can actually solve your QAQI problems across your big systems and not get stuck like in a washing machine in the PDSA cycle that never ends. So in conclusion, historically, there's been confusion over what KPIs to select for healthcare and how to do QAQI to meet them. And this has led to the use of this PDSA and this over-collection of data from chronic disease patients, and it's not really beneficial. And doing all this data collection under these PDSAs doesn't really take into account what the QAQI departments actually have to accomplish in the big picture. In my new model, the SPRAB, Functions of Healthcare QAQI Departments approach, it's more applicable. It's streamlined data collection to only the necessary data. And yes, it does take a while to get through a few cycles of like doing research and making action to introduce change and then benchmarking the change. But the good news is you can publish it and you really know what happened. You know causal inference and you can feel really comfortable with your numbers at the end. So if I've made you really excited about QAQI and healthcare, just contact me and I'll help you out. We can implement the SPRAB model in your system. And especially if you're sick of PDSA, definitely you want to call me. Okay, well, I really hope you like my presentation and feel free to connect with me on LinkedIn or to see my blog on my website, which is probably where you got this, or to follow me on YouTube.