 Hi everyone, thanks to the organizers for the invitation to speak at this meeting. My graduate student just sent me an article from PNAS showing that parole approval goes down rapidly right before lunch. So with that in mind, I'll try to move quickly through this. A lot of folks have talked about economics, so I'm happy to dive a little bit deeper into that topic and see what we can come up with. Let's see here. Okay, just a little bit of a primer on cost-effectiveness, a couple slides, so we're all on the same page. Why do we do cost-effectiveness analysis in health care? It's basically because there's not a functioning free market. You know, when you go to a cafe, you're the one making the decision about what to buy, you have a pretty good idea of what it's going to taste like, and you're the one that gets the benefits or the harms. In health care, none of these things are true. Patience is not the one making the ultimate decision. They don't have full information about all the benefits and harms, and oftentimes they're not the ones paying for it, and that's really why this field of economic evaluation in health care has seen such rapid growth. Another point is that there's a lot of technical terms, but when we talk about economic evaluation and cost-effectiveness, we are not talking about saving money. I hate to break it to you. We're talking about spending money more efficiently. Getting more for the amount of money we spend. Sometimes that may mean you end up spending less money, but our goal is not to decrease the amount of money we spend. Our goal is to improve health with a given amount of money that we have available. So that's really value is what it's about. And weighing these two things, how much health can we generate with a given amount of money? So we're always, all these economic evaluation approaches essentially take heavy consideration of health benefits, taking into consideration epidemiology. Cost just happens to be something we throw in there. So you'll see that more when I go into one of these models that's been published. Just one other thing is the standard metric in the field is a quality. Now we can talk about, we can measure clinical events, we can measure life expectancy, but we want to also consider quality of life. And this metric quality-adjusted life here just combines quality of life with length of life. And that's basically what we produce in healthcare, right? We don't make cars, we don't make battleships, we make length of life and quality of life. We're trying to increase those. So that's why people like the quality. It really kind of encompasses those. But we also measure these other things too. And then there's a standard metric we use called incremental cost-effectiveness ratio. Again, that's basically how many qualities can you buy for the amount of money you spend. It's kind of like miles per gallon a little bit. And a simplified way to think about it is if we look at a difference in cost between what we're doing now and what we might do in the future and the difference in effectiveness, which would include both benefit and harm and just graph that, we can see in this cost-effectiveness plane some of it's pretty straightforward. You don't want to hurt people and spend more money. If you can save money and help people, it's obvious you should do that. You don't need an economist to calculate anything. Most of the time we're in this upper right quadrant where it's going to cost a little bit more money, but we're going to have better outcomes. And that's where we spend most of the time. And within that space, we can talk about the cost-effectiveness ratios. And Dr. Sung mentioned how much should we spend to get one year of perfect health? Dr. Sung mentioned $50,000 for quality, which has been a pretty standard metric. I think in the United States now, it's changed now. People looked at it a little more closely. It's probably closer to about $100,000 for quality. World Health Organization says three times per capita GDP. So somewhere around that ballpark allows us to at least say is it might be reasonable value or not. Okay, so that was a whole quarter's worth of health economics in like five slides, but just wanted to have a little bit of a laying the foundation there. Thinking a little bit about what do payors want, particularly in regards to genetic testing. We've done some focus groups and key informant interviews talking to folks about diagnostic tests. And I think one reaction they have is we've seen a lot of things that are vague that supposedly can do a lot of things sort of well, but there's not real good data for any of them. And just because you keep adding things to your assay doesn't mean it's better. They like concrete examples. Why do this? What's the clinical action that can be undertaken and why is that beneficial? We've got that in spades here. I think that's pretty obvious. And then another comment here again around this. This is more related to exome sequencing and it's an old findings, but the same concept of we want to see good, solid, clean evidence for what you're putting on your test in front of us. And I think they've been a bit dismayed from some of the things that are presented to them and they're asked to pay for. So that might come into our thinking. That obviously influences cost-effectiveness, but also it's a strategic issue. And this is just to note that when we do economic evaluation again, I just want to highlight that 80% of the work you do when you do one of these things has nothing to do with cost. It has to do with epidemiology, risk, attributable risk. How much can you improve outcomes? And you actually are doing a risk benefit calculation, quantifying it. And then you add in costs. So there's actually valuable information that can come out of these calculations. It also provides decision makers with information about their overall value. And I think really importantly also about uncertainty. And I think those are the three things they've been most receptive to in the settings I've seen in the United States. Important to keep in mind, economic data, it's just one piece of information that goes into decision making. So that's really important to remember just because something's cost effective does not mean you're going to do it for a variety of other reasons that might change your decision. So let's just step through this example and think about whether it might be cost effective, kind of in a qualitative sense, looking at some of the major factors that could be important. The first is one of the single most important things that can drive value is how frequent the outcome is. Well, in this case, it's not so much working in our favor. It's 1 per 1,000, 1 per 400, depending on what you're looking at. Number needed to test 400. It's not terrible, but it's not necessarily working your favor too much, fairly rare. I think the fact that this is such a severe outcome is really important, particularly with tens, with the mortality effects can be really important. And also the long-term sequelae over the lifetime can really be important in influencing the value here. The other thing to think about is what's your alternative to genetic testing? Well, there are alternatives. We can completely eliminate carbamazepine induced Stevens-Johnson syndrome, right? Just get rid of carbamazepine. So you actually need to think about that. What does that mean? Is that an approach that makes more sense? It's a possibility. There are other drugs out there. And then kind of getting into the epidemiology and the strength of the association, it's really important here. I think everyone's kind of got the sense of it and mentioned it, that in this case, we have an unbelievable relative risk here with these markers. It's incredible. But it's also what it comes down to is the positive predictive value is not that high. So basically, if you're positive predictive values in the single digits, 90% of the time, you're going to do something to that patient, and they didn't really need it. They didn't need their drug switched. And in other studies we've done in pharmacogenomics, this can be a fatal flaw. Because if you're switching to another drug and it's not quite as good or it has a little bit higher side effects, that can quickly swamp out the benefit. So that's something that needs to be considered. And then around cost, I think the main thing here, obviously, I don't think there's a whole lot going on. It's mainly around how much is our test going to cost and also something around the treatment of the adverse event. And I'll come back to that in a second. There are a few other issues here. I mentioned alternative drugs. And just also, thinking broadly, how to, I think a couple of people mentioned it, how do other people react to this information, family members, et cetera. And these are probably secondary effects, but something worth thinking about. So fortunately, there's been some really nice economic evaluations conducted. And Dr. Sung presented their work previously, but I wanted to just dive into that just a little bit more. So this is the study out of Singapore. And I didn't pick this study to poke holes in it. I picked the study because I thought it was nicely done. So she looked a little nervous there. So you can't read this. I just want to show you, we build these decision trees. The authors include a lot of different options. The main thing I wanted to highlight, since you mentioned it, and that was that they included that option of just giving everybody a different drug with no testing. And that's really important because that's a potentially viable alternative. And included a lot of downstream effects, not just adverse events, but efficacy of the drugs, et cetera. So you can see that these can get fairly complicated. And then just thinking through some of the inputs. Basically, I kind of feel like almost everything people have talked about here, all the data, all of those things get fed into one of these analyses. So we start out with some of the economic stuff, but there's also issues around the fatality, the prevalence of the genotype, the strength of the association. All of that gets fed into here. Important things to consider are the cost of the alternative drug, which may not be real expensive, but is more expensive than the drug you start with. The cost of genotyping in this case was $270 US dollars. And then look here at the cost of treatment. I think you can see that I doubt the costs in the US are anywhere close to that. $17,000. That's like per day, maybe, in a US burn unit. So for this analysis, those were the relevant costs. And again, population frequency is going to be really, really important. Now, it might take you a sec to get your head around this slide here, but this is a sensitivity analysis saying, what happens if we change some of these parameters? What happens to the cost effectiveness? And on this axis is positive predictive value, which is a function of the association and the prevalence. And then the prevalence of the side effect. And here is the frequency of 1502. And you can see that they're highly influential. And you can kind of pick out a spot here in what general region might you be cost effective. And you can see quickly when your frequency of the variant gets low, it becomes much more difficult to show that it's cost effective. And I think analyses like this can very quickly give us a general sense of which populations might make sense and which populations. There's just no way that it makes any economic value sense to do that. And I think this type of thing would be very useful for policymaking and maybe helped inform some of the policies in Singapore. So there's been a few other nice studies that have been done, particularly in Thailand, looking at both carbamazepine as well as alipurinol. Generally found they were cost effective, but in this case, because the alternative drug was more expensive, it didn't really pan out that it was highly cost effective. It was kind of a reasonable value, but not a strong economic value. So that's something, again, that's important to consider. So what about the US? That's easy. I didn't find anything. Informal search, I didn't find anything. It's not real surprising. A couple of things to think about, just a big picture. I think there's about 30 million Asian-Americans in the US. And I don't know, Josh, if you had an estimate how many people were exposed at some point to a drug that might cause the syndrome. But let's just say, what if it was 5% at some time in their life they were exposed? We're talking $300 million. And you could buy a lot of things with $300 million. Nice cohorts, stuff like that. So it's important to keep that in mind. The cost implications could be significant. And even if it represents a good value, it might not be affordable. And then, again, I think just in terms of cost effectiveness, really some data on the incidence of the variant, prevalence of the variant, the risk or costs are likely much higher for the side effect. But the cost of alternative drugs might also be higher. So I think those are all things that would have to be considered for the US. So looking at payer policy, I just grabbed these off the web last night. Anthem, a very large insurance company in the US. And you can see here that for carbamazepine, they say that it is medically necessary. If there are evasion descent, and this is kind of a catch phrase, if there are no other alternatives to carbamazepine. So I don't quite know how you interpret that, but interpretation might be that, well, there's quite a few alternatives, so you don't need to test. But they're definitely open to the idea. And I think they're kind of following along, I would think, the evidence and the FDA labeling. And then here, from Aetna, just pointed out some other testing, SIP-2C-19, I don't know if I kind of messed up the pasting here. But for SIP-2C-19 for clopidogrel, they gave that a thumbs up. Alan, you'd be happy to hear. They said they'd pay for that, but they say they won't pay for warfarin. So they are looking at the data. They are looking at the evidence. They're making decisions. But they do consider this medically necessary for carbamazepine. So I mean, I'm not seeing real strong signals that payers are putting up an absolute roadblock here to paying for these types of things. And I think that's important. So some things to think about. The number one thing I would say is, for economics is epidemiology in the US, kind of what do things look like? I think the second thing is understanding how patients and clinicians respond to testing. Do they just start avoiding the drug altogether? Because I think that would be a general loss to society. And that might be able, you could probably do that with a study of several hundred, up to maybe a thousand people where you test them and see what they do. I think keeping testing simple and sticking with the best evidence is probably a good idea. Thinking outside the box, how do we get a cheap, fast testing platform in global health and doing things like economic prizes for developing such a thing where there's not enough market incentive to do so. I have a few slides on value of information analysis, which is a new approach health economists are using to put a monetary value on conducting generally large investments, large studies. We've done some work in cancer genomics with SWOG, looking at different applications. I think the only thing I'll point out is that you can get big differences. In this case, our work directed them away from EGFR more towards breast cancer tumor markers. I think the other thing I'd like to point out is that these numbers up here are in the billions of dollars in terms of societal value generated by a study that might be 20 million dollars. So we're looking at return on investments that can range from 10 times to 100 times the investment. I think the interesting point here is that it's a fairly rare occurrence, but yet I think there's still tremendous value so it might be interesting to look at that. And you could also use this for trial design. So in summary, I think there's pretty nice evidence from the studies that have been published that testing can provide good economic value. I think clearly need some evidence in the United States before we can make that type of assessment, particularly looking at subgroups type, different types of patients. And we need to be cognizant of overall budget impact and real world policy considerations when we're looking at that. So thanks, I'd be happy to take any questions. Yes, is there a NNT number that you use in economics to say that's a right number to allow, to go ahead with intervention? Yeah, the question is a right number needed to treat or number needed to test? Is there a threshold and the answer is no, because it really depends on what the cost consequences are and the health outcome consequences are. So it really is a case by case basis. But I think it matters for budget impact. One of the issues that comes up is the question of research. So you have a potential candidate marker, but you don't know enough about it yet. And you might discover in the future if you have enough study in a healthcare system outcome platform that a certain subset actually meets the criteria. So how do you leverage the research money in the context of actually getting these studies done to know the answer? It's a little bit of a horse and cart problem because if you require the stringency of utility before you even study it, you'll never get the study done. And this is a, in cancer research, this is a big issue. People are getting now genotype for everything, for every gene that they know about, but that's a research tool really. Yeah, it's a great question. And I think it's possible to think about a situation where maybe you have an assay that has one or two or three things on it that you're sure about, but maybe has some research pieces to it also that's maybe not part of the intervention. It's not returned to the patient, but it's utilized for research purposes. And that way I think it's easier to get the buy-in from healthcare systems, clinicians, et cetera, but is an extremely cost-effective way to combine implementation research with more applied basic research. I'm sorry, Mike, go ahead. David, so in the UK we've got something called NICE. And when you have a rare disease, the kind of thresholds that you use for cost effectiveness are completely different, are not used in the same way as thresholds for more common therapeutics in common diseases. So here we have an adverse drug reaction such as SJSTN, which is rare, it is a rare disease. So why should we be using the same threshold that we would use for common diseases, for something so rare? I think the short answer is probably that we shouldn't. So NICE is a technology assessment group and they're pretty strict, but they make exceptions as many are indicates. And my guess is that in the United States that people have a higher threshold for something like this. And so potentially you would be looking at going above 100,000 still being reasonable. It's not clear where that line might be, but I do think this is a special case. Yeah, this goes back to the question that Mark raised. And one of the things that we found useful in terms of using economic analytic tools and decision modeling when there's uncertainty is that the sensitivity analysis can identify which unknowns have the biggest impact in terms of the results of the model. And so in some sense when you have a whole list of questions that you could invest research dollars in by understanding which one of those has the biggest impact in terms of the answer that you're looking for, you can actually direct you to say we should do this study and not that study. And that seems to be useful. Yeah, I'd agree. And a lot of times those analyses can allay decision maker, policy makers fears. Sometimes we'll get hung up on one certain aspect of the policy and you can show them that well actually it doesn't have a big impact. Let's focus on something else.