 Good afternoon So I'm Aaron Edlin and I'm the Richard Jennings professor of law and professor of economics here at Berkeley And more important. I'm a member of the Hitchcock Professorship Committee and We are pleased along with the Graduate Council to present Professor Christopher Murray who is our fall speaker in the Charles M and Martha Hitchcock lecture series As a condition of this bequest, we are obligated, but we are also extremely happy To tell you how the endowment came to UC Berkeley It's a story that exemplifies the many ways this campus is linked to the history of California in the Bay Area Dr. Charles Hitchcock was a physician for the Army and he came to San Francisco during the Gold Rush Where he opened a thriving private practice in 1855 He established a professorship here at Berkeley as an expression of his long-held interest in education his daughter Lily Hitchcock Coyt Who is still treasured in San Francisco for her colorful personality as well as her generosity? Greatly expanded her father's original gift to establish a professorship here at Berkeley Making it possible for us to present a series of lectures With many extraordinary people over the years The Hitchcock fund has become one of the most cherished endowments at the University of California Recognizing the highest distinction of scholarly thought and achievement and the fund allows us To bring to Berkeley leading scholars and public intellectuals So we are glad to thank Charles Hitchcock and Lily Hitchcock Coyt for helping us to bring to Berkeley scholars like Christopher Murray We're extremely honored and pleased to have Professor Murray here with us today, especially as we're faced with global health challenges Dr. Murray's chair of Health Metric Sciences at the University of Washington and Director of the Institute for Health Metrics and Evaluation IHME His career has focused on improving population health worldwide through better evidence He's a physician and health economist and his work has led to the development of innovative methods to strengthen health measurement analyze the performance of health systems and Understand the drivers of health as well as producing forecasts For the future state of health many of you I'm sure became familiar with IHME forecasts early in the COVID epidemic as I did Dr. Murray has led critical analysis during the COVID-19 pandemic to understand its impact on Health systems and the population as a whole and the effectiveness of policy interventions to mitigate it and The White House European Commission and many governments and organization use his IHME's COVID-19 forecasts as a source of evidence Dr. Murray also leads the global burden of disease collaboration, which we'll hear about today It's an effort to quantify the comparative magnitude of health loss due to diseases injuries and risk factors by age sex and geography over time It has a huge network of scientists almost 10,000 7700 apparently and decision-makers from 156 countries Who generate annually updated estimates? Professor Murray is an elected member of the National Academy of Medicine and the 2018 Co-recipient of the John Dirks Canada Gardner Global Health Award. So without further delay. I'll present you professor Christopher Murray Thank you very much for that intro. It's great to hear the history of the Hitchcock endowment and lectures and Let's see if this presentation can live up to those expectations So I'm going to talk about the global burden of disease study and I'm going to cover a number of details It turns out this talk is timely because We're at the 30-year mark on this effort And there's actually a paper or a viewpoint a perspective in nature medicine next week That's really the same as this presentation So fortuitous. So first off, what is this thing I'm talking about? And I think at the simplest level I think of it as rules-based evidence synthesis for health or for global health More specifically what we're trying to do is provide everyone in the world a highly standardized set of measurements for every major disease injury and risk factor by each country large countries at the state or province or regional level and Over time so you get a time series view of health problems And that's a good basis for thinking about what are current or future challenges that may emerge The history goes back to the World Bank very interestingly because it was the World Bank in 1991 that initiated the study asked myself and Alan Lopez to undertake the first assessment Because they wanted to inform their world development report for 1993 the first effort at The Bank sort of having in their flagship publication one on health but they found out that there was no standardized measurement of health outcomes even death and Thus was born the idea that we needed a standardized approach and That initial effort was really for eight large regions in the world 100 plus diseases 10 risk factors five age groups and Over the years as demand for this from various types of decision makers has grown This effort has kept going And has become this very elaborate as I'll go into effort to quantify a very extensive set of outcomes Not only do we provide this as a public good so that you can go online And I'll tell you how to find it if you are interested And get a pretty nice snapshot of the last 40 years of health trends in any place for any outcome you're interested in but it's also produced a Actually ever-growing set of peer-reviewed publications. We're running now close to just under 500 a year from this effort and nearly 2000 since 2010 and or 2012 and that's a watershed I think for when the GBD effort became far More inclusive of investigators around the world In addition to making data available to everybody as a public good Publishing lots of articles in the academic literature We have as we've built this global effort global collaboration produced or helped others. I should say produce specific policy analysis on the burden of disease for various governments and here's a few illustrated for Singapore for Portugal for Norway on this other examples this slide other policy uses whether it's a lot of sort of forward-looking planning exercises Frequently countries use the GBD results and this is both for low-income middle-income and high-income countries And I won't belabor all all of them on this slide and a last slide of examples that focus on Nigeria which was one of the more recent ones where The analysis around the burden of disease in Nigeria was a part of a commission on health in Nigeria Which in turn led to the adoption of their new health insurance program. So a nice example of Harnessing the sort of evidence to make a case in a particular place For a set of policies that hopefully will eventually improve health And there's what's another interesting thing about the GBD is it's as much used in high-income middle-income and low-income countries It's not just in low-income countries or just in high-income countries. So we've seen quite a diverse set of uses I'll come back to India Brazil and Ethiopia at the end of this presentation Now if you're interested in what I'm going to talk about you just go online to health data org and Everything that we have available you can query instantaneously with quite a lot of options to explore the data in a tool called GBD compare and This is just an example of a tree map that shows you the results of health loss in the world Broken down into non communicable diseases in blue and the communicable diseases in red and injuries in green This tool by the way is in 14 languages for those of you who want to explore it in other languages as well Now what differentiates the GBD from other Either disease specific or country specific efforts for which there are many Is the adoption from the beginning of some core principles some of which people find they don't like But there's certainly enough that do like them that they continue to use them and the one that's perhaps the most controversial is The notion of making best estimates So unlike some institutions in the development world or in the international arena That only report when let's say a government reports data We believe that you it's more useful for decision-makers to have at hand your best estimate Regardless of how much data is available in that place and the premise there goes around to the idea that when you have No data, it's very easy in the policies Setting to equate no data to there is no problem and perhaps one of the best illustrations of this was after the first round of GBD results that came out in the World Development Report 1993 and then in publications in 1996 We found a substantial burden due to mental health disorders Which had not been on anybody's policy agenda outside of high-income countries Now that there were survey data from let's say 10 developing countries at the time But we then make you know statistical estimation for all countries and said although there's no data in your country It's very likely you have a substantial mental health problem that actually had a lot of impact on Government's willingness to collect new data, but also to fund or think about policies around mental health Now this best estimate mindset has been adopted by many actors in global health. It's quite common at WHO now In part because of the GBD at UN AIDS at UNICEF But there are other institutions the World Bank's a good example where they do not do this except in certain areas So you have a very You know a mixed view of this best estimates principle, but it's core to how we go about our our work The second principle is that it's very useful for more senior decision-makers and For the public actually to get the comprehensive view of health It's hugely valuable when somebody studies one topic in detail and can really give you a lot of insight into let's say COVID or into Gender-based violence or tuberculosis or cancer But it's also very useful to have a comprehensive accounting and so we have always sought to make our assessment truly global all countries as well as a comprehensive set of diseases and injuries so we have a cause hierarchy which meets the Criteria of mutually exclusive and collectively exhaustive so we every outcome is captured in that framework and then we have a reasonably comprehensive but not As the same approach on diseases for risk factors because they risk factors can intersect and affect the same outcome. So you don't have that same necessity for a comprehensive mutually exclusive collectively exhaustive list So, you know having that comprehensive view gives you lots of insights You would not otherwise have and here's just an illustration. These are these tree maps that are on that tool GBD compare and It's our apex measure in the GBD process called the disability adjusted life here It's a measure of healthy life lost and it's showing it for the world back in 1990 44% of the burden of disease was from Noncommunicable diseases in the world go ahead two decades. It's up to 55 percent and go ahead to 2020 You know just in ten years you go up to 62 percent But the orange box there the communicable maternal and neonatal causes is Substantially larger because of the presence of COVID. So in the absence of COVID we'd be up to 66 percent of all burden is now NCDs so you get this perspective of this rapid epidemiological transition that's occurring. This is at the global level It's also occurring pretty much everywhere outside of Sub-Saharan Africa at this very accelerated pace particularly South Asia and Southeast Asia and Latin America and it is something that you only get that view from a comprehensive accounting Another way to see that incredible speed of transition is to look at that same sort of detail But now by age group and this is just deaths not years of life lost So the y-axis is millions of deaths. This is back in 1990 the red colors are communicable diseases the blue colors are the cardiovascular diseases and purple is diabetes and green are the injuries and You can see that there was still a lot of death in the world under age five The first bar is the first week of life than the next three weeks of life from the next 11 months Huge numbers of deaths in children go to 2020 and the number of deaths under age five Except in that first week of life is dramatically reduced so extraordinary global progress and we see this huge increase In deaths at older age and now you can see in in the sort of pumpkin color at the top the big burden of COVID that emerged in 2020 so this transition in the world is Rapid it has profound implications for most middle-income some upper-low-income countries and Comes from this comprehensive view that we try to foster Third core principle of how we've gone about this 30-year enterprise is a heavy heavy emphasis on measurements that are comparable Over time in the same country and across communities across states across racial or ethnic groups across countries and Comparability of measurement may seem obvious, you know I think economists always aspire to a comparability of measurement in metrics like GDP but frequently in public health there hasn't been an emphasis on Comparability of measurement because the coming from a sort of epidemic Background the thought was it only mattered if you sort of got the the direction of Travel correct. It didn't matter if you were actually making a comparable measurement as long as you could tell what trend was And I think the GBD has changed that view quite profoundly so that now we do Ensure comparability by Re-estimating the entire time series over time Every time we have an update with new data or methods so that you can always make a meaningful comparison over time so to illustrate that Here are for Six regions the world first column is Latin America than the Caribbean six regions in Latin America the Andean countries Central Latin America Tropical Latin America and then for comparison the Middle East and North Africa and this is diabetes top row is deaths Middle is years of life lost bottom is years lived with disability that measure of morbidity and You see the different time series produced with different rounds of the GBD 2016 round the 2017 the 2019 the 2021 round and in some cases they don't change very much Except we just add data points But if you look carefully in a place like Middle East and North Africa the death estimates as well as in tropical Latin America changed Even back in time to 1990 and that's because new data came to light that told us much more accurately What was diabetes deaths back in their earlier period of observation? So it makes many users somewhat Confused because the past is changing, but it is our commitment to keep up with the evidence and re-estimate the time series Our last principle is to provide a series of summary metrics For the burden of disease and those are not just about death a lot before the GBD came along all global policy discussions in Global health were framed around death numbers or death rates There was completely ignoring things that largely caused morbidity or disability think mental health musculoskeletal disorders some neurodegenerative diseases Those were being ignored because we weren't counting loss of health and so a big emphasis from the beginning from 1991 on the GBD was to try to capture in measures The totality of health experience and so we have all the traditional measures disease incidence and prevalence and risk factor exposure But we also produce these summary measures years of life loss due to premature mortality Years live with disability, which is just disease prevalence Times the public's view of severity more on that in a moment We put the two together and that's our apex measure called dalleys and then we complete take all that information and compute a positive measure called healthy life expectancy Now people always ask well, how do we compare apples and oranges? How do you compare time spent with dementia versus time spent? you know sick with COVID or time spent with You know major depression and the way we do that is we ask the public and We have now collected in the last I'd say 12 years more than a half a million survey respondents around the world a Big chunk of those by the way in China, but still many countries have been sampled and The interesting thing about these questions where people are asked paired comparisons and which one represents a higher or lower state of health the results are extraordinarily consistent even from rural Africa to the United States very different socio-economic context and yet the ordering of health states or The the healthiness of health states if you will that are derived from these surveys remains remarkably Consistent around the world, which is helpful for what we're trying to do another principle that we Try to emphasize a lot is sort of ground-truthing the results like yes We try to capture all the data. We have quite sophisticated Bayesian statistical models but at the end of the day we want to make sure that it reflects some ground truth and We try to address that both through various metrics of quality of the data to make sure that what we're doing or the data sources going in aren't In some ways have a some fundamental challenges and we have a very large network of collaborators I'll speak about who help bring their knowledge of local context Their knowledge that that particular survey or study or data source has some under-recognized bias and we then have produced a variety of Internal consistency checks that go along with this commitment to face validity So we require because we're looking at this comprehensive view We require that all the deaths due to a list of causes add up and equal all the deaths overall Now that may seem obvious But until we came along that was not the case and if you added up what WHO said in the late 80s There were three times as many deaths being claimed on the cause specific estimates than the number of people actually dying and So simple internal consistency requirements which turn out to be quite complicated to sustain statistically Make the modeling quite a bit more complex, but also we believe that triangulation Leads to more coherent estimates We do this for many other things as well like we take the large body of surveys on anemia and The much smaller body of studies on anemia by cause think about malaria or hemoglobinopathy or iron deficiency And we make the cause specific anemia is add up to all this bigger body of evidence on on total anemia now some of these examples there of when we take Models and data and models by by disease and then we force it to match what there's more data on which is all caused mortality we then end up having to adjust the cause specific estimates and here's a Colorful graph called a heat map just showing an illustration of this by age By location on the rose for tuberculosis where that exercise of triangulation of internal consistency To make it all work We have to reduce the estimate the first round estimates of mortality in armenia an age group 10 to 14 by 16% that sort of orangey box and then in others that that correction factor is really quite small We also go through a very elaborate Review and governance structure for such a complicated multi-person effort So we have across the eight and a half thousand people that work on the study globally a group picked from around the world Across different disciplines that is internal to our collaboration and they govern the study And so this is called the GBD scientific council and any change in methods must be approved by this group And so there's you know every month we meet and go through presentations of either of new data or methods or analysis And that's the sort of internal review process there's a External group independently funded by the Bill and Melinda Gates Foundation called the IAC That meets every six months to review our work And so we go through a two or three-day presentation of progress on different aspects of the GBD and they give useful critique and Suggestions for improvement in the future and here's just the makeup of the IAC. It's chaired by Peter Piot Who has stepped down from running the London School of Hygiene Truckel medicine just this year Okay, so that's a lot about what it is what we're trying to do some principles What about the data? Where does the data come from? And so we have a quite structured approach To finding all the published studies on the epidemiology of any disease or any risk factor And there's for those of you who are in this field know that there are guidelines for how you do that Well, they're called the Prisma guidelines and we aspire to follow those Then there's a large volume of household surveys that are conducted every year around the world And we try to capture as many of those that are where the data can be made available And then we go through and look for censuses hugely important vital statistics by on cause of death Cohort studies that may not have been published But people are willing to share the data and then the burgeoning volume of electronic health record and Claims data that tells us about Patience interactions with the health care system Lots of challenges in analyzing this data will come back to the challenges. Here's just some illustrations Here's the number of data sources in the latest round of the GBD by country So, you know some countries like Canada the US or Mexico or Brazil or China There are more than two thousand and as many as ten thousand different data sources going into the analysis That's also true in most of Western Europe and then you have places like Mauritania Where we may only have, you know, 300 data sources in the entire effort or For exact for that example Turk, Manistat, North Korea and Greenland are the most data sparse overall Now we've learned early on that managing lots and lots of data sets each with each of which can have You know millions of records in a data set is itself quite a messy time-consuming task and so to reign in the chaos of this We give each source Which may include lots and lots of measurements for lots and lots of people or many locations But each source is given a unique identifier. We call it an NID And that then allows us to trace the use of that source through all of the different steps of analysis And here's just our accumulation of NIDs over the process of the This online data bank called the global health data exchange Which you can go and use for your own work or interest where we now have about a hundred and thirty three thousand or last year It's now about a hundred and fifty thousand NIDs for which the actual data set you can download for about eighty eight thousand of the hundred and thirty three thousand data sets One of the key sources in the low and middle income world for those of you who've ever worked on public health data in those settings is the US AID funded demographic and health surveys and here's just a map of how many of those we get by location Which is a complex function of funding and other national interest Ranging from you know one country for example that has 23 DHS's the Peru stands out with their continuous DHS to places where we've seen only one or two historically Here is something that is absolutely critical for our understanding of health around the world and that is cause of death data and You'll see on this map very quickly that we have cause of death data That's pretty good for most of the world except Sub-Saharan Africa with the exception of South Africa and then a few Countries where for strange reasons because we know the data is there But they don't release it in detail that can be used places like Tunisia or Bolivia That you know do register quite a few deaths, but they just don't release the data to places like Pakistan where there isn't really very good vital registration. There's a sample scheme now for those of you are interested again in our commitment to Making public goods for people to use There is this tool called the global health data exchange and if you're interested in Parkinson's disease in You know Croatia you can put that in and you can find the data sources that exist on that disease or risk factor Now just in case you're worried That there's a lot of bad data out there, which there certainly is All the reason that there are more than 8,000 people involved in the GBD Right now has a lot to do with trying to evaluate source by source the quality of the data and that there's a There's still always I think when you're looking at data quality. There's a very objective part You know, what is the sampling frame? What was the respondent rate? What would the assay use to what was the sensitivity or specificity of the assay or the instrument? And then there's a subjective component that oh people Suspect that either the results are anomalous or the interviewers and the rainy season didn't go to certain villages Or due to conflict they couldn't sample in whole provinces There's a long list of reasons and then there are you know Frank efforts to Contaminate or or manipulate data, which by the way are pretty easy to detect because it's actually quite hard to make up data That meets a series of tests Now because we get so much so many different sources of data We have to what we do call crosswalk We have to go from the sources as they come and Map them into if the data had been collected in what we think is the reference or the preferred way So think about measuring diabetes There's some 20 different methods of measuring diabetes in a survey in the community that are out there and There is a WHO standard and so we try to map each of those alternative measures statistically into the more desired measurement approach and And so you know in that effort at crosswalking or mapping we end up evaluating the overall quality of systems and Here's an example of something you can go online and find which is our regular assessment of the quality of cause of death and verbal autopsy which is an alternative To actual registration of death where in a survey you go ask people structured questions What what's our assessment of the quality of cause of death data? So we not only produce best estimates, but we try to tell the user How much data is there and what is the quality of that data and you can see again this very income related challenge on cause of death data where Low resource settings the quality of the data on cause of death is poorer the very places where the death rates are often the highest We've covered that okay, so another dimension on the GBD is This sort of ever-expanding scope we keep adding more subnational detail We keep taking residual categories in the cause list like other cardiovascular disease and pull out and try to make more specific analyses for more detailed causes and every round of the GBD There's people who make proposals to add more detail For each disease we not only take a take a disease like diabetes We also look at all the clinical outcomes associated with that disease. So we call those sequelae So we have about three and a half thousand sequelae. We estimate for Covering all the different outcomes for a disease process We cover 88 risks 25 age groups and now a time series of 41 years So what that means is that we now produce about 60 billion Estimates each round of the GBD and the graph on the right is just the growth of the detail From that first World Bank Spod sponsored study Estimating for 1990 to where we are now now Anybody who's tried to make sense of extremely messy data has come up on Some of these eight problems. These are quite general. We have lots of places with no data or very sparse Maybe even more problematic. We have places where there's two studies and one says 10% have you know Dementia and the other one says point 1% Which is right. So conflicting data where it's not sampling. These are fundamentally Widely on non-overlapping uncertainty intervals or confidence intervals We have lots and lots of variety and case definitions Cause of death definitions across countries over time in the assays in the instruments You know, there's two dominant ways to measure physical activity in the world And the two approaches are really hard to align with each other We have lots everyone Everyone in the quantitative fields are trained in techniques to deal with sampling error So that's a familiar territory for people But in health data, we have a much bigger problem, which is all the non sampling error all the other things that lead to noisy measurement Whether it's how you train your interviewers. How long is the instrument? How long is the recall? To you know, administrative incentives when we use data from health service providers Problem number five that we run up a lot is there are certain groups that are often excluded from data systems or Partially excluded often the poor have less access So as people have pushed for more and more use of health service data You have to grapple more and more with the challenge that some groups may be underrepresented in those data sets Another problem that we face is because we write these models or make these models to fill in for the sparse or missing data Those models are much better if we have predictors of the outcome that are good predictors and for some cases like cardiovascular disease across, you know metabolic disorders smoking alcohol diet physical activity We have tremendously good predictors. So even if we have no direct data, we probably have a very good handle on heart disease Whereas for suicide, we have very poor predictors both over time and across place we have lots of cases where measurements look anomalous and the challenge is is it a measurement problem or is it a true Outlier in the sense that something special has occurred in that Population leading to very high or very low rates And I think that's probably one of the thorniest problems which is sifting from what's really interesting those outlier True outliers from, you know extreme cases of measurement error And then we do have as people push the burden of disease tech methods down to the county or race Ethnicity level in places like the US. We have, you know, extreme small number of problems and over the especially over the last 15 years There has been researchers at in as part of this collaboration and at IHME in particular That have developed a series of Bayesian statistical tools That I won't go through the alphabet soup that have been produced and published and are now sort of regularly kept up to date We have a new meta regression method for example as part of this suite that will be published next week in nature medicine That sort of extends Dose response meta regression in so that it's more useful for some of the work in the GBD I'll skip over that. Okay, so This is sort of this evolution where the methods You know the why did we make this big change in methods starting about 15 years ago? And it was really driven by the advent of Cheap statistical power so before that I think it was difficult for us to Estimate some of the Bayesian models that we would like to have estimated and so there was a real shift in approach to these Bayesian methods Driven in part by the affordability of computation Like many universities and specific to IHME. We have quite a large computational cluster That supports some of those tools Now at the same time we were making that big shift in methods and approach we also Made our collaboration For the GBD much broader So it started with a very small group at Harvard and the World Health Organization in the early 90s and then the home base was at WHO for a period and that group expanded and Then as it's the home base for the study ended up being here at not here at the University of Washington at IHME the We made a major push to broaden participation So now we have about eight and a half thousand collaborators from a hundred and sixty one countries with very large Self-organized groupings in certain countries where they have spun off and done lots of more detailed analyses And then a number of sort of medium-sized groupings and then you know places with only one two or five collaborators The collaboration is growing on a pretty much a linear basis so percent growth fortunately is Given this the process of getting people engaged and trained to participate But to give you an idea what this means for us is when we write a paper we published a paper in you know one of the many hundreds each year on burden attributable to different risk factors of cancer Probably 1500 authors those 1500 authors that participated in that project Generated some 15,000 comments on the first manuscript and so the the exercise of team paper writing and balancing out the the Insights and critiques from such a large number of collaborators is itself a whole Interesting process in team science So we have and we're not we don't really have a strategy to deal with this study growth In participants in the study Those of you interested You know which countries have the most it turns out to perhaps not be what you would have expected the top five are Iran, Ethiopia, US, India and Australia Then UK Italy, but then China Pakistan Brazil Indonesia Nigeria come next So we've had really very good success in Getting investigators in low and middle-income countries to also participate in this effort So we're we view the collaboration as a Critical resource for the study and it's part of why we think It is as useful as it has become Now anything as ambitious as this enterprise over the the last 30 years Have has lots and lots of limitations and I want to like highlight some of the bigger ones Because of course if you picked any given disease or risk you could make a much longer list of limitations so most obvious there are still lots and lots of Data gaps Makuna and really the solution for that is not better estimates Which we'll certainly try to keep trying to do but it's also helping Make the case for filling in those data gaps through improved national data systems. There's a very nice Viewpoint that will be coming out in nature medicine next week with this package of papers that we have From the Minister of Health in Ethiopia should have making exactly that point that The the GBD is being useful in Ethiopia, but it's also being useful as a catalyst for identifying these data Makuna Secondly we have found like many that communicating uncertainty to decision-makers is challenging And it may be they're right and we're wrong It may be that uncertainty intervals are of huge importance and interest to the research community But you know risk-neutral decision-makers may legitimately say they just don't care They're gonna take your your you know expected value or your midpoint estimate and an act on that And that's what in fact they do and we saw this by the way during COVID in real time just how Uncertain intervals on forecasts weren't really relevant to most users, but very highly relevant to the academic community third there's a lot of scrutiny about What are the primary data sources on one end and lots of debate about how you could tweak models to make them better and Papers and vigorous discussion, but there's a huge chunk of activity in the middle That gets very little discussion and scrutiny, which is what we think of as data processing So think on the cause of death data you get in all this vital statistics, but you have to go through this step of dealing with What we call garbage codes in the cause of death data Where somebody has assigned to death to something you can't die from yellow nails is a is an example or you know general atherosclerosis or General heart failure these are not specific causes and so there's a huge Processing step that goes into going from the raw data to something that you should model And that doesn't get as much discussion debate and scrutiny as we think it should fourth to deal with sparse data our requirements for consistency The mathematically required relationships that exist between between past disease incidents and the number of people who have disease today Those relationships between how many people have a disease and the excess risk they have by having that disease and observed death rates In order to capture all those relationships We buy nature end up with highly complex methods that some people really don't like they want simplicity But our argument is that Simplicity at the cost of validity is not a great trade-off There isn't really a case to be made to do something simple just for the sake of simplicity If it makes it worse if it if you get the same validity then by all means we should seek to be simple Another limitation one that we still haven't really grappled with in any real way But I think is starting to come out in in many debates Is how generalizable are risk outcome relationships? so we have a set of relationships that say between eating red meat and heart disease and They're studied in certain populations often white populations in high-income countries and how generalizable is that risk outcome relationship? And I think there are now literature questioning some of the generalizability for certain Exposures and that's something that we have not fully captured. We do it for two risks only Let that risk outcome relationship vary by population because there's enough evidence to support that The number six is this sort of perennial debate we have where sometimes our results disagree with expert consensus statements on a topic and You know again going back to that mantra that we have of rules-based evidence synthesis We have no mechanism of directly capturing people's opinion. We have to go from data data processing to results But we're very happy to change the rules about what data gets in and how we analyze them but we we often not often but sometimes will have come up to a different conclusion and That's probably a marker for where further research and debate should take place and then as I've mentioned There's just a huge list of disease specific limitations Let me round out the presentation with two topics one is the What's happened from our? Spontaneously organizing large sets of collaborators in three countries so in Ethiopia we've had a very active leader at the Ethiopian Public Health Institute for the collaboration of ok and He has managed to organize hundreds of academics across universities and people in the government and they have created a thing called the National Data Management Center and then a burden of disease unit there and Then they have been driven by a very close relationship with the Minister of Health And not just the current Minister Leah today say but the the two ministers before To make this collaboration serve specific policy questions that come up Examples of what they produced they published this year their subnational paper Which you know also had a very strong participation from the government in that publication not just the academics They're producing or have produced an atlas at the subnational level of health problems, and then they've been working on Three policy documents one is the sort of essential benefits package for the government a health sector transformation plan and then How to help inform national health accounts? In Brazil The model slightly different they've had a very long run in collaboration. They're very self-organized And what they have done is created a collaboration Which I think were most impressed by the fact that it survived the big change in government from You know from left to right in the government and yet they stayed both an active collaboration and used by government And so that sort of stability of the collaboration in the face of major government change is really super important they are extremely industrious on producing studies as part of the GVD network and You know take the basic Number crunching that we support the network with and turn it into more meaningful Analyses and policy documents they just put out this year the second time round They've done this a very large set of papers 23 and all Representing a large number of institutions in Brazil and academics in Brazil on 23 different disease risk factor topics that was another Testament to their ability and this this general success of the GVD in fostering the sort of national Efforts in India the effort is a tripartite effort led by Lalit Dandona Who has appropriately appointments in three institutions IHME the public health foundation of India and the Indian Council of Medical Research Where they have both been publishing papers, but also writing policy documents one of the leaders of this collaboration Moved on to become the health person in the planning ministry now called Niti Iyog And so there's been a very direct input from the GVD work in India into government planning and right now They have continued Focusing on certain diseases and the very interesting exercise right now of using our new suite of long-range forecasting methods to look at specific scenarios for the government of India Time for the 75th anniversary of India and saying what may happen How what might the next quarter century for India look like if you pursue different packages of policies? Little bit on where we're going it's been an interesting exercise in Trying this really large team science effort and we've learned lots of things along the way lots of things that we hope we will Do better The demand for what the collaboration produces and the interest in participating and just grows steadily But what we would like to see is more of the analyses that for example are going into the Brazil or Ethiopia or Indian examples the actual, you know running of the models is done in Local institutions, and so we have a plan over the next five to ten years in this collaboration to increasingly Decentralize some of the estimation While always fixating on those principles I started with which is the comparability and the quality and so we are Trying to build tools that allow for that sort of more distributed analytic approach second big area and one that's probably the the Lesson we learned from COVID which is policymakers really can understand and use alternative scenario forecasts in a very specific actionable way and so we Published for the first time in 2017 and then again in 2019 our long-range forecasting framework I will not go through it tomorrow I will talk a lot more about the future and what we both see coming and you know What are the challenges for doing this in any credible way? And then what we can do about the future so that's going to be very much more on What do we what do we think we see coming and what can we do about it? But it's just part of our very strong emphasis on building up this set of alternative forecasts so that decision-makers can try to see the magnitude of risk from climate change in the context of antimicrobial resistance in the context of You know low fertility or Interstate conflict or new pandemics Another direction which is sort of following a theme is not only looking at the burden of disease at the Subnational level, but now we have an NIH cooperative program To estimate the burden of disease by county race ethnicity group and here's just an illustration of that This is still looking at deaths not the totality of burden, but that's eventually going to come. So we produce for five OMB 1997 race ethnicity groups separate discussion about getting to a more detailed race ethnicity group But for now the five shown here white non-Hispanic top left black non-Hispanic in the middle top a American Indian Alaskan native on the top right Asia Pacific Islanders top left and Latino on the bottom and then total on the right And this is one cause HIV AIDS. We have this for several hundred causes and you can see very quickly how a Lot of variation in HIV is related to race and ethnicity But even within black non-Hispanics there's extraordinary variation with super high rates in the southeast in Florida and South Georgia and selected places in California and Then in places like Mississippi, you know comparatively lower rates. So lots of Interesting food for thought for why Explaining what we see so this Phase of the work is highly descriptive. It's that this is what we see. It's a time series back to 2000 We see lots of interesting trends where things are getting worse for some causes in certain parts of the country And there's a parallel project not part of the GBD But also die to me to look at health spending by disease by race ethnicity and county So we will be able to compare a time series of spend with a time series of outcome And I think that opens up a lot of interesting Difference and difference type analyses where you know this place spent a lot more on diabetes What did you get for it compared to a place that did not? Another map here is interpersonal violence very different spatial pattern than what we saw for HIV and You know very much Very high rates in certain AI and communities and some high rates in black non-Hispanic communities Last on this list of new things is our What's being launched next week is we've introduced to help the public? decision-makers and research funders Understand how strong is the evidence supporting risk outcome relationships? We call this the burden of proof risk function analysis and at the end of the day We end up giving a star rating to each relationship between a risk factor and an outcome And there is some nice open-source new meta-analytic tools support this as well as a sort of almost Philosophical piece in nature medicine as to why we're doing this to help people Navigate the constant confusion in the media as you know, it's read me good for you It's read me bad and and opinions changing from week to week So that's coming out next week and will progressively be built into the the annual assessments of The GBD there's an online tool where you can not only get the overall result But you can drill in and see all the studies that are exist in the published literature on You know a factor let's say red meat and an outcome Let's say colon cancer and see exactly what our knowledge is how messy how heterogeneous it is and why we say for Example that relationship is is a rather weak one. It's a one or a two-star relationship so hopefully that will make the GBD Useful even for individuals trying to navigate the Choices for themselves on these risk factors so that you can have this sort of more comprehensive view That's not driven by whatever the latest study that's been published So let me just end and then open it up for questions for You know, just what is it that we're being doing and hope to keep doing for the next 10 or 20 years And that's a global collaborative effort that continuously tries to improve data methods ease of use transparency and utility for Everybody's decision-making from individuals to two governments. So thank you very much for this sort of look at the GBD over the last 30 years And there's a and here's a microphone if anybody does have a question We have a brave soul great Shaded your remarks. I'd love to hear a little bit more about the decision-making So who are the consumers of this data? What kind of actions are they taking? and Where where is it all headed? Maybe long term 15 20 years out Yeah, great question, you know the We started with a view that the primary users were People in planning departments of you know departments of health ministries of health or Ministers of health right because there's a series You know, there's this there before people even decide to adopt the policy They first have to consider the policy options people don't consider policy options until they think there's a problem and So that agenda setting part it turns out there's enormous use of what we produce For helping set agendas, you know mental health was not even people weren't even talking about policies in Mexico for mental health Because there was no discussion of mental health until you know, it suddenly became Put on the map so to speak and so we've seen that over and over for different diseases. It's also why We get so heavily lobbied by groups who are trying to get a disease or a problem put on the agenda Because if it's not in the burden of disease then now we've got to the point There's such widespread use of this study That maybe they can't break through to get attention so for example We recently added Again a five-year effort Funded by the British government and the welcome trust and the Bill and Melinda Gates Foundation We extended the GB to include antimicrobial resistance, you know super bugs who's dying from those and it was driven by their desire to Let people see how bad antimicrobial resistance was as a problem in comparison to all the other problems so High-level decision-making agenda setting It's an input to other types of analyses that governments make when for example, they're looking at Evaluation of what's happened the time series we produce is very useful So there's a another constituent that says, you know, we funded a lot of these programs Did they were meant to reduce, you know, child deaths from diarrhea? Did child deaths from diarrhea go down? You know, there's a lot of sophistication that should go into Would they have gone down anyway or would they have gone up etc? But you know the starting point is this the descriptives as Times gone by we found the the users or the audience is broadened. So we now have many NGOs Who use the burden of disease in the locations where they're they're trying to work to help figure out what programs They should deliver and some even have built it in to how they manage their organizations. So PSI is an example It's a it's a big NGO that works in the reproductive health space in many parts of the world and they internally manage and Allocate resources by what they think is the burden that each program as measured by us is reducing So lots and lots of examples of that and then I think as we've gotten more into Risk factors and then our COVID experience is there's a lot of general public interest in these So we have you know, the star rating on the risk outcomes is as much for the general public as it is for anybody else You know, we'd like in the as time goes by to have Active, you know Well organized Collaborations in every country in the world who are both doing, you know Harnessing good science to do the best measurement that that feeds into this sort of global view in a comparable way But also feeds more directly into local decision-makers because I think our experience in many is Unless there's local ownership. It's unlikely to be active on which is why we put so much emphasis on growing this collaboration other Questions I had two questions. How many people are currently working at IHME and Given that the data gathered here could play a very important role with governments When they come to policy decision-making, I wanted to understand what are the quality checks that go into data collection and consistent management of that Sure, we were 500 people at IHME only half of whom work on the GBD All right, so it's about 250 all-comers on the GBD Some people You know work partly on the GBD and then in terms of the the sort of quality steps Depending on the type of source there are different quality evaluations So if it's a household survey, there's a sort of standards and a metrics we look at around response rates Based on the each specific indicator that comes from a source then we Also look at, you know, sort of the face validity of some of the results So it's not as simple as just looking at survey level metrics when it's final statistics There's a set of techniques to evaluate completeness. These are called death distribution methods that were developed in the 1980s 70s and 80s by demographers. So we evaluate how complete what percent of deaths are captured We look at the fraction of deaths that are assigned to garbage codes and if it's too high we don't use the data and then we also look at how detailed of Aspect of the international classification of disease has been used and that all goes into that that Star rating for call-to-death data and so that gets done each cycle and then in other areas and disease specific surveys We do other types of checks for Claims data. There's there are a similar side of sort of standardized metrics as well With apology for later rival Giving global burden of disease just a little bit of research that looked like it was World Bank commissioned So I'm wondering one question is was there an earlier attempt that failed or didn't take Or was that the kind of the first and the good attempt at it and it's held and the other part of the question is it sounds like it's been institutionalized by the World Health Organization with participation of Harvard and wondering what other institutional and or university Entities might be yet at work or in strong support of the of the concept and the work. Thank you Sure. Yeah, I covered a bit of that at the beginning It's been one study the whole time it started by the bank and Then the initial round was based, you know It's two people myself at Harvard and Alan Lopez that the World Health Organization Who were the initial we were the initial drivers of that with a very small team And it is just steadily grown and so now it is this, you know, eight and a half thousand person collaboration We have formal agreements with WHO as part of that but also with You know many many institutions that are part of that collaboration and so It has become this sort of Global standard for for health measurement Now there are certain parts where there's alternative measurements that are produced So for example maternal mortality HIV TB Some childhood illnesses. There are alternative multi-country global estimation efforts but for 95% of diseases and risks the GBD is really the only source for better or for worse What did I do? There we go. Sorry Yeah question You know I just say thank you so much for the presentation. It's really helping me get through the last hours of this Nemke poor fast I've got a couple questions. One of them is Related to when you said that it's hard or it's hard for one of the challenges in collecting There's a special challenge in collecting data when say a country is involved in a conflict or something So maybe regarding the case study for Ethiopia. Have you found like or what are the ways you get around say? having a violent conflict in the country You know, I think the the group there Faced lots of challenges with the war and even now just Going about their work with the conflict So very challenging for them You know it is what it is in the sense of the data that they can get a hold of and so for some parts of the country This was very true for certain provinces in Afghanistan even before a year ago That you have very little data. So then you are Depending on the statistical models to sort of guess for you what's going on there Now the the conflict part per se People dying directly from conflict is a separate Analysis so we have there's a separate analysis of What are called? You know a fatal discontinuities? Where the source of data going into the the work is not Government statistics because they don't work for conflicts or in that for that matter often for some natural disasters and so Intriguingly the main source of data for measuring what's happening in conflicts due to conflict are news reports and There's a group in Sweden that does the best work But we look at all the different groups that have been trying to collect those sort of news reported items And there's some rules about which ones you believe they have to be very specific It has to be an event on a given day in a given place with a certain number of people affected But even then it's just a huge You know, it's it's a very challenging topic to figure out conflict deaths What's the Swedish group called? Oh? my goodness, um I Am blanking sorry But I can certainly find out for you And I guess the other question you brought up just a moment ago that with something like COVID You were able to get a lot of like public interest and public engagement. I'm also wondering with Diseases that are less or gain less attention than COVID like say malaria are able to get public interest or Really like spread information to the public for those diseases or does it doesn't end up being like more Attributable Attributable to people who suffer from those diseases are impacted by them you know the success of gathering global attention depends On some extent on on success with the media whether local media in a particular place or global media and You know, you might hope that that wasn't the case, but it is and so we do spend a lot of time as Too many groups who are trying to help shape agendas Thinking about various ways to get attention in the media and It is it turns out to be the case that the reason if your primary endpoint is, you know Making the world a better place and trying to change policy in some direction for the better The you may not care about the media, but it is this vehicle to Impact and then in turn it turns out that it's easier to get media from malaria for TV or for any disease if there is a Scientific publication that explains why you're trying to get the media's attention and That's even better when it's in a top tier journal So there is this ecosystem that says if you not all things that are in top tier journals Interest the media, but if you really want to get a lot of media interest there's exceptions, but generally over the long haul you do want to publish in Lance at JAMA New England Journal science nature and and then use that as a way and Then there you need to follow that up with more targeted Outreach to policy makers once they sort of oh there's something in the media. I should pay attention to this Maybe a question you will address tomorrow, but I wonder if you could say anything about How thinking about risk factors how you will address climate change as a risk factor? It's very complicated. There's direct and indirect effects. I see that you have included Non-optimal temperatures lately so any words about that Yeah, I mean I think like so many groups We are spending much more time thinking about climate and how to Capture that in the GBD Which we've got some parts as you mentioned, but also more importantly how it captured in the forecasts and So, you know, we've got risk curves now, which I'm sure can be strengthened and improved with more data for temperature and 17 diseases and Then I think there'll be a debate about some other diseases where the evidence is sort of equivocal Then there is The question for the forecasting about other pathways and so we are exploring various ways that you can get some of the relationships between population health climate economy and then back to health right so some of those sort of broader effects and so Pretty active area for for research for us and we're trying to collaborate with a number of the other groups You know some even based around here like the climate climate impact modeling group. I'm just wondering if you could speak About anything you guys might be doing specifically around dementia and if you could Maybe give any advice to somebody who's just starting in that area of research or point to any Resources that that you know of that might be helpful Yeah, we have quite a Sort of in-depth effort on dementia And we are launching with a number of partner organizations a brain health initiative Where we will put you know Not just the sort of measurement of current burden, but looking at the forecasting around Brain health and that sort of course includes dementia So on that area interesting things There's lots of interest but lots of measurement challenges on mild cognitive impairment There's lots of outstanding debates on the mix of Alzheimer's versus other dementias versus you know mixed I Think that's a pretty open area lots of value in Understanding that more You know we've started risk factors for dementia You know trying to just usual things systematize the published studies look at other cohorts that we haven't seen So there's a lot of potential work that would be interesting on on the epi side of the risk factors for dementia You know we know education, but what are some of the other ones that are large And then on the health economics side or the spend side there's quite an interesting set of studies Underway of Where does money? How much is direct care? How much is informal care? How much is you know? lost productivity for early dimension, so Yeah, I mean the two people at our end was several but on the on the more economic side Joe Dillman and then tail boss who sort of oversees the the epi work on dementia the other questions