 Yeah, hey, maybe we could, people can take our seats and if I could ask someone to step outside for those who are having coffee and tell them we're getting started here. Yeah, thanks everyone for coming. I'm going to turn it over to my colleague here. We have Ed Kelly. I'm director for service delivery and safety and since I work in this building and Fin does not, I'll officially welcome you to Sal A of WHO and thanks for getting through our almost activated new security system this morning and we're very pleased to be with you for the launch of this WHO bulletin theme issue. But yeah, before Lord, you want to say anything before we get started? Oh, all right. Thank you. Yeah, so welcome. Thank you all very much for coming. This is a new collaboration for us with UNewider. We haven't worked with this UN-affiliated Institute before, but it's been a very preferable collaboration and we have a really nice selection of papers on quality of care. The reason that we're interested in highlighting quality of care is an issue is that in moving from the MDGs to the sustainable development goals, countries really did a lot to improve coverage. So we had a lot of progress on quantitative targets, but the outcomes haven't always followed. And where we see that gap between quantitative improvements and outcomes not really following, we know there's a quality problem. And every country has not solved, no country has solved the overall health care quality problem because countries move very quickly from a situation of too little too late in terms of care. And this is across, you know, the entire spectrum of all diseases to too much too soon. And both of these are quality issues. So if you look at the papers in this issue, you'll find from a wide range of countries in all regions, authors trying to investigate the reasons behind the quality chasm and really looking carefully at how patients should be involved in determining quality, how we measure those outcomes, what the indicators are, and how we're going to make progress in that area. So thank you very much for coming. For all the interns here today, I'd really like to encourage you to consider this as a focus of your further studies because this issue is not going to be solved easily. So please consider it as you're moving on in your qualifications in public health. Thanks very much. Great. Thanks, Laura. And just maybe I'll turn it over to Finn to kick us off here. Okay. Thank you very much. Good morning, everyone. My name is Finn Taub. I'm the director of the UNU Institute called Wider Word Institute for Development Economics Research. It's a great pleasure to be here to contribute to the opening of this launch event of this WHO Bulletin Special Issue. I mean, if you would allow me, I'll just say a couple of words about UNU Wider and then a few words about the issue. And then I'd like to add a few words of thanks to colleagues who have played a key role in this process. UNU Wider, the United Nations University World Institute for Development Economics Research was established a bit more than 30 years ago in Helsinki, Finland as the first research center of the UN University. The university has its headquarters in Tokyo and has about 14 institutes around the world. Wider is the institute that focuses on development economics research in an interdisciplinary way. You might want to take note of the fact that one of the founding parents was a Marcia Sen, so of course we have long roots in terms of human development indicators, human development issues going all the way back to when we were created. Today, Wider is a unique blend, if you wish, of a think tank research institute and UN agency and we basically work both on sort of global development issues, taking up themes that we believe are coming up and are going to be important, but then we also do provide policy advice to a range of governments from South Africa to Mozambique to Tanzania to Ghana to Myanmar to Vietnam and so on. So we sort of try as we sometimes formulated to deal both with the bigger development issues but also trying to engage in specific country context. I would encourage everyone to take a look at our website for access, free access, free downloadable access to both books, journal articles and so on. I'm only to about, well I guess it's about 8,000 publications on key development economics issues that I believe are worthwhile digging into. Now how could, how did sort of this particular WSU Billeting theme issue come about? Well this was already referred to. At UNU Wider we take data and evidence serious. We do believe that data and careful analysis will have to play a crucial role in achieving the SDGs. On behalf of UNU I was one of the 68 agency representatives who were involved deeply in the process of formulating the SDGs. So I sort of, how can you say, got a keen awareness of the fact that the call for a data revolution is quite unique in some ways but it has to be followed up. This is particularly the case in relation to health. It's very clear as has just been mentioned that coverage has indeed expanded a lot both in education but also in particular in health and I think it's important to say and reiterate what was just mentioned that this has not always been followed in a way which would have or imply high impact. At the same time I don't think one should overlook the very distinct advances that have been made. At Wider we did a major program on research in relation to foreign aid and aid to social sectors was one of them. Now one of the key messages that came out of that research was exactly this focus on we need to address quality issues not just expand, expand, expand because as economists will put it there are diminishing returns to some interventions. You need to think about the quality if you want to have impact. Now the papers of the theme issue they do provide illuminating insights into the current landscape of the quality of care research in both low and middle income countries. They make a strong case for investing in more and better data on health system quality an area where countries must lead but where development partners add critical value through the production of public goods as well as data collection tools and technologies and analytical methods. Now if you would allow me I just step out of my sort of slightly more formal situation right now when I woke up this morning I realized it's the 14th of June 2017. 37 years ago in Switzerland our first child was born. I can attest to the fact of understanding very much at the personal level what quality of care means because the quality of care was not there. So it's just to sort of illustrate that these things are both at the global development level but they also concern human beings and they concern the very fundamental aspects of development which is to increase people's welfare and well-being. So this is a core issue both to development and to welfare and to the lives of our children and their children. I would like to thank our partners in realizing this theme issue from WHO at Kelly and Shamside from the service delivery and safety department for being a great partner in co-editing the issue. The Bulletin Editorial team with a special thanks to Kittis Bartolomeos for being editor. We were really lucky indeed to have you. We also thank Larry Golligli and Kaleen Selig for their outstanding vision and support. It has been just such a pleasure. Now Professor Margaret Crook well what should I say the star expert in the quality of caring low and middle income countries we have thoroughly enjoyed interacting and have appreciated very much our collaboration. Now authors some of you are here we were indeed impressed with the quality of the selected papers. I mean wider producers about a hundred academic papers a year that appear in refereed journals this particular theme issue will be a core element when we look back to what was achieved in 2016 and 17. I am delighted to be with you here today both personally but also on behalf of you and your wider and I'd like to thank everybody for being here today and for being ready to contribute to a process of learning both from the research papers and from our interactions. I look forward to our discussions and appreciate the possibility for being with you. Thank you very much. Thanks for that and just before I hand it over to Margaret a couple of thoughts just from WHO's standpoint. As someone who's worked in this field for a long time the the progress that we made has been notable I think but I think we should also be humble about the lack thereof as well. If you look at some of the work I started my public health career working doing research on IMCI the integrated management of childhood illness and the quality of care delivered to kids in Niger West Africa so it's based in this tiny town on the edge of the Sahara Desert Tawa Niger as part of a USAID funded project are some USAID colleagues here today and some of the results we got in terms of you know how pardon me we traditionally measure sort of quality of care which is how many times when you have the opportunity to do something right how many times did you do it right the results we got were around sort of generally 60% of the time whether it's sort of diagnostic work therapeutic work or communication work that the health workers were doing in these very overcrowded health centers a number of years later in the US publications by colleagues like Beth McGlynn and others showed that actually the US did only as well and often less well around 50% of the time did we get it right and you can see in some of the baseline data that are in this particular issue in different countries presented it were kind of at the same level and so I think there is there's some lessons to be learned there around some sort of universal universality of this issue that that we should get that I'll just come back to we've also we made some progress on the issue of measurements which is the other big question within quality of care but not enough there's still a multiplicity of measures and not enough unification around measures we when I first was at the OECD we were doing the OECD quality indicators I had good friend of mine I can use the name Jan Mainz is from Denmark he's thought of very well thought of in the quality world told me flat out we will never be able to report on quality of care across countries because the systems are too different it's you won't be able to talk about it now 10 years later quality of care indicators are regularly reported by the OECD they were reported at their ministerial meeting just earlier this year so we have made some some progress but the number of indicators on which we can agree is very scanty it's around 10, 11, 12 even for the OECD and on improvement yeah what can I say there are as many different methods of quality improvement out there as there are sort of practitioners in the world on it and I think looking at how we can come together on that is another is another area but you know I think the the prospect and the and the possibilities are very high Laura referred to this and also Finn our new pardon me leadership here at WHO Dr. Tedros is taken over as the director general has picked up the mantle from our current director general on the issue of universal health coverage as being a top priority if you look at how WHO has formulated universal health coverage quality is not part of the UHCQ but it's not it's not something we've thought about it is we have basically thought in a supply side orientation around universal coverage sort of build more things get more health workers and push out more drugs and we will solve the world's health problems clearly that is not happening in most parts of the world and some of the some of the evidence here will show that there's a big love at the current time in at least in WHO but in other circles too on conceptual models and how can we frame sort of health systems strengthening in UHC for me this is not so interested in all of that and and if anyone in this room before the end of the day says raise at their hand and says can I just ask what do we really mean when we're talking about quality I'm sorry you're going to have to get out of the room because we're we're not going to get into those sorts of questions but I think we spent too much time talking about the definitions of quality and framing quality and not enough about learning from the tons of experiences that are out there and and building from the bottom up on that so I think we do have those possibilities and that this could be a big unifier across not just for health systems certainly but also with our colleagues in maternal and child health a number of them are here and you've just had a really positive and energizing meeting this week on a new network that's active in this space but also with our disease programs each of the big disease programs here and then also neglected tropical tropical diseases all have different programs looking at quality of care whether it's indicators community engagement or sort of improvement activities I think there are a number of very promising initiatives one of which is led by Margaret and the new Lancet commission which maybe she'll say a quick word on and there's also work within WHO and our team on this that's led by shams and looking at trying to bring together within the house a task force on this as well as a group of external team looking at this and a learning laboratory that's looking at how we can advance learning within this space and the learning agenda around it as well as policy questions which we have a meeting on national quality policy issues which has been a missing a big gap for some of the champions locally they haven't had a policy environment in which to work so I think all of those are very positive but I think you know the bottom line is we should be pushing this agenda because for me it's where the supply side of our work actually meets the demand side and as a supply oriented institution as WHO is we miss this piece too often we have a program our patients for patient safety program some folks are in the room helped us start that where we had patient safety champions who they themselves or their family members were harmed by care and they told us their stories and it's amazing if you take I frequently start my presentations with an example from a woman from Uganda business woman from Uganda and who saw her child die in hospital and a man from New York City who saw his mother pass away and the experience they had of the medical teams insensitivity and the lack of proper protocols and treatment and communication there's some universal way we are constructing our systems that a person in Uganda and a person in New York City can have such similar experiences of poor care that we need to take a look at and I think this again highlights how what a unifying issue this could be for us going forward in a new era so I thank you and wider in particular for pushing us forward on this issue and Laura and the team for for making space and a very crowded publication schedule I have a new appreciation for how solicited they are but also for Margaret and the team for for the organization around the papers and all the authors who are here we really pulled together something I think that is that is really very interesting so without any further comments I thank you for that and I'll turn it over to Margaret thanks very much there we go okay good morning everyone it's wonderful to be here with you at what is I think been over a year's worth of work actually probably 15 or 16 months so a wonderful culmination of the work of many of the people sitting around this table and I couldn't be happier actually with what we have in front of you today which is the special issue of the bulletin of the WHO so let me go back to the MDGs the era that's just passed the MDGs have had indeed many successes I would say that chief among them is the much greater coverage of lifesaving interventions that they ushered forth that means something very specific I think some of us are bean counters and love to count numbers but really it is meaningfully different now to be a pregnant woman in many countries and know that you're going to be able to get delivery care out of facility that there is a path forward on that that you have somewhere to take your sixth child so I think those are those are incredibly important achievements and those achievements in coverage actually were accompanied in parallel by an incredible explosion of great research on coverage understanding how to measure access at the individual level the population level and even the equity of access which is such an important dimension who's getting care where are they getting care how hard is it to get to care many colleagues around the world have contributed to building the evidence base on coverage and measuring it systematically across countries and what that allows is for comparison and peer pressure and and for progress so I think the the research part of this can't be understated and the importance of that in moving health for people improving health and so we know that the MDGs have also had notable successes in actually improving health improving survival and they've done that undoubtedly in several areas they have not succeeded in other areas and it is I think instructive to reflect on where the MDGs have struggled actually where we have struggled in achieving the MDGs and often that has been in some of the more complex conditions maternal mortality which requires a chain of events to happen to happen on time to happen with care pieces to be there of the entire health system we haven't done very well for newborns actually globally we've made it some improvement but really not nearly as much as we need to and now today in 2017 we're actually past the dawn of the sustainable development goal era was reminded by your wonderful huge circle that health you know is doing its usual takeover of the rest of the the goals being in the middle but then what do the SDGs really mean for us those of us who think about health systems and people's right to health well the SDGs mean more conditions that we should be thinking about it's no longer maternal child health and a few infectious diseases it's a wide range of conditions it means more complexity the people who are not saved by current interventions are sicker people they're the sicker newborns they're the septic newborns they're the moms with severe complications they're people with chronic disease now that will need lifelong care also we know multimorbidity is a growing issue as populations survive some of the early years and grow on to be older which is a goal that we all share and want they're getting sicker and have often two or three conditions and then let's not forget this while health is changing and the profile of health conditions is changing people's demand for good care is also changing we are no longer in a world where people don't know what good care looks like they can easily text their cousin in Dar es Salaam if they're in a small village and find out what happened when she went to clinic people see on tv the way that good doctors behave with with patients and and worse doctors behave with patients the connectivity and by the way empirically we're seeing it in our research people with cell phones have different demands from from health care even adjusting frankly for their education and we are seeing greater education of course and we are seeing fortunately growing wealth so demand for care quality is growing as well so all of these all of these circumstances i think bring us to the question of quality it is difficult to imagine making progress on the sdg agenda and it is certainly impossible to think of how we can satisfy the needs of populations without thinking carefully about quality and so actually ed i will define quality here because i do think that one of the the challenges has been that it has become a splintered and very multifaceted issue i don't think it's that complicated quality to me has at least two key features it is health care that can either improve or maintain health of course a lot of health care is preventive so we don't have to always be improving health but we should be helping maintain it number one and number two it is health care that brings value to people above and beyond the the improvement in health remember many conditions are self-limiting people want to come out of a visit feeling respected so they know that they can go back when then when the real urgent need arises so this notion of value to people is something that i think we haven't been thinking enough about in thinking about quality so that's that's what i would say a definition is i i think that there are several elements that are particularly worth paying attention to and here now i'm taking the perspective of the patient the user of the system so what do i see when i enter a clinic i've gotten coverage i know where to go i come into clinic and i'm really thinking of two fundamental elements when i'm judging quality i'm thinking about the competence of that provider in fact is my perception of a competence that may have led me to this clinic in the first place is this person going to have the skills to help my baby the the reference you made to your son's birth we all everyone in this room who's had a child knows how hard we all look for the right exact provider with the right support system that's going to help that delivery happen or whether when our child as my daughter did fractured her arm where is the right exact place to go that will that will give me the technical competence i need so we need competent care everyone has the same demand for that but the second part is we want a good experience at the at the fundamental level this is a personal experience of health care this is i am also a customer i am not just a patient begging for your time and so i expect respect i expect ease of use i expect a good consumer interface frankly with the system and so the patient experience and the competence are two fundamental parts of of quality in terms of the process of that care and then if i do get that what are the benefits to me if i do get that hopefully i will have my health will be better now hopefully that can be objectively measured but sometimes what matters most is actually how i feel and whether my symptoms are improved we've seen increasing wealth of evidence including recently a very prominent article in in the oncology field showing that patient reported outcomes what people tell us they're feeling are actually tracking very very well with survival and with other hard clinical outcomes so we've got to be paying attention to how people are doing that goes for a post part of mothers whether they're depressed whether they're looking after themselves or they have symptoms as well as people with chronic and other illness so we care about health certainly but also good quality brings other benefits those include greater confidence in the health system for the next time i get sick or the next time my child gets sick or god forbid we have an outbreak of ebola or some other condition where now i think okay do i have a base of trust with the local clinic with the health system at large i assure you if you don't have that base of trust no amount of government radio messages at the height of the crisis are going to drive me to the health system the third benefit is greater and better utilization those of you in the fields of hiv tb in chronic care know very well how difficult it is to retain people in care and yet when we go around the world and we ask a health program manager what do you how do you know your your program is working the first thing they say is well we can retain people and they're right the idea that people stay in care is a vote of loyalty to that system and so retention actually an appropriate utilization timely utilization avoidance of over utilization by the way is also really important and also efficient utilization so thinking about economics you're not always going to one center which is overcrowded but actually the quality is good enough at your nearest facility that you could go there all right so utilization is another one and finally a benefit of high quality of course is finances so populations who get good quality care can go once instead of getting the three second opinions that they might need the second the third the fourth and you might think i'm an exaggerating or you maybe you think i'm talking about high-income countries where we're very picky no actually it's Liberian one of the poorest countries in the world where we found on average for every formal health care visit there were two additional visits to informal providers to just make sure that this was on track okay so this matters and this has an effect on people's pocketbooks and their economies so i hope that i persuaded you this matters i don't think i've had to persuade you if it came to this to this last equality is a critical element as we move forward on the sdgs so then the question arises what do we know where is the great research just like on coverage research was fundamental in in making progress happen what about quality and there i agree with ed i agree with many of the comments that were made there's a lot of work going on in many countries and yet i would say that today in 2017 our talk outstrips our knowledge and our assumptions outnumber our facts and out our evidence there are extremely few systematic comparisons of quality indicators across countries oecd accepted i'm really thinking now about low and middle income countries indicators where we can say with assurance this is nationally representative right this is measured in the same way across countries and we were at an extremely basic step in terms of being able to do the sort of benchmarking but let's forget about the international comparison what about countries themselves what do they know about how their health systems are working well actually very often they don't know much or they know pieces and that's perhaps more more accurate they'll know how a tv program is working here because those partners are actually reporting and doing a great job they have a patchwork they don't have an overall sense of how the health system is performing and so i am extremely grateful to the bulletin and colleagues at u and u idar and the world health organization for working with us to put together this issue on quality what we try to do here and quality measurement more specifically what we try to do in this issue is to to far and wide reach out to the best researchers working in this area of quality measurement to say bring us your best work bring us your best evidence talk to us about measurement how do we do it across all conditions it wasn't just one set of conditions or another tell us tell us what we need to do better and i think we have a wonderful set of papers more importantly even i would say than the empirical findings which we'll hear about in a moment is the stimulus that we hope this issue brings to the field of quality research i'm talking about research and measurement i'm talking about description i'm talking about analysis i'm talking about better evaluation of improvement the full range of research needs to get bigger and better and soon um finally let me just say a word about the lancet global health commission on high quality health systems which i have a great privilege of chairing together with dr. mohammed pati and 30 amazing commissioners many of whom are actually in the room and advisory council members this is a in 18 months effort to try to define quality in a clear and simple way that health ministers and and line managers of health systems can understand not just experts at who or in academia and also to describe the best that we can quality across conditions sdg conditions talk about improvement what's working and and very critically about the equity of quality who's really getting good care in countries because we don't think it's necessarily the poor uh so that work is is ongoing and we hope to publish a final report in october of 2018 so just over a year from now uh the time is running fast um so we look forward to all of your input on that through our website um which is hqsscommission.org um happy to talk about that on the side but i want to return now to this uh incredible group of authors to tell us about the findings in in the special issue thank you so much again thank you margaret so we have a selection of these fantastic authors sitting up here and we're going to hear from them in turn looping from the world bank is going to start off telling us about infection prevention and control practices and primary care in kenya um if you could walk us through your findings thanks who is managing the presentation okay because it has a lot of animation so i may need to okay you may need to good morning and i'm going to talk about our results uh from uh analysis of compliance with infection prevention and control practices in kenya uh this is a part of a bigger project that is the kenya patient safety impact evaluation uh that uh i am uh part of with a group of colleagues from the world bank but this is only one of the two that we are using to measure quality of care okay so first um why is this important i don't think i i have to motivate a lot for this audience healthcare associated infections are a priority a global priority and IPC practices are relatively easy and cost effective ways of preventing a health related infections um but the problem and the challenge as as the um the speakers have emphasized now is that there is very little research on the extent of the problem and particularly in low and middle income countries and for primary care settings most of the studies are focused on small samples focused on only one dimension or one of the sites and in many occasions on cell reports so to address these limitations we um build or uh use an observational patient tracking tool to measure uh if IPC compliance in different domains and sites and we conducted the largest patient safety survey across low and middle income countries that we are aware of with more than 1000 health facilities are in 1700 healthcare workers and more than 14 000 patients so what did we do first we decided uh we selected three procedures that we were going to observe they are examinations injections and lab tests we decided on five domains that we were going to track is they are hand hygiene protective gloves injections and blood sampling reusable equipment which is basically disinfection of esthetoscopes and uh thermometers and waste segregation and the idea is that what we wanted to track is the the average experience of a patient throughout patient services in Kenya then we developed no goody back when we developed a tool and basically what we did is that we built on a WHO tool so we're already available and validated but we put them all in one checklist it's a two-pager where we actually track all the indications which are the cases in which uh there is risk of pathogen transmission in the in these uh of procedures and also the actions that the healthcare workers took and the idea was to fill in this checklist in uh on average five minutes that uh an outpatient consultation uh last then what we did is that we train assessors and very importantly we use tags so all the patients that we track have adhesive tags and the healthcare workers as well so that we could actually follow them through the healthcare facility or through different sites for the healthcare workers we also measure knowledge at the end of the observation and we also measure whether they had available supplies so the main results first I don't think you can see it very well but I'm going to try to describe it first is that we found very low compliance with the 20 IPC practices that we analyzed 32% on average and this figure that is showing there the the top of the figure shows you the indications which are the cases in which an action should have taken because of the risk of transmission of pathogens and the line below tells you the violations the safety violations and in summary just to to give you a point of the analysis that the first one the 3.1 and 2.9 is if a patient only goes through examination which was 50% of the patients usually all the all the indications are safety violations so it's three indications and three safety violations the last one is if an a patient goes to examination and has at least one lab test and at least one injection they are exposed to 23 indications and 13 violations so it gives you an idea of the main results the second one is the second important result is that there is significant variation across all domains so hand hygiene which is the corn store on IPC the compliance was 2% which was very low and then very high in anything that was related with infection injections and blood loss so 87% on the practices that we measure and 82% of the waste aggregation of needles and syringes the rest of the of the practices were to needle to very low compliance the next in the next one and we found very significant no-do gaps so in the first one just to give you an example that the compliance knowledge and supplies for hand hygiene so what you find in the first one is that in 83 of the cases healthcare workers actually knew what to do in which cases to wash their hands in 70 70% of the cases they had the supplies that were required for the practice but in only two point four of the cases actually you they practice right and what happens is that when we condition that and do that average only for the healthcare workers to actually knew and also had the supplies that only jumps to 4% so 2% on average for all of them but when you only do them the average for when they had supplies and knowledge it only jumps to 4% so what are the the other element is that there is a very weak association with anything that we measure at the healthcare worker or health facility the level the age the location the gender where the facility was big or or large many of the things were weekly associated where are the policy implications on this one is that there is a progress in in some areas so what we found is that all the campaigns that have been conducted for injection and blood draw safety actually have been effective because for instance in one of the indicators where they were using new needles and syringes it was 100 percent compliance and that was a surprise for everybody because every time that I have presented these people don't expect these results however one of the things that we found is that this weak association between knowledge and supplies and all the other characteristics means that is more about behavioral elements or constraints and that remains the biggest challenge on what to do how to affect behavior you can move on okay some caveats and takeaways to finalize caveats are that we cannot currently link these compliance indicators to help outcomes we don't have a research that actually indicates how to weight them and we are centering this in the interaction between health care worker and the patient so we are missing a lot of things that happen outside of that interaction for instance the rest of the waste management and health care workers may change their behavior while they are absurd we actually tested this and we didn't find evidence for this and the main takeaways first this observational tool was effective for assessing compliance with these practices across every type of health facility in very short period of time because on average it took five minutes for each interaction 99 percent of the patients and 100 percent of the health care workers consented to be absurd which was another thing that was concerning at the beginning for us there is high variance but overall low compliance and the conclusion is that improved improvements will require a broad focus on behavioral change that's it thank you what I'd like to do is suggest that we go through all the summaries of the papers and that we take your questions at the end is that if that's all right I just I'm sorry to say that Marcia Lazarini who is going to present the next study on improving the quality of the part of hospital care for children in Kyrgyzstan is unable to join us there's no other author present from that paper who would like to speak about it anyone involved with that study and see there's no one from your group no okay so we'll skip that one but and we'll go directly to Guarav Sharma over here on my right from the London School of Hygiene and Tropical Medicine who's going to speak about the quality of routine essential care during childbirth clinical observations of uncomplicated births in out of Pradesh in India and if you're finding it difficult to see the contrast on the slides please note that most of the figures and tables of course are in the journals which have been distributed to you Garva thanks okay second on our paper which is in this special supplement and I'm presenting on behalf of the other co-authors and would also like to acknowledge our funders so we chose to focus on routine care because that seems to be neglected in programs and also in research and we did this in three districts in Uttar Pradesh next slide so just to I was only given five minutes so I mean it's a lot to back in the five minutes but all the details are in the paper but we conducted an observational study using clinical observations using 14 enumerators who visited admissions emergency wards labor room postnatal wards wherever pregnant women who are likely to undergo normal vaginal births were and we did this across 26 health facilities eight in the private sector and 18 in the public sector and to calculate the required number of observations at each facility we had done power calculations for all of the key indicators and we used W. Chogailan's and available tools to develop a data collection instrument and we conducted extensive pre-testing of the tool itself and also did a thorough pilot study for three days in the public sector and four days in the private sector to finalize logistics during data collection so we realized that in this setting there are not many normal births that happen in the private sector so we needed to spend a longer time at those facilities and data was collected between May and July so overall we collected data on 42 items of care using a very comprehensive framework that included adherence to evidence-based practices harmful care practices harmful behaviors by the health worker and harmful practices but also aspects of women-centered respectful maternity care and what we find is overall across the board quality of care was very poor with a mean score of 35.7 across all of the facilities so women on average only received about 35 percent of the recommended practices and so neonatal care was slightly so this is only immediate newborn care which was slightly better than obstetric care which were which were both poorer but we found a significant difference between the quality in the private sector versus the quality in the public sector and so 40 so women on average for overall essential care at the time of birth received about 45 percent of the recommended practices in the private sector but only 33 in the public sector and you can see the rest there and those were significant significantly difference across the sector but these are aggregate scores so obviously they hide a lot of variation so maybe if we go to the next one so if we look at certain key indicators the quality was as you can see in these graphs the quality was really poor so for example and and the indicators with a star in them are the ones that were significant across the sectors so regular monitoring of labor using a part of graph less than one percent screening for preeclampsia less than three percent infection prevention was better in the private sector assessment of maternal blood loss better in the private sector provision of women-centered respectful care practices things like explaining the process of the labor allowing a companion to be present offering a choice of position were really poor less than four percent and also for newborn care this was overall poor quality obviously I mean there are problems with many of these indicators if we look at individual indicators and the validity of many of these indicators is also questionable and I mean research work is ongoing to look at some of this but so overall patograph use infection prevention maternal blood loss and assessment of fetal artery seem to be done in the majority of private sector cases compared to the public sector and the other interesting can we go to the next one the other interesting thing that we found is that so unqualified personnel frequently provide institutional maternity care in this setting and in both the public and the private sector but if we if we look at this graph a larger proportion of doctors provided care in the private sector compared to the public sector but greater proportions of staff nurses auxiliary nurse midwives traditional birth attendance dies other unqualified personnel provided care in the public sector um so this was an interesting finding that we found if we go to the next one so uh so that was on the descriptive analysis and then we conducted a multivariate analysis using a mixed effects regression model with a random effect at the level of the health facility to account for the clustering and after adjusting for all of the confounders related to the women's characteristics health workers characteristics and caseload at facilities day time of admission we find that there was no difference no significant statistical difference between the care care provided between qualified and unqualified birth attendance we found that private sector provided a six percentage point a small six percentage point higher overall care at the time of birth we didn't find any association between facility size or the volume and the quality of care at birth and but um but uh we found that quality of care during weekdays as compared to weekends uh during weekends the quality of care was three percentage points poorer so having said that i mean it's it's uh defining that slightly nuanced in the sense that there were problems with sampling the private sector in this part of india there are lots of private sector clinics so maybe every 500 meters there is one private sector clinic and we didn't really have a sampling frame for the or a census for all of the private sector facilities in the district and 13 of the private sector facilities that we contacted refused to participate in the study so their quality may have been different to the ones who agreed to participate in the study and they may also have been observer bias due to the general perception that private sector is better because it has um you know better infrastructure or shinier facilities or better better qualified health workers but so in summary these are key findings and if there are any questions we can discuss thank you back to you margaret for your study and several countries on primary health care services and quality thank you laura uh okay so um so together with with our team um that uh are the co-authors um in this work which i'm also having trouble seeing but i can see them in my uh printed program including some wonderful doctoral students postdocs and colleagues in uh tanzania colleagues uh in tanzania we um ask this question um why does quality vary how much does it vary and and why uh why does it vary uh and and particularly looking at the range of countries that are listed here seven low and middle income countries go to the next slide please um so let me say a little bit about a variation and why is this interesting i think for a very long time in health care in particular i would say in uh wealthier countries with extremely well developed and expensive health systems there's been a lot of interest in understanding variation with the idea that variation that is not accounted for by patient status right and patient need is what's called unwarranted variation and may reflect bad quality may reflect deviation from evidence because if you think about it controlling for what the patient needs uh the evidence base should be pointing everyone in the same direction for the care of conditions certain conditions um and so uh and particularly this is the case for primary care sensitive conditions that require pretty algorithmic set of uh things to be done and so we thought why not use that same approach of variation analysis to the systems of lower income countries and to analyze the the variation levels but also what factors might explain it um so next slide please um so this study is based on a nationally representative uh set of health systems surveys incredibly rich data set from the service provision assessment surveys um from Kenya Malawi Namibia Rwanda Senegal Uganda in Tanzania these were done between 2006 and 2015 so relatively recent data and these surveys contain an audit of all the facility stuff infrastructure equipment medicines staff personnel um as well as i think this is a more exciting part provider interviews direct observations of care and even patient exit interviews so we do have a multi-dimensional view of quality from these from these studies in this study we particularly focused on two critical services antenatal care and sick child visits and the way sick child visits work is basically the observers have served any sick child any child that was in um for any illness uh to be seen by a clinician in in a clinic i should also note just to the point about private and public these surveys include both public and private facilities in all the study countries and the weighting actually makes these representatives so they do tell us about what's going on across the health system next please um so the way that we define clinical quality here was the proportion of essential clinical actions completed out of a very uh i would say parsimonious very small set of what we consider the most essential antenatal care actions and sick child actions so eight for antenatal care nine for sick child care these are taken from global guidelines i m c i guidelines a nc guidelines but we wanted to just really focus in on the fundamentals where the fundamentals done you know was the mom asked about past the for example birth complications this kind of thing um and these were in the domains of history exam diagnostics and counseling and management so again this fans span the range of care um what we then did is we used multi-level random intercept models which is a way to um understand data that are that are clustered so these the way to think about this is one provider might do multiple visits right so the visits were clustered within providers of course one provider several providers are within one clinic so that's another level of clustering and then of course provider clinics are within countries and so we applied multi-level modeling but also included a country fixed effect to make sure that we captured inter-country differences as well next slide please okay so what did we um what did we find first of all there were a lot of data which is a wonderful wonderful thing in terms of illuminating what's going on and i should remind everyone this is data collected by direct observation so somebody standing in the corner of the room with a checklist and going down to see what actually is getting done not not what they said was getting done not what providers that acknowledged us but actually what did they do for these sick kids and the moms um so we have 2600 ANC visits almost 12,006 child visits which is a lot of data in this quality sphere as we've discussed earlier 80 percent of the facilities are public facilities again that's because the majority of provision in these countries is from the public sector um so we also assessed in addition to our key outcome variable was the observed quality but we were very interested to see obviously the factors that might explain variation and one of those factors might be facility equipment and supply so we wanted to um also test that so for example facilities scored on average between 50 and 75 percent of basic inputs and this is based on the service um uh readiness index the WHO has developed uh just to say this facility got the stuff it needs to to give this care um so about 50 to 75 percent stocked um the average quality that was observed to be given was about 62 percent I think as you said 60 percent is roughly what gets done well 60 percent is what got done in anti-natal care and less than that of those eight or nine recommended actions in sick child care I can also tell you these numbers are slightly upward bias when we take the full range of actions that could have been done not eight but really 25 or so that really should be done for a sick child these numbers fall even lower but these are the most essential so this is in a way the best case scenario thanks okay a little hard to see here but um these uh these lines represent the mean and the inner quartile range 25th to 75th percentile of the observed quality of care and this um is for for both actually sorry I don't think you can see very well yeah there are two colors here but if you look at your how's your bulletin bulletin issue um you will see that uh let me let me point you to the page let's let's open it up together um page 412 yes very good okay so all right um so um that's why they're not showing up they're very light bars are for um uh anti-natal care and the darker bars the ones you can actually see on this slide are for sick child care so let me just point a few things here first of all you can see these bars move all around each bar by the way represents one country's worth of clinics so these are across all the different countries um so there are countries that are doing better than others that's very clear from this from this graph it isn't always the obvious things though if you look at Namibia which is the richest country in this group by far actually the richest country it's scoring in terms of sick child care about the same as Kenya um so that's interesting to us why is Kenya a much poorer country doing a better job and then other countries with similar incomes to Kenya on the other hand are doing less well so you do see these inter-country differences the other thing I would like to point your attention to is the size of that bar so this is the the majority of the data are quite scattered there's wide variation uh within each country also of the care delivered what you again can't see but that better in the journal is that um interesting to us was also the fact that countries that did well on sick child care didn't 100 of the time do well as well on anti natal care some did better on anti natal care some did worse I would say that of the two services anti natal care is more formulaic right it's preventive care people are not sick in front of you so it should in some ways be easier to score those eight items but we didn't see those clear um we didn't see that well on average they did do better on a and c but but some countries didn't um all right let's move on to the next slide this you obviously can't read it all um and that's okay because i'm going to summarize what this says on the next slides please this is for the geeks in the room um okay so this is the uh the the quick interpretation of what we found probably the top line is after well there's several several key messages after we adjusted um so we were trying to explain what we're seeing what the quality explained the observed quality um what we did is we tried to use all the factors that we can think of from a well-developed set of conceptual models what could drive quality could it be provider motivation could it be provider codder could it be uh the equipment in the clinic could it be i i think we had 20 variables in this model because the data are large enough to permit that sort of exploration um we also then looked at patient level factors maybe if the if the child was younger maybe if the child was sicker so we included visit level factors as well as provider level factors clinic level factors in finally countries at the end of all that we weren't really able to explain very much of the variation in these data okay so in particular after all of these uh factors were put in uh into explained quality we found that nine only 19 percent of the variation in antinatal care but 41 percent of the variation in sick child care um um sorry let me go to the second bullet the full models explained 40 percent of the variation so 40 percent of the differences in quality in a and c and only half of that in sick child care um what i want to do emphasize in the first bullet though was actually what we found surprising was there was a lot of variation within providers from visit to visit that actually less for antinatal care and more for sick child care the providers clearly could do more and did more for some kids and they they didn't do it again for the next child so this kind of uh intra provider variation visit to visit variation um what we also found though that after all um all our um covariates were taken into account all the explanatory factors were taken into account the single biggest by far explanatory factor for the variation was what country was this visit in 80 percent of the explained variance was due to the country fixed effect in these models um so that's that's a key finding i'll come back to that in a moment um other findings um a and c quality was lower amongst physicians and clinical officers than nurses um was higher in private clinics and was better in facilities with better infrastructure equipment and management um those were some differences um but let me explain the size of these differences being in Uganda was linked to 30 percent versus um the the comparison country which i believe was Malawi in this case the poorest country was the was the comparison so being in Uganda versus Malawi was linked to 30 percent better care while better infrastructure was associated with three percent better care just to show you the size of the country effect versus the size of the uh of the effect of the um of the inputs uh to care so huge huge variations across countries let's go to the next slide please so i think the question is why what what are we seeing and and what does this lead to um so um on this chart what we're showing is each of the set of bars there is showing you the performance of worse and better facilities in each country and um and those of you following this this is on page 415 in the in the journal um what we're showing is that the lightest bar on the right hand side shows you what the top quartile the top quarter of facilities in the country could accomplish so look at that within the same country there is a set of facilities doing substantially better than the poorest facilities which says to me there is knowledge in this country and demonstrated ability to provide some better quality care um and so what i'm i think really eager to do and i would really encourage this is uh uh for all the interns and researchers in the room is an important direction for future research is understanding best performers understanding positive devian analysis understanding what's going on in those clinics that can help guide us um i think the other big questions to my mind are why do countries produce such different levels of quality what is it about those countries that's doing it it's not always income that's not the that's not the single um predictor as you saw was in the maybe a case um and so one of the things that our commission um is looking into in greater depth is whether pre-service education really is all that comparable or sufficiently or sufficient as we look across countries whether that can help explain some of these differences frankly our providers prepared um and then i think the other piece is how do we actually get providers due to the right thing for every child not just for the one child a day but for every child because again we are not asking much in this index we're asking for eight or nine items to be done properly basic uh assessment of the child and so i think that my last slide oh uh no that was just a backup slide for those interested in what exactly was was included in these quality measures so thank you very much thank you margaret i'm sure we're going to come back with questions on this study a number of myself um barbara i'd like to hand over to you now um barbara's going to tell us about some work done with colleagues in who on developing global indicators for quality of maternal and newborn care thank you very much good morning everybody um um sorry um so yes thank you for the introduction so um i come from the liver for school of tropical medicine and i'd like to share with you the findings of a feasibility assessment um run with two so two with four other people but two of the courses are actually here in the in the room today um on quality of care indicators developed through um with WTO so the background is that with the increasing focus on on quality of care but lack of standards there was clearly a need to develop those um as a matter of obviously being able to measure what is actually happening with the quality of care so the discussions had been ongoing um as part of that group since 2010 then in 2013 um 70 um experts got together to discuss what would be the core indicators that need to be included to cover uh quality of care um with regard to mothers newborns and children and they came up with a list of 19 indicators uh six on maternal care five on newborn care um four on general service readiness and uh audits or death reviews and then four uh in which cover children's health which we had not included in our assessments since that's not our area of expertise what we did is uh we did not collect data prospectively for this analysis instead what we did is we um used the uh used data in the repository of the center for maternal newborn house coming from two different programs one um which ran in 10 countries so that's nearly 1000 facilities in Africa and in Asia of different levels so both BMOC and CMOC facilities data collected between 2012 and 2015 as part of a baseline facility assessment so we've used tools um adapted tools from from those um produced by WHO AMAMDD which covered both service readiness as well as outcomes and then we also used um for for one of the indicators we used data that came from a study done post Ebola epidemic again similar tools extended us but adapted to to the to the actual work what we did for the assessment is for each of the indicators we tried to identify um data within our set that that would match the indicator where that was not possible we we tried to use proxies then we extracted the data and then looked at the results um so here what we're presenting is the availability of information per indicator um and then as next step we tried to assess the indicators with regard to the clarity of the actual indicator the wording of it but also the availability of data with within the routine registers and facility records and um where when necessary or where appropriate we try to make some suggestions in terms of how you could possibly make the indicator more feasible for use so this grid gives you an overview of that matrix of clarity versus availability of data I don't necessarily want to go through all the details obviously you can you can read through those in in the actual bulletin but what is maybe interesting to to notice that um out of the 15 indicators we looked at we've actually so 10 of those are clearly defined already though only four are ready to be deployed as is at the moment um we've got um five indicators for which data are already available and then the further 10 need maybe um additional sources of information or certain tweaks to make them to make them usable in terms of then be the major issues for indicators which were not clear not ready to use um for the clarity and adapting of of terminology um so there are examples such as use of terms like prolonged labor which needs to needs to be unpacked it doesn't kind of necessarily follow the same standards across the board it is not necessarily used university across similarly for severe systemic infection terms such as operational again maybe subject to to some interpretation and then use of stockouts so I know that last year there was a discussion led by the by the WHO in terms of what stockouts actually mean and I think the systematic review showed that there were 56 different um definitions of of the terms so obviously that's that's a that's a recognized limitation but one that also affects the the indicators proposed for for the quality of care assessment in terms of time frame so again some of the indicators do say that within a specified time frame it would be useful to actually um already specify this probably for things like um so definitely for things like availability of drugs but also for for other indicators to make sure that the standard of measurement is is same um across the board so you can make comparisons more more easily in terms of information availability so we've got some indicators um whereby information is available but will not be available through routine records you would need to go through patient records for those so um things around uh measurement of of blood pressure amongst women coming for A and C visits but um also for for women being treated for clamptail with magnesium sulfate but then there are others like the one on receiving oxytocin within a minute of of birth now that's that's a very good indicator in terms of being very precise whether that's actually feasible to use in the in the field this is maybe a little bit more questionable so um again something that uh that might require maybe slightly different sources of information such as observation or review of patient notes now some information is just generally not available immediately in the in the routine records so that comprises um essential elements of care for newborns um kangaroo care um availability within facilities and oxygen supply so you would again need to maybe adapt some of the tools to to be able to answer this now the limitation for that is that it's got resource implications so again um the the balance between having a lot of information information that is useful um but if yes if you need to adapt tools if you need to start doing observations it will cost more money you would need more people to to deploy those those indicators in terms of data availability so where indicators were identified as available generally our finding is that you will find information so i think in terms of non-availability up to 10 percent of facilities across the different countries would have missing information which is really um somehow it shows gaps within a particular facility rather than a system as a whole with the exception of obstetric complications which are notoriously difficult to capture through routine records the registers rarely have specific columns that would capture this information instead it will be added somewhere in the in the um in the labor register but that's the probably the the indicator which is most volatile but it affects three of the indicators that we're actually trying to assess quality on so it's an important one to make a note of in terms of missing information similarly for newborn death um information on weight categories not necessarily available throughout um maybe sort of that's an indicator that could be somewhat somewhat simplified before it's deployed and um also in terms of sepsis so women with with sepsis the indicator at the moment suggests that we should look also the readmissions which again a great idea and a kind of a great way of of measuring um quality of care but linking records of women who might have been discharged and then coming back to the facilities is very difficult to capture on uh in the through routine assessments and then there are also some country specific challenges so in terms of stillbirth countries like South Africa do not have records on the macerated versus fresh stillbirth everything is combined together here we're only trying to look at inter interpartum um and then uh similarly for kind of for other countries there are gaps so Asian countries Bangladesh Pakistan not necessarily best record keeping in terms of stillbirth so in terms of the overall assessment um the the the indicators that we looked at so the 15 um that there are that great in terms of the scale of the work so they include um input process and outcome indicators they cover um the the critical elements of care which which need to be included in terms of having a sort of a tester for for for assessing quality um there is a mix of denominators so some of the indicators cover individual patients are this goal to facility level others yet need to be extrapolated to districts which is good on one hand because it gives you a mix of um somehow perspectives but at the same time at the same time it makes it a little bit maybe uh more challenging to make it standardized um and then and therefore usable and um a point that um in terms of kind of the different perspectives um that's only the camp provision perspective that is included in the indicator and in these indicators so the overall conclusion is that further work um to ensure the usability of of indicators on quality is is needed however the the list of indicators proposed in in 2013 and 2014 is is actually is a very valuable contribution to get the discussions going to um start putting a framework together in order to be able to assess quality of camp thank you very much thank you bible thank you so i hope you're all holding on to your questions for the individual authors here we're going to hear the last one in the series now and then we'll take those questions claudia hanson from the carolins Institute is going to present a meta analysis of random mice control trial data on neonatal survival and community approaches to reducing that thank you yes thank you very much giving me the opportunity to present our work and and i present you on behalf of my co-authors for sure and i would like to put it a little bit in the context not also what we heard before so this paper is a little bit written with probably more of the same is not the right thing to do so that's how we actually started to uh to to think about and conceptualize this paper and moreover at the beginning the title was actually what can community approaches actually help for weak health systems so it became something a little bit different what it's now but i think it has quite good messages so we so i was involved in the in the uh in a big trial using a community approach with home visits to improve newborn health which failed and that was also one of our point of departure so we looked a little bit through what has been published and the background is that 2009 WHO said home visits to uh for mothers in in high mortality uh regions can can reduce neonatal mortality probably by something like 40 percent so then there were quite a few meta analysis done then there was our trial and also another trial which we which we identified and we did a new meta analysis and what we did differently looking at chili and trying to ask out so the question why did our trial fail is we sorted them by neonatal mortality and saw that there was an effect actually when you looked at the trials which were can you continue no no one more yeah so there was an effect if you looked at the trial which were really done in very high neonatal mortality setting of something like 25 percent when it was lower still high mortality it was only 10 percent and then when you go down and the trials which were done in neonatal mortality setting of below 30 there had no effect at all anymore so the next next slide yeah so then we also looked a little bit at the health system factors and we saw we looked also what did these trials actually do not all trials actually reported whether they were able to improve facility delivery which is a key thing because you want to improve the uh the uptake of care and it's clear in settings where facility delivery was already high these community approaches didn't do actually much then we also looked at the different context these trials took place and we realized that one of the trial which was very effective in reducing neonatal mortality had only two facilities per 100 000 population but our trial which failed had 17 so there was much more uptake of care already going on in our settings than in other settings so we saw really a trend that these community approaches worked less so in lower neonatal mortality higher uptake of care and settings with a with a actually better established health system the next one so what does it have to do with measuring quality quality of care so what's the implication now for measuring so I think it's important to know what you you have to know your context where you do what what you want to do and I would like to really strongly express that we have very different context and we have very rapidly changing environments and I like this picture a lot which is from a small hospital in southern Tanzania so I worked there 2001 and I came back to do the trial and this small facility had 500 deliveries in 2001 and had more than 2000 deliveries today and access completely changed their mobile phones their motorcycles as you see here a completely changed environment and 80 percent of the women today deliver in in facilities so why why we have been good now really to see if you press one more really to see how many women are coming we have the demographic and health service we really know this we have a pretty good idea we also get better if she's up with the readiness of facilities we have the health provision assessments but we are not really very clear about the clinical practice what is really the effective care delivered and we assume that our trial got really stuck in the health facility quality of care so improving actually that women are coming and that they bring their six children doesn't mean that mortality declines if the quality of care is not good enough and quality of care is something really about giving the right woman the right child at the same at the right time really the right measures which is pretty difficult me as an obstetrician I have been struggling for this to learn actually five years so the next so how can community actions now support the delivery and probably also the measurement of good quality of care so I think these approaches which we have been looking at have much complemented actually weak health system so the question is now can we design other ways which are more supportive better strengthening and can improve some accountability so we would see this as a way forward actually what community approaches which I really needed could do in the context of the weak health systems thank you very much okay thank you Claudia the floor is now open so who would like to have questions either about the conduct of this study the results that the authors have shared with you here areas for future research or difficulty with with indicators um take your questions answering please thank you very much just to comment on the last paper because um you metro analyzed a lot of the trials I was involved involved with I'm slightly confused as to why you would metro analyze what's called community-based interventions I mean if I said to you metro analyze hospital-based interventions you you laugh at me because hospital-based interventions cover everything from surgery to medicine to pediatrics to obstetrics um in the community these were totally different kinds of interventions one you know the women's groups were very largely working in rural extremely remote poor deprived marginalized populations with high mortality rates and the mechanism of reduction of mortality was much more to do with changing behavior social support and if it did affect choices about care it was perhaps improving the selection of care at a time of a crisis where you're faced with very limited options so I'm finding that I absolutely agree with some of your conclusions by the way about the importance of context but I don't see how you can metro analyze such different interventions yeah anyway you're you're fully right and we have turned around this paper many times since thinking too right so we have the criticism ourselves so that's that's clear but still all of them actually wanted to improve certain healthy newborn behaviors yeah like breast feeding and they all wanted that women seek care so in that they were actually so the pathway was on so the approach was different but the pathway how they could affect neonatal mortality was probably not so different because all of them wanted to have more breast feeding less warm care not putting the baby in cold water so they all wanted to achieve this and care seeking when the baby was was sick so the pathway was probably a similar thing but the approach was different and I also think in this I even think that the in in in the new context we have now because I think it's really so much different than 10 years 15 years ago in my for me it's I have been in places where I've been 15 years ago it's completely different situation I assume that this accountability and the women approaches actually have more potential than really the home visits yeah because they can now open up and actually support the support the health system so I agree fully with your concern and this paper actually drove me mad because I have done all this meta analysis at least 50 times who's next yes please very much for all the presentations they were really stimulating and I think it's unpacking an area where much more needs to be done as Margaret said many questions I guess but to start with Margaret when you described the variants you saw across quality of care and systems you then mentioned zooming in on pre-service training as a potential factor to say my dad explained some of this variation when you looked at the systems that countries have are you thinking also of looking at other drivers that may look at quality such as incentives for performance insurance could you speak a bit to that and then for the last presentation on the community interventions you mentioned in settings with very high mortality you could see a positive effect of home based care and could you analyze could you describe in more detail what was actually required to get that intervention on place was that done in a study context with all the support or had that dimensions that were more comparable to routine health systems because where the system is weak and you put an intervention that's not supported then that results that one may see in the study that is fully designed to kind of compare the intervention with the control that and that effect may not be replicable Margaret would you like to start and then Claudia on the next one thank you so much Bernadette you can see Bernadette is one of our commissioners it is getting right to the heart of the matter the absolutely so pre-service education was one possible explanation for the very large country effects that we were seeing but I think the broader point here is that there are factors above the clinic that's really maybe the main point to make they're all baked into this word you know Uganda or maybe that we that we essentially use in the model as a stand-in for everything that's going on in the broader environment for these providers delivering care so I think going down your list absolutely so pre-service education but even before that the selection of providers and the secondary education that they receive and are they ready to become a health professional is a harder perhaps and an incredibly important issue and then once they graduate what is the what is the payment environment right how is the motivation contributing to this to this performance absolutely insurance system what about regulation and how how do they feel do they feel the obligation really to to do their very best because we do know they do less than they know so absolutely all of those factors would be critical to examine and the reason we were not able to is because actually there aren't good systematic measures of them that are available for us and that's going to require much more in-depth studies within countries rather than so much across countries so thank you for that question so I will add on a bit on the on the context the first two the papers mostly really reported on too little also on the context they were all done quite program in programmatic to context so all in in in normal health systems and many of the differences between the health systems we actually couldn't analyze because this data are not available so like for example health workers and how fires the service but it was very clear if you looked a little bit more in detail that the trials which were done in lower neonatal mortality settings had higher health in newborn behavior and that we also would expect here they had higher breastfeeding higher facility delivery better warmth care so and then it's actually logic that the effect of of counseling the women better can't be high anymore because how how should it how should it work if you increase facility delivery by five percent what should be the effect on neonatal mortality it can't have yeah because it's not it's not logic and that brings us back that we have to know this and we do not have to and we have to know it for certain settings because they are huge variations also in the countries so not only in India but even in Tanzania you have a facility delivery between 30 and 90 percent you have to know this yeah and one approach of community might work in that setting but might not work in the other one and we don't have the measurement data for that we cannot even if if development aid goes some somewhere they don't know what is right for that setting because the data are not there yeah we have them we have something on the national level but we have seen with a large last paper also clearly at the subnational level at the facility level the data are not there there's a lot collected but it's it's I could now report for hours why it's not usable you're running off the hook yes please over here all right thank you I'm Andrew from Malawi I'm an interested party if the presentations congratulations of the presenters the question I have is how can we translate all the recommendations and findings into action thank you we'll go we'll go along left to right Barbara one of your findings translated into a recommendation please and we're going to ask every author to give us one thank okay so thank you that's actually yeah it's not an easy question it's not easy for the first one to have to answer it and in terms of the recommendation for the indicators I think the work is already ongoing in terms of updating those because by all means we need to have measures that are standardized that can be usable in terms of action for actually changing quality it's then persevering and measuring not only for the sake of measuring but actually doing something with the findings so it's the whole quality of care loop that's probably not a very specific answer but in terms of the the indicators yes they need to be updated and they need to be so we need to have a set that would be usable across and at different levels okay you got that update indicators right and okay for for these and Kenya measurement basically the next step is what interventions to try in terms of behavioral change for hand hygiene that is the cornerstone of IPC so I have been thinking already on some interventions to do the government to try to test because that's the most important is the knowledge is not and a supply so how do we change that so that's one but there are other ones that you can think of there the million dollar question and I hope our our second panel will actually get exactly the heart of your question because there will be policy makers who will tell us some of their perspectives but to me it is the combination of two things passionate leadership and country that doesn't come from research that comes from something else and but secondly that passion and that commitment meeting relevant timely research on topics that matter and that part is within the realm of the research community so are you wide open please tell us no well I I'm not going to give you the the full answer but I mean I might want to just make a reference back to somebody called Cain's he was a key economist and when he was thinking about how do you influence policy making he did say well first of all we engage but he did also say that what goes on in policy mine policy makers minds often reflects finding from previous research and academic insights maybe the link is not direct but the link is there so bringing evidence to the table bringing evidence forward to policy makers in a clear and useful way is an important element in this process and I would say that that's sort of part of what we are trying both with this issue but also of course with the policy relevant work that is carried out the country level thank you okay over to Claudia I'm not sure was any question still addressed to me I'm not sure the question is for each author what is one implication one action from your findings one implication of your finding so are you suggesting that community approaches should no longer be funded once countries reach a certain development level they're not going to have any effect I think they we should we should rethink the role yes accountability yes yeah but just supporting and preaching facility delivery when facility delivery is 90 percent and the quality of care is crap doesn't have anything here here all right over to grab and that's one so from our work I think the main recommendations were that measuring quality of care is important and we have presented an overall essential care at birth index but if people are interested to look specifically at newborn care or obstetric care they can also do that so the index can be disaggregated into different domains so we are advocating that there's more measurements a systematic measurement effort including in the private sector which provides a substantial market share of maternity services and there should also be further investigation into the high prevalence of unqualified attendance working providing institutional services so what we found is in the public sector because human resources shortages are so immense doctors are not available they tend to rely on unqualified personnel in the public sector but whereas in the private sector they're much cheaper to hire than say you know hiring a qualified nurse or and you just hire them train them on the job so they're much cheaper so there needs to be further work looking at why unqualified personnel are being used and lastly I think we need quality improvement initiatives in hospitals on a large scale and link those two functioning accountability mechanisms so those are the three so me as an obstetrician I always think there's really a huge danger with this facility delivery if the care release not good we had the highest ever maternal mortality in French and and a few other European hospitals some 200 years ago we don't have them that high in the low income settings that was because of all this issue of sepsis so if if hospitals don't provide good antisepsis techniques they actually could kill mothers instead of helping mothers and babies and that's my my really big concern so and we have done a lot with really getting the women now in and that's nice but it's it's more or less now really our safety obligation to to get this now also really safe otherwise we are doing the opposite of what we really want and I'm there always a little bit worried yeah crap is maybe a little bit too harsh yeah but I'm really worried as an obstetrician and worried because of the over so of the sepsis and also using too much interventions and influencing too much deliberate care also then in a way which is is not really helpful for the mothers and the babies thank you so so I just have a comment a question really a bit of a reflection maybe especially for Margaret and Gaurav but it's really for anyone who's been working with an index so I came along here thinking measuring quality of care I'm going to hear lots of geeky presentations about psychometric properties of indices and all that stuff which has a lot of baggage attached to it and I haven't heard anything about it at all so I just wondered if you had any reflections on that I'm quite happy not to but but actually then maybe there are some issues there that are quite serious about measuring measuring things and what do they mean sure I can start and then Gaurav well Joanna be careful what you ask for we can certainly delve into that as well yeah you know we've spent a lot of time thinking about this this issue which is really how do you aggregate quality is a multi-dimensional construct so let's start with that we I said this many of us said this many pieces matter right and we get it glimpses of it really coming forward so in our group at Harvard and within the commission we're actually spending a fair amount of time thinking about the psychometric properties and we've examined different ways of combining because they're really the key questions how you combine these indicators into a whole you know principle component analysis which ones matter is it you know how do we weight them what's more important and I have to say having explored and this is a very partial answer and it's about these particular observed care elements so this may not hold for every indicator in the book but one thing we're finding is that it's actually pretty simple additive methods do a very good job and are very clear and I think this is about also important and to Andrew's point the clarity of explanation and being able to say what is it we're doing there's no hocus pocus we're doing the following the one thing we are thinking hard about in in the group is obviously there are sometimes many more kind of microscopic actions that sometimes accrue in a history taking right whereas there are fewer available physical exam items and so to the extent that both history and physical are important sometimes what we're doing is we're taking averages across those elements rather than weight you know because if you don't do that then of course the item with the most elements wins in terms of its weight so we sometimes are doing equal weighting of the components of care based on the basic medical notion the good medical practice requires completion of steps of care that's about the only adjustment I would say that we're making but I've seen others do you know much more and we've tried and don't find it illuminates go everything wanted to say something so I mean we had a lot of we had a lot of data so we thought carefully about how to make sense of all of this and for the 42 practices we essentially map them so for the 42 items we map them on to clinical practices so based on their logic clinical logic as well as purpose so for example for let's say something like active management of third stage of labor so if everything was done only then were they given a one otherwise they were all given a zero where some other indicators so for example monitoring regular monitoring of patograph use that was a single indicator versus some of them were composite indicators I mean I know other people that have kind of undertaken Delphi type methods or use even advanced modeling and all of that but we tried to keep it very transparent so that you know people could use it as it is and we said at the very minimum these are the essential practices that must be done for every case so we've tried to keep it quite simple but obviously I think although it's useful to communicate a large amount of information when used for quality improvement purposes people will need to really look deeply into what is driving the index so for example let's say skin-to-skin care might be really poor which brings the entire newborn care index down or respectful maternity care might be poor which is bringing the offset to care index down so I think it'll be important to look at you know with specific components and further I mean why does why does a particular thing happen so for example if rates of active management are poor is it because injections are not available or is it because of training is it you know what is the issue so it's like so although they're useful to communicate in a concise way I think there's lots of limitations thank you very much I mean that is obviously an extremely important topic let me just reflect not specifically with reference to the health sector but but but more broadly within the development field I think one of the key lessons that is emerging is first of all the clarity and transparency in both the definition and the use of indices is an area that requires more focus more attention than it has in the past there's been a tendency to put forward interesting fascinating indices just to sort of provoke the debate did you know that the multi-dimensional poverty index used by the UNDP does not adhere to the declaration of human rights that is actually not respecting the fundamental rules of the game according to that so I mean what I'm trying to say is that as researchers as policymakers we really need to try to insist on clarity that we do understand what goes into a particular index and we actually understand what are the assumptions lying behind now that would often lead to simpler methods simpler more transparent indices and then in all cases make absolutely sure that they're unpacked that you do try to think about robustness that you do try to think about what the individual elements of a particular index actually is doing and then finally there are methods being developed now where instead of relying on indices that you can think about how the whole distribution is actually moving this is an area that's called first-order dominance but there is actually work there that can push our insights forward which do not depend on the exact waiting of specific indices thanks yes just a quick point to build on that because I think Senator Margaret's points on on waiting has always come through and I'm glad it's been brought up although probably better that it wasn't sort of a feature of our own discussions here yeah someone who spent a lot of time on on indices both in the US and then on for work at WHO we are working there's continuing work in a number of different areas there's there's a tension between colleagues of which I sort of would consider myself from from the data side who look at these indices and sort of roll their eyes because everyone knows just based on the points with Finn's comments on new science accepting however you wait it countries will end up as first on the list or 13th on the list or 15th on the list or communities or whatever the group is and but for instance when I when we did the first US national health care reports I sat in the back of the room on while my boss presented my work that's a strong tradition in big bureaucracies and the sort of the committee staffers were there and some of the representatives and senate people were there and they would hold the books out there and they would flip flip flip stop at a map flip flip flip stop at a map flip flip flip stuff so they were looking at these composite pieces we put together for the states based on state sort of how they were doing on prevention or how they were doing so they matter because they grab you can sum up very complex things and you get people's attention but immediately you need to explain then to say well here's the reason why and personally I think if we could move more particularly in the quality field because one of the issues we also have on the data side with indices is that often these studies were looking at specific populations so that's one solution to it but often when you're talking about bigger groups or policymakers they're concerned with different groups of populations and right there the statisticians heads explode because you've got elderly people and that you're trying to put in one index with perhaps care for newborns and it just doesn't work and so for me one of the things I think we should move more on is this whole idea of how what's the rate of travel how are we improving and rate of improvement and percent improvement and in there you can start looking across different measures and how quickly things are improving so personally that would be my we had to come with come up with one index in the future that we measured it would be how it's on a subject you'd approve of. Could you just explain the Uganda findings being 30% better and the appalling Indian findings on quality of care by simply the quality of training of midwives because it just strikes me that Uganda has probably got better qualified midwives that's why this is just a hypothesis and we know that in India there is no midwifery cadre and that auxiliary nurse midwives most of them when they are passed out and given their certificate of training have never actually delivered a baby and this is some new stuff that's emerging and actually we're working with the ministry and with others at the moment to see if we can come up with some solutions to this to try and develop midwifery within India which is a major issue because one of the major obstacles are other cadres of health workers that actually obstetricians and nurses are not that keen on having a separate cadre of properly trained midwives which doesn't seem to be a common thing in Uttar Pradesh and finally WHO needs to get its act together because the definition of an SBA is all over the place and everyone is calling I think on your data you'd probably classify the doctor the nurse and the A&M as an SBA but actually most of them don't have it I think I'm an SBA you know I'm qualified I'm a doctor I've actually done lots of deliveries I'd be a disaster at a delivery so Anthony quick response and then a great question and I was just referring back to our table four in the paper where we show that for anti-natal care compared to a nurse or midwife physicians had eight percent lower performance actually and nursing assistants three percent lower so this is now adjusting for all the country variable what would be very interesting to see how those nurses midwives performed in say Uganda where the training might be better but these are precisely the kinds of conversations that need to be happening we need to be looking at the structures the basics I say no I completely agree that's that was one of the reasons that we decided to use the term qualified versus unqualified because we didn't really assess the competencies but I know like Japaigo did a study recently where they looked at the training curriculum of auxiliary nurse midwives and said they don't fulfill the you know all the 21 competencies outlined there I think I mean in addition to measuring who an SBA is I think there's not much research on the enabling environment for SBAs which is the main issue so I think maybe in measurement efforts trying to measure the enabling environment at health facilities rather than I mean and focusing on the content of care rather than you know is it a doctor or a TVA or whatever that's okay one last comment from Claudia and then we're going to have coffee I just would like to make two comments first with the midwifery and why it is important so I'm also always when I'm teaching I'm saying just look at the two books from WHO the integrated management of childhood diseases and the integrated management of pregnancy and childbirth the child houses speak like this the pregnancy is big like this so you can't do it as an in-service training you cannot put it on lowly trained health worker the second point is the is the retention of of of practices most of the nurse midwives are rotated every half a year so even if they are a little bit getting into it after half a year they go to urologist or wherever because it's not a protected card if you would have midwifery we also would have a protected card which stays in the maternity we're going to take a coffee break and come back at 10 past please coffee's outside thank you thank you lively we will be discussing some of the critical issues around measuring quality that's been raised in the last session as you may have already seen in the editorial this issue highlighted one of the reasons for the lack of data of quality of health care services in limited income countries was this emphasis during the mdg era that margaret mentioned on coverage rather than the challenge of providing high quality services and then as highlighted by the authors earlier and presented in places where studies have been done these quality measures that currently exist and used have their own limitations some are not sufficiently validated to be recommended for use in other limited income countries for various reasons measurements of qualities were not done consistently and and therefore in such cases were not able to generalize the findings be beyond the steady settings and compare these findings for other stations so what we want to do in this next session is to give our presenters an opportunity to comment specifically on the validity and the practicality of some of their studies so we'll pose two questions what do they think are the main challenges with measuring quality of care and what do they see are the greatest gaps with measuring quality of care since we don't have a lot of time they don't need to answer both they can pick one and and be able to and comment we'll give about few minutes about three four minutes five minutes for comments and then we open it up for discussion so we have again our authors for this panel i'll start in the same order okay okay thank you i have chosen the question what are the greatest gaps with measuring quality of care i'm going to focus on two main ones the first one for me is a tools data gap in my specific example we chose or we built on tools that were already developed validated by the WHO and they are available in their webpage and also in the CDC's webpage and we were very surprised that there was no data even for a few health facilities in Kenya and in many countries in Africa some of the data that was available was like one decade or 15 years ago so it was very difficult to find you know like information about what is the status what so i think one of the things even though we need more data and more different type of data we are not even we haven't cut up even in terms of for the tools that are existing having like basic data from countries especially in low and middle income countries and nobody knew for instance that you know the compliance with a new syringes and nails was 100 when we presented these results in the government they were happy but it was a shock people couldn't like for two minutes they couldn't talk right like you i couldn't follow for for two for for two minutes the other things measuring for instance knowledge and practice and supplies at the same time gave the government a lot of thought because of course you know like the first thing that you think is like we have to train healthcare workers and that wasn't the main constraint so it allows you also to focus more on what is the next intervention or what is the next thing that you would like to test and another example is that our main intervention is inspections of health facilities in Kenya so we have two different treatment arms and when the government was designing the new regulations for actually for random for the random control trial they have no information on how the health facilities in the country or even a sub sample would face or would would be scoring according to these regulations so we did a pilot and based on that they actually defined that they were they couldn't close all the health facilities because 97 of them didn't comply with the minimum so they changed their regulation so to give people the health facilities to improve in a year so that's the power of data that actually is not there and my second gap is the link to outcome gap and again the main issue that we have found is that it's very difficult to determine what matters the most even in this checklist that I was telling you about it when they were trying to put a warning sanctions into this new regulation they just decided to put an equal scoring to each of the 300 items because there is not a lot of evidence and they didn't dare to actually put more scores here and there so there is very little evidence in a lot of these indicators that we are testing how strong is the link to the outcomes to see which ones matter the most in the example that I show in the paper for instance it's two percent of hand hygiene but does that matter more is that more important or is equally important that is in the examination or in the lab or the injection room or in other settings of the health facility is that more important that other you know waste aggregation when we cannot actually follow so try to understand where to put more weight because that will define in a setting that is very constrained in terms of resources it's very difficult to go to the government and say focus in these 500 items equally and we could have more information and again we try to find evidence we couldn't find a lot of evidence that allows to actually link all these indicators to outcome those are my two okay next would be great thank you Kitas you asked first about challenges I think they fall into two categories the what should we be measuring and secondly how are we measuring it I think I know what there are legitimate questions about what elements of quality matter we said there are very many elements I don't think there is sufficient clarity we even saw it when the papers were submitted for the special issue there are many people doing many things some of which are peripherally related to quality it's difficult to say something is completely unrelated to quality actually because it's such a large construct but but some clarity about common definitions I like the point I think we're made several times about even for very narrow sounding indicators there's just not agreement on how are we supposed to be measuring that I think what to measure matters at the micro level having the same definition for indicators but I would even say at the kind of deeper level like where should we be investing our resources in measurement measurement is expensive it's hard to do so of the thousand things that we can identify that are related to quality which 10 27 should we be measuring and one pitch I will put forward and this is work that's ongoing is that while inputs are important to to quality for sure you need stethoscopes and medicines they are not themselves actually very descriptive of the quality of care that people receive and so one push for what matters is further along the continuum toward what actually happens to people in the care context and also their outcomes so that's that's my vote for what in terms of how I think we have a couple of problems that that we're seeing what is survey crowding I think we keep complaining there's not enough data but I actually see in some ways the opposite there is a multiplicity of surveys when I sit when I think from the outside sometimes it seems like oh there's not a lot going on or datasets are not available that we could look at in the way that every for example wealthy country looks at its own data those datasets are public and people can can can examine them that doesn't happen as much it's improving for some of the surveys health facilities surveys for example and others but when you actually sit down with a health ministry official in charge of data they say oh yeah we've done 10 surveys of facilities in our last in the last five years I'm thinking about our friends and colleagues in Ethiopia and I know we're going to have the former minister with us but I mean that is a data rich country the service provision assessment was done a very large one including a census of facilities just a few years ago a gigantic census of basic emergency etc care was just done a new spot is going to the field and I think that's maybe an extreme example but there's a lot of surveys now the flip side of that is that often the only people who know the results are the people who wrote the report and I do not see a direct link between many of these surveys and action in fact I don't see reports and strategies citing these surveys even though they're maybe very well done and nationally representative so there's a big disconnect and a lot of what I'm thinking of is dormant data in countries so on the gaps I think there are a couple of gaps I want to emphasize I think there are many but I want to emphasize too one is the people's voice the user voice in this entire enterprise you know my entire research career did not start with assessing health system quality it actually started by asking people why do you or do you not want to use the health system I was interested in utilization and I was interested in people's preferences because I was struck by the very different levels of utilization of essential services in sub-Saharan Africa so working closely with colleagues there over the past decade it became clear that quality was a driving force in people's decisions and so now we're shifting our attention to understanding okay well what quality are people getting but I'm not forgetting the user I think what's really incredibly important is understanding people's experience I think the maternal health community has really grabbed this agenda with respectful care and measuring that and developing ways to think about that but also the outcomes of care that mattered to people already mentioned this before patient reported outcomes perceived quality what do people think about the competence of this provider because we may or may not agree that they can tell all the elements of course they cannot in every case but it's certainly their perception certainly will drive what they do right so knowing that perception is important and by the way that is linked with another complete gap which is understanding what is the demand for quality in any one setting I think why many of you will be familiar with the observation that many people are suspiciously satisfied with care that some of us I think would probably many of us in the room would consider completely terrible in many countries why aren't satisfaction ratings so high 90 percent 85 percent 94 percent you know and and of course social desirability bias and surveys people wanting to be nice and kind are all the driving things but another factor might be that there's actually frankly just a very low expectation that people have and as we think about this issue of improving quality it seems impossible to move that mountain without people demanding better care and so thinking about demand generation in the population for better care just like we drove demand for utilization right is absolutely critical as a gap and I think the other final gap to mention is that and this relates to your question about how do we aggregate things is it's one thing to understand how how people fare for particular need or service but for the ministry of health they're often more interested in how is my primary care system performing or how are my hospitals performing is referral working so rather than taking a disease by disease view or condition by condition we're interested in a system assessment of quality is the system performing as it should and I think that's still going to require quite some thinking about what's the right set of things that convey a system quality performance yeah I would like to add on a on I agree with all what has been said and these are the known challenges I would like to put now the health worker as a key person which has not been put in the center of quality of care but these are the ones actually providing the quality of care and we meet with the data we collect we also need to trust the health worker like we have done actually in the in the high-income countries in a health worker reports active management of labor is done within a minute then we cannot come up we better do observation to check whether it was really done in one minute yeah so I think I find this ridiculous so we have to trust them really to some extent because these are the people in the end really providing the quality of care and we have to listen also to to what they say we have done a qualitative study and the health worker said it's all like rain sometimes you get supervision sometimes not sometimes we have drugs sometimes not so that is also not a good environment for really providing providing quality quality of care so this is a this is a challenge the other gap which I see is that we have been under utilizing a health management information system so we have a lot of service they are somewhere they're even not on repository so districts can't use them districts don't know what is there whereas the health management information system is actually in their hunt but it not in a way that they can easily actually use them and it's also very difficult to use them for for more comparison I just said a PhD student who tried from Cambodia and it took a whole month to compile from three provinces data over over two years this cannot be so they must be somehow readily available readily like you have it actually with the demographic and health survey so we have to we think with our technology we have how can you utilize health management information system to drive quality of care I think I mean so much has already been said but just to add to what Margaret was saying I think linking information from different sources different data sets that are already available for example I mean data from clinical observations with EMOC assessments or service provision assessment results and actually seeing what is being done or that is important and also I think it's it's useful to try and get as much data as possible so for example there is some evidence that provider effort is the key determinant which determines like quality of care so information on their motivation or the incentives that exist or management structures within the facility I think getting that sort of information is also quite important apart from that I think I think having a strong research team locally who knows the in that community and ensuring that high quality training program is given to the research assistance is very important because some of these concepts so for example around respectful care or other things you know you really need to pay emphasis to these things and like observations are a good method but they also have many limitations by themselves and there are also many other methods to assess processes of care but I think one thing to keep in mind is the possibility of any observer bias or author and effects that can occur and if you plan for these things from the start if you if you I mean make sure that you can order the observations by the time that they started by every individual research researcher you can kind of account for all of these things so I think those are some of the challenges okay thank you very much so I think I'd like to echo a lot of the comments that have already been made but just maybe to add to the to the discussion on challenges in measuring quality of care so as I mean as already has been mentioned the topic itself is really broad so looking at the example of the special edition of the bulletin I mean the diversity of topics covered is broad therefore kind of it's quite difficult to maybe find ways of capturing things in a uniform matter so one of the aspects is the multiplicity of stakeholders involved in the work so at global level you want to have trace markers of quality that will be applicable throughout the lower down you go in terms of facilities there are many more that that are important that need to be measured and reviewed locally to improve the quality at the level of the facility in terms of the user perspective so that's that's kind of somehow almost a different type of discussion in terms of what Margaret has already alluded to in terms of what people's expectations are which sometimes are dictated by lack of experience and kind of but sometimes it's kind of its cultural aspect as well where you don't challenge what you receive obviously having some sort of care is better than not having any care which then links to benchmarking what kind of how do you define quality is receiving any sort of care and improvement on no care of course you want to strive for the best by kind of then you also need to take the limitations of whatever is available on the ground as a limitation of the kind of the level of quality you can aim for and then the big issue is then is really around data so we've I think kind of it's it's it's nothing new to kind of to say that there are quality issues in terms of data available within the facilities to help measure things but it's also an issue about timing so sometimes so data should be recorded at the time but by the time they are available very often they're out of date so again it's already been mentioned the context changes very rapidly things change very fast so how do you actually tap into it in so sufficient quickly to be able to act and actually improve quality in terms of source of information so within kind of our work we've looked at facility information so the registers and records do not cover everything that would be relevant the systems that are used like the HMIS will not cover all of the aspects of quality that are important it should somehow be measured but then there's then this issue of the alternative sources of information like exit interviews like observations which I've already mentioned in my presentation have further kind of costs and resource implications which need to be taken into consideration but that's a challenge for measuring quality but then another issue is the difference between the records that you will come across and what the reality actually is so there are so the poor recording systems is maybe one aspect but there's also sometimes deliberate action on behalf of the people capturing the information who are under pressure or for kind of alternative reasons put in information that isn't necessarily reflecting of what is happening so the maybe kind of find out a standard example is filling of paragraphs post delivery because you need to meet certain numbers so kind of the quality standards you're trying to fill but filling a paragraph once the baby is out doesn't help anybody apart from maybe the record-keeping staff but then also the fact that if kind of the quality improvement really is based on a punitive system so if something goes wrong the only thing you'll get is scolding from your supervisors from people from the district level chances are you maybe you will not want to share information if you don't want to share negative outcomes chances are you will not be able to improve it because then nobody realizes exactly how how big an issue is and then the final point maybe in terms of linking of of coverage and then kind of measuring quality versus outcomes so the high rates of of coverage we've already discussed doesn't that doesn't necessarily mean that the quality will improve so an example here would be the WHO multi-country survey on maternal newborn health from 2013 where out of the 29 countries assessed the ones with highest coverage rate for essential care were also the ones with highest mortality so there is obviously kind of a quality gap but how you tap into this again maybe it's not so clear so kind of again the safe childbirth data analysis which again shows that coverage is improved adherence to essential practices during childbirth is actually improved if you look at the standards being followed but the outcomes don't necessarily follow this so then the question is how do you tap into it so these are some I hope kind of um again not kind of to paint to to better picture of where things are because obviously it's a complex issue it's one that requires discussions it requires kind of coming up with some sort of consensus on how you measure it but challenges exist and I think kind of it's an exciting challenge to have to maybe try to input into how to how to overcome them thank you okay thank you I think our panelists have touched upon many things availability lack of data quality of the actual data linking and measuring linking them to measuring to measuring outcomes how to increase demand how to make the data relevant for policymakers with aggregation so many things so we can open up now we have about 20 minutes for questions and comments so if you can just say who you are because we have remote users so use the microphone and comments questions general yes yes look sorry small intervention I know there are people in this room with tremendous expertise actually in this area not only the authors here I just wonder just to really encourage comments and your own perspective please okay I mean thanks a lot for some extremely insightful observations maybe I can try and provoke just a little bit I'll be sort of a little bit dented it doesn't mean that I don't appreciate a lot of the progress that has been made but when I as an economist try to look a lot of the data or a lot of the surveys and a lot of the dormant information that's there I can't use it take Ginea Bissau as an example I have a very good friend medical doctor who has collected information for a long time and he said there's this fantastic database but there was no information of economic social and so on variables in that fantastic dataset so when he came and said to me can we collaborate I said well I would be I would be happy to but what it would take is that I would have to start from scratch because I need in order to generate answers to a lot of the kind of question we're discussing I need to have the economic social and other variables in the datasets there are of course some datasets that do have that but I would like to submit that a lot of the information that's out there doesn't have it so maybe one of the challenges that I can see is how to somehow get some communication around that and if you wish maybe clean up a bit and try to ask the question up front what is actually required in order for the various surveys to respond to the actual questions that you want answers to because it seems to me that that's not happening right now thank you thank you I wanted to just follow up on a couple of issues and Gaurav raised this Margaret when he challenged you a little bit about the observation method which seems to be the dominant method that was used in the surveys I think what we heard from the Liverpool group was how few actual indicators are available for use at the front line that relate to quality and I want to go back to Andrew's question which is what's the practical value of all of this measurement I think it's enormously important that you have exposed through all of these papers the state of quality but dynamic improvement is going to require real-time data and there were none of the papers the closest that it came was the paper from Ethiopia but even there it was a survey that was a team went in every few months to spend three days on site that's just not real life for the front line providers who are going to be expected to be at the front line of improving care so I think there's a huge gap that is revealed by the lack of good tools to measure quality at the front line and I just be interested to know the thoughts of the panel on how we're going to solve that problem because if you combine the lack of available indicators that reflect real issues around quality the complete absence of any measurement of patient experience we have just a giant way to go in trying to bring real-time measurement to bear which is what's going to be required for real-time improvement I mean anybody can comment on it but I thought that it might create some reaction and that was what I was hoping for the different measures of the front line and the timing etc so we we faced a similar problem when we were designing our impact evaluation in Kenya and when the government wanted only to measure this checklist and we were like but what if those are not the set ones that matter right so now we are collecting data with covering for cover standardized patients with direct observation with this one we're trying to have like a wider set of tools to measure with the idea that later we can also say okay where are the ones that were highest color related with the ones that are closest to a patient we are measuring also patient experience but with the idea that is a big story but try to understand where the key things so that we can contribute somewhat to understand where the key tools that the government could measure with more frequency lower cost etc but it's a big question we have very little data on that that connects as it's comprehensive and representative so the idea is that is the whole census of three different states so hopefully we can contribute somewhat to that later I think we do not need surveys for example to assess whether a health facility has all the things which is needs to provide quality of care so we can ask the nurses to report this once a month and in most countries actually they do it only that nobody notice this it's just lying lying there and then I also feel one can maybe look at innovative ways also we can we can have nurse midwife registers and they say I did that many deliveries and I got this many in-service training and I have done this and this and this is my key challenge so that they are maybe re-registered every year so we can think of a little bit something out of out of the normal box of doing surveys which might then be much more sustainable quickly I'm on since maybe is data linkage a way for like this economic you know having economic data and wanted database and if it hasn't been structured from the beginning haven't thought about that will also create problems so I'm just a prompt so Andy I think economic data and the absolutely critical need to understand what quality costs what facility you're spending how are in resources invested actually generating or not generating quality care I think there has been much more work for example and and even that hasn't been sufficient in understanding whether resources are you know correlated with volumes in terms of productivity assessment I think what we are saying this needs to move to is not productivity in terms of patient scene but in terms of patients treated well and patients getting better and those the linkage is absolutely critical because countries we're talking about in this room primarily are have incredibly constrained budgets they can't do everything that we're measuring and would love to do so that so this is an absolutely critical linkage to be very concrete for example in some of the service provision assessment surveys or Sarah surveys they're basic basic economic factors completely missing so actually half our surveys don't reveal whether the facility is urban or rural that kind of matters you know and another one is that we don't know anything about the wealth of the neighborhood or the or the the context in which it's in we do that by doing data linkage with combining data that were themselves highly imputed to try to figure out whether the facility is operating in a resource rich or a very resource poor neighborhood these these things obviously matter these are the very things Bernadette and others that are above the level of the clinic that actually make a massive difference to what's going on there none of that's captured right now and I don't all have to be in one survey but if it isn't then there has to be a clear way to bring those variables in I think on the second point about peer your point about what are these surveys contributing given the need for real time data I don't think it's interesting right you're getting lamppost bias here a little bit the data that we have are what we have and what we obviously what was featured in this in this issue and that has to this point been coming and the best quality data have been coming from nationally represented these nationally represented facility surveys is that the only source of data that's relevant absolutely not what we want to start with with all data actually is asking what's it for who is it for and I've already highlighted and I think others nodded vigorously that actually most of these shares are not even used in country so what's the point I do actually think that the measurement the minimum measurement package around quality has to include facility surveys much leaner than the current version as a way of tracking quality across the country from ministry of health to know where's it moving to see whether people are are getting the care they should be getting including some mix of these tools some of them can be simplified and made more efficient including using cell phones to reach patients getting patient voice in but the piece you're emphasizing is what's HMIS doing for us what are other methods doing for us and I can tell you just look at this quality issue they're under delivering in terms of comparable robust data right now and that has to be an urgent point of action can I just add something so one thing that I just wanted to highlight and Dr. Matthews is also here I know that I know that like the ENAP group is doing a very big validation study on indicators around the time of birth including for sick babies maybe you want to say something I think 12,000 12,000 births across multiple okay okay but I think the their results will also be very useful to you know have the which in the let us know which indicators are the valid ones which can be integrated into the HMIS system and so on so okay I'm Teresa Diaz I'm I took over for Matthews position coordinator for modern evaluation epidemiology in the MCA department there's a lot going on with routine systems and and specifically with DHIS too getting not only interoperability but apps that that provide printouts and analyses and so forth but there's still this problem with aggregate data and I think and I'm really always say this I think we have to have a strong voice about individual patient records I think that you can't provide quality of care without individual patient records as a clinician myself if I don't know what happened the last visit how can I provide quality of care the next visit and being able to have those individual patient records in a routine system you would have a lot more information than you have now and being able to have these more routine kind of reports these more electronic kind of reports for the you know the clinician for the district not making them do the kind of analysis I think would be very helpful thank you I'm Samira from the department of maternal newborn child health mine is a comment around the challenge I think this came up as I saw ex-minister of health from Ethiopia walking in the key challenge is really getting the commitment and leadership from the policy maker at the country level if the policy maker makes it his priority then I think measurement of quality would be a standard procedure for any programming so perhaps the next session will highlight the issues around it how do we make that ownership and commitment happen at the country level then if we have that then all the tools and whatever comes with it will make sense because data is for policy making and decision making thank you thank you that's for the next session okay go good morning Ramash Krishnamurti I work at the literature HQ here I just one observation for the last 10 years of visiting many many health facilities for doing the HMIS work that Teresa was mentioning as well as structured data what we have noticed is that there is a serious lack of structured data at the level of care at the at the inpatient registries the inpatient registries are highly unstructured and sometimes completely incomplete if you will and whereas the outpatient data is more of a quality issue than inpatient data inpatient data at least contains many of the transactions are happening at the bedside but in so far as aggregation is concerned it is not well done and outpatient data quality is a very very serious issues in many countries and in those countries where we have found where outpatient data the throughput determines the quality so just to give you a case in point we visited several facilities where the throughput is 3000 patients per day and the quality is extremely low because the doctors have no time to enter the outpatient records so we have to kind of really understand what is going on in the in the health workforce space that I think there's a comment made and also the structured data space as to what is minimum that that doctors need to collect what is minimum not what's maximum because they need to care for the patient more so than recording the event so how do we do that this transaction cost allocation model in in which where minimum of the minimum amount of data is is being structured so if countries get that kind of a guidance I think it would become a very important game changer thanks Rich okay now yes my name is Honas Gonzet and my advisor in Quality in WHO America in Washington and this is just a comment building on comments by Dr. Tarpe because I think that the final objective is improving so having data to improve something and the comment is that quality of care and we are saying care is what finally happens to the patient and there is there is such a big level of heterogeneity of situations that I will think that somehow Dr. that was opening the space to think about quality in health services and systems that we have to address economic components but also stewardship governance components that really affect and will be probably the components that might make a change and then we can use so it's only opening that space I was really uh impact when I read the the name of the bulletin quality of care but in order to improve quality of care we need so many things at the health systems and health level health services level so that's only that thank you this fall as well on the last comment my name is Rima Jolove I'm from the maternal health task force at the Harvard School Public Health and also helped to coordinate the ending preventable maternal mortality working group and we've just completed a long process to develop a set of indicators for as Margaret has mentioned and others have talked about those things that are above the facility level but that are really conditions that are necessary for quality of care and we went through a process to map identify prioritize and evaluate for quality the available indicators for things like adequate resource allocation and health system strength and even would it be helpful to have a standard set of disaggregators for those social and economic uh uh contextual things for all indicators for which that makes sense so um I think the question is how to make that work usable and diffuse down to the point of service um because I think those things are enablers for for quality of care hello hi everybody uh my name is Timitai Erbobu I'm director of advocacy for MSD for mothers which is a pharmaceutical company and it's our commitment towards maternal health improvement maternal health um my question is really it seems that basically there's a lot of practical things that needs to be done here to what extent have we actually thought about the role of private sector in terms of doing some of these gaps and so like for example you mentioned um uh challenges around collecting data um uh use mobile technology within facilities having management information systems for healthcare workers and these are things that basically the private sector um do and have tools and and expertise in um just wondering what to what extent has that been considered cheers yeah excellent observation is this something somebody wants to address now but we can also raise it during the second panel at the country implementation and the role of private sector working in the countries unless anybody would like to okay well make a note and we'll be sure to bring it up in the next session okay thank you very much and I don't because I would like to keep the schedule we're moving on to the next panel want to make sure all our names are up okay so this is the session you've been waiting for um the evidence is well and good um and data are exciting to some of us but really what I think is very clear to everyone is that evidence is just one input into the more important process which is the one of creating change and creating improvement for real people in real countries every day and so I couldn't be happier to have with me some of the leaders from four countries actually of quality and health systems to reflect on the challenges that they're facing in both improving quality for people but actually interacting with these sorts of data with the evidence that's out there the problems the the successes as well so I do think this is going to be frankly the most stimulating and exciting session the way we're going to do this is I'm going to ask introduce them and then ask each of them a question one by one and then really open it up to all of you to ask them questions okay so first of all with us we have today quality directorate leads from Liberia Indonesia and Mexico along with the former minister of health from Ethiopia we'll we'll introduce them one by one in a moment and I understand that our colleague from Liberia raised here from the airport so he deserves extra credit for the huge effort he made to be with us today so they've traveled from long and far and I think we're all going to be very eager to hear what they have to say so actually if you don't mind perhaps I could I could start with you that's okay great okay so this is Dr. Philip Bema who is the health care health care quality management unit head the ministry of health of Liberia we all know Liberia it's an incredibly famous place now for those who didn't know it a few years ago everyone knows it now and that's because you faced one of the most horrific epidemics in recent memory and that was the Ebola crisis of course in the last couple of years it's really I think highlighted many many things for many of us and it's going to be studied for years to come it's already resulted in a great deal of introspection both in Liberia but also globally looking at the global response but many people are claiming that the health system is really the first line of defense against outbreaks and yet we see a proliferation of people thinking about pandemic preparedness on the one hand and this emerging notion of resilient health systems health systems that can bend adapt deal when something difficult comes along a shock comes along here in this meeting though we've been talking about quality of care as one key component of health care performance can you help us think about the links between resilience health system resilience on the one hand and high quality health systems on the other hand how do those things connect thank you thank you very much that's a very important question and I'm so happy to share the Liberia's experience in the room no doubt the Ebola outbreak in West Africa exposed a lot of things and so we'll talk about resilient health system and quality to be honest quality is the very backbone of resilient health system now what do I say so before the Ebola crisis in 2014 Liberia has been emerging from a kind of 14 years devastating civil crisis in 2006 we had a general elections and we're trying to rebuild the health system so before 2014 actually there was a health system in place but the quality of the health system was never measured was never tested and so Ebola kind of exposed that in a very big way what we saw during the Ebola outbreak was that the system was was completely dysfunctional this there was a system there but the system could now withstand the shock that came as a result of Ebola and that kind of helped us to see that there's a reason to concentrate on quality because if you see the the cost associated with responding to the Ebola crisis and you see what we should use to put into strengthening quality across the spectrum in low-income countries you'll see that it's just huge and I like the way the discussion has been unfolding this morning kind of looking at the economic part of quality but we just look at what Ebola dating West Africa and quality you see that sometimes if we don't really invest in quality then we can pay a bigger price so when the Ebola crisis hit we kind of ask ourselves do we have a resilient health system in place those critical questions kind of help us to see that really we didn't have that in place because when we took stock of ourselves to see what we had in place we noticed that there was some key critical gaps associated with quality for example infrastructure we cannot talk about quality and when there's no health infrastructure in place the SARA report that I just talked about the SARA was performing Liberia majority of the health care facilities do not have adequate water system in place so we can talk about quality if you don't have water system in place you don't have waste management system in place you don't have infection prevention control system in place so health infrastructure was completely poor the other area that kind of highlighted the gap was the issue of self-heal more resource in Liberia because it's a low-income country we don't have many doctors and the doctors that we have in the countries tend to be skewed more towards the urban areas because in the rural areas you don't have a lot of facilities you don't have internet connections you don't have telephone coverage you don't have a scratch card you don't have many other things so majority of the health care workers tend to be skewed towards the urban area so the rural area are left completely vulnerable the other thing that the volacrassist highlighted was the issue of supply chain so in life to be honest we cannot talk about quality if we don't have adequate drugs or the appropriate drugs to treat infections and malaria so in Liberia there's a more like a push system rather than a system that is working so that kind of highlighted a critical gap the other issue that I have was the issue of health systems strengthening what do I mean the whole issue of quality culture is done in Liberia quality tends to be abstract so when you talk to policy makers you talk to decision makers about quality they are like what are you talking about here we talk about quantity first before quality because a lot of people do not have access to health services then you're talking about quality so the whole issue of quality tends to be abstract in their mind so prioritizing quality is a major issue so just a kind of reflection from the Ebola outbreak the Ministry of Health decided to do a deliberate effort of setting up a quality management unit now local quality across the spectrum thank you very much for those comments and it was very heartening to hear you say that quality is really the heart of a resilient health system I certainly think that's how people think of it as well in terms of trust for example and confidence thank you very much to my right is Dr. Sebastian García Ciso who is the who is in the quality of health care and education so he doesn't just have one sector to worry about but really two I don't know how he does it in the Ministry of Health of Mexico and obviously Mexico is such an interesting place for so many reasons both because epidemiologically you're dealing with a range of challenges still some maternal child health challenges in parts of the country and yet throughout the country also an explosion in non-communicable disease particularly diabetes that was featured in in the special issue which I think was declared a national emergency I believe very recently and so I think and also by the way I should say Mexico is a leader in health system measurement and in quality measurement and you know when we were talking over at the coffee break the first thing I asked him is you got to tell us what not to do on measurement before we all make the same same efforts and invest the same resources that don't always pay off so I think that's an important insight but the question I have for you right now is we care about diabetes we want those people to do well similarly for hypertension but as a health system person as a health system leader how do you combine the sort of disease focus the focus on these population groups with an overall performance of of the health system in terms of its its quality of care thanks thanks market well to begin with yes it's a complicated agenda to have two different areas quality of healthcare that has prospective quality and also regulatory quality of healthcare and then education which is has to do with training every single health professional in the country and linking universities with provision of services so complicated agenda but I'll try my best to actually answer your also complicated question basically the Mexico has been moving towards universal health coverage since 17 years now at the end of the 90s we did a a national survey that showed that 50 percent of our population which was back then roughly 50 million had access to social security and healthcare through social security in a a pure bismarckian model of social security but then 50 percent of population didn't have access to anything so we decided to launch back in 2003 a comprehensive insurance scheme based on general taxation called seuro popular and since then we've we've sort of moved towards having this universal health coverage perspective and that's when it gets complicated when when it's not a matter of financing but a matter of how you transform your your system to provide universal coverage and not just financing everyone within a country to access healthcare and so this has been the latest struggle on on on a point in which we now have roughly 100 percent of our population covered in terms of financial resources to for healthcare but then we don't necessarily have the results that we expected from that so we've increased about five times the available budget since 2002 this time but then we have improved five times in terms of results we we've developed a series of strategies since the launch of seuro popular to measure and monitor quality of healthcare one of those is a national system for for quality so a set of indicators called indicas if you google indicas in capitals you might you might get through it and and in there we have over 12,000 medical units facilities reporting every quarter master in terms of prems so what what patients say of the care they're doing and then and then also some medical auditing in terms of what the health records are actually expressing in terms of whether they're fulfilling the things that they should be doing and there's a big challenge with this because one thing is to have a system and then having units reporting and then the other is for them to understand that the system is for them for improvement at the local level so we got to a point roughly five years ago in which everything was green everyone was very happy everything was being done according to the plans everything was working and then in terms of quality if you have everything green then you have a problem because then there is this lack of recognition of the need to improve and so we decided to change the system so we've been working together with OECD IHI to basically transform this and link some more objective or hard measured data in terms of for example clinical results and outpatient services and in-hospital results from care to these to these soft self-reported instruments and then the results are amazing suddenly you start realizing that you do have problems that there is a lot of room to improve and units are starting to realize this again some years after they reported everything was okay so every quarter semester we also survey about million people on what they refer as experience with care and this this gives us a lot of information because you see for example in places where you have less educated population particularly indigenous population then expectations are really low and we understand that so they're very happy with any care they receive and then these changes as as you go into urban or more develop areas of the country so we have to deal with how do you design specific elements to measure quality of care within this particularly heterogeneous context so now focusing on your question and how and how we are we are targeting a particular condition so diabetes is a very complex problem in Mexico it has to do with lifestyles and our genetic predisposition to diabetes and it has to do also with with the lack of access that we refer to in the first place and access has to do also timely access to healthcare and the other one the lack of results from people actually acts in this system so if you have 13 million people diagnosed with diabetes or estimated diagnosed with diabetes as we have then you have a massive problem if you're not actually dealing with the disease and then that reflects also in the in the very large incidence of chronic kidney disease for example and the cost of this represents of course so what we're trying to do is focusing on seven priorities so we have a national quality strategy that includes all the indicators for everything includes many of the general actions in terms of patient safety improvement mechanisms the overall quality management program but we also target seven particular conditions that are priority in our country so diabetes and metabolic syndrome is one of them of course acute myocardial infraction in which we have three times more mortality in hospital mortality than in the average in OECD countries so it's a big problem and also linked to our epidemiological transition breast cancer cervical cancer acute lymphoblastic leukemia in pediatric population and mental health with emphasis in in depression so this prioritization what allows us to do is to actually have a stronger approach and more tools to actually bring down this national strategy very comprehensive and probably idealistic strategy into very specific things that need to be done at the local setting by administrators by planners by clinicians by all the different actors involved in quality of care and in diabetes it links different things in terms of one following procedure so making sure that everyone knows what they have to do to actually giving them tools to train on these procedures and knowing what to do so we we've created massive online open courses for staff in general different conditions we've created algorithms for this and they're online and available anytime for everyone and we've also developed specific tools for planners and administrators to make sure that they understand that if a certain drug is not available every time at a clinic this might have a problem for the quality of care for that particular patient not receiving this medication at that point so all this prioritization what it allows us to do as well is to generate something that we've called externalities at the facility center so basically even though it's linked for diabetes or acute lymphoblastic leukaemia or acute myocardial infraction by targeting a specific facility by specific topic and training personnel there you might change the whole the whole spectrum of conditions seeing there so for example in hospitals for acute myocardial infraction if a patient comes with an acute abdomen it'll benefit as well because people have been trained in triage people have been trained in how to access acute a clinical career immediately and allows us to basically show results in a better way which is also a good thing in the system i don't know if i've actually answered at all your question but i'm i'm trying to say too many things and i'm too gentle i'd like to actually realize what i've said yes will that be okay okay so i'm picking up a i think a few important themes that are resonating with some of the conversation of earlier today what i do find it fascinating that you're in charge of both the health care quality and the quality of training of health professionals which you know is directly related to this to this issue of how good are the providers when they get to clinic actually and are they prepared to deal with the disease burden that they will be seeing every day so it's interesting to see in mexico the combination of those roles in inside the ministry number one number two i'm fascinated by this idea of a general sort of quality infrastructure infrastructure for both data gathering and tracking but also for improvement that you were describing you know that data that are available to facilities for example to see on a regular quarterly basis perhaps and also the the i think frankness with which you said it doesn't always work the first time that the indicators you know are all green that's meaningless and and linking that also to the low demand for care that we've already discussed amongst some populations and so therefore the need to improve so i really like the dynamism element that you brought out and i think the the other thing you're pointing to is look you've got this data and improvement infrastructure but sometimes you need a turbo boost for those conditions that are really killing people that are really causing suffering the high burden conditions in your case you have a list including diabetes and then what do we do for them specifically to really dramatically improve not to wait another 10 or 15 years but to dramatically change care today so i think that's a really interesting framing right having a strong base and then having intensive efforts perhaps we're needed so thank you all right i want to exactly that's what you exactly he's very near me so i can even hear his thoughts okay so i want to next introduce dr ecca viora from the director general of health services the ministry of health of indonesia she's traveled very far again we're delighted to have her with us today you know that um in the issue we have had many articles on particularly maternal and child health care but also an attempt to make linkages between those services and a broader universal health service package universal health care package we heard about this in mexico as well and indonesia is a very big and very complicated country of thousands of islands thousands of municipalities real geographic barriers so many challenges and many friends i wonder if you could comment as you're pushing towards universal health courage as you're pushing toward helping establish a minimum package i think often we talk about minimum packages we talk about benefit packages but i wonder what you might say about what minimum quality of care are people also going to be able to demand or expect when they when when they move towards universal health coverage and how easy difficult what are the challenges in making that consistent that quality of care consistent across a highly diverse and heterogeneous country like indonesia thank you chair thank you for this this opportunity i said to the our perspective for the quality of care related to maternal newborn and child health maternal mortality ratio in indonesia is still high it is not achieved as mdg's target and now is brought to the sdg sdg's target as you mentioned indonesia is the world's largest archipelago we have more than nearly 70,000 islands across the equator with over almost 250 million population and multi-ethnic population and also it is the union republic with decentralizing administration and consists of 34 provinces and 514 district and cities yeah hell is a state is integral part of the national development to reach healthy indonesia through three strategy in the current five-year plan 2015 2019 the strategies cover national health insurance to reach universal health coverage in 2019 and the second infrastructure improvement and the third health paradigm expansion expansion through healthy family-based approach challenges are faced in the element of health system that includes of issue availability and readiness readiness of quality of healthcare infrastructure particularly in the remote area for the quality improvement of maternal and child health in my country quality improvement in maternal child health care start with the firstly we are development the national guideline on integrated antenatal care as the service standard it is consists of 10 steps discover waiting blood pressure check nutritional status examination of uterine fundus fatal heartbeat tetanus toxoid injection ferrosurfat tablet laboratory test case management and counseling and secondly came to this dissemination of the national guideline to sub national level for to the provincial and district health office through technical consultation meeting training of trainer and various technical assistance and the thirdly assessment of quality implementation which is done through field supervision monitoring and evaluation sub national level then send periodically report of implementation status combined with assessment on the progress of maternal and child health quality improvement issue and proposal solution and the fourthly the national level respond to the assessment report that includes adjustment and input for refinement of strategy related to the quality improvement however the result of the quality improvement performance various among areas and among province and district this is due to the variation of decentralized session stages capacity of health worker and constant in fulfilling health facility standard particularly in remote area including in adequate budget support from the local government as regard to the mcs quality improvement experience we learn that skill attendance of meaningless compared to the compliant to follow basic procedure and performing life-saving skill and second measurement of quality of care is expensive it has to be linked to integrated training and accreditation approach for the hospital and for the primary health care professional society need to be the guardian quality they have to be engaged properly all this issue should be solved interdisciplinary intersectoral and interstate holder collaboration including public private partnership and engagement of the local community the national committee on quality improvement of health care as part of the health system should be the priority step to plan comprehensive action of quality improvement in the country currently the national program priority 2015-2019 for health services this development aims to increase access and quality through health care availability and readiness particularly in the remote area as happened in the mc at example evidence are required to move forward with quality improvement action this includes various researchers and students to provide profile of quality status and review of the current progress as input for quality adjustment we also need lesson learn and the best practices from more advanced country with many experience in quality management of health care services as part of the health system thank you very much dr viewer i think your your comments really highlight the essential nature of having a wide set of constituents behind this agenda that it isn't just for the ministry of health to say this is important but you you mentioned professional associations patient groups facility managers and others and i think that's a point that perhaps hasn't been highlighted quite as quite as much how many actors need to be in the mix it takes a community a country really to to make this happen so thank you for emphasizing that and the complexity of making those communities and work together i want to move to dr cassetti birhan admasu who is the former minister of health of ethiopia between 2012 and 2016 and as a former health minister of a very large and also very complex country with diverse populations and many health challenges i'm sure he has many things he could share with us today but i would love to hear about your lessons learned dr cassetti around policy makers and what is it that they need to do to act what also stops them from action sometimes maybe they have the will but it's difficult to act can you explain to us what motivates action what stops action and then on the other side side what can researchers in the academic community think tanks and the broader community do to make the research and the data more relevant to policy makers what don't you know that you wish researchers will give you answers to for example and how should they be interacting to with with ministries of health yeah thank you very much first of all i would like to thank organizers for inviting me although this has like little to do with the rollback malaria partnership and advocacy but i am very glad that i have taken part in this forum very simple questions to a very easy problem well i mean i i can't tell you from my experience and from my interactions with my former colleagues that call quality is an important agenda for leaders so i you find no minister who doesn't care about quality um in africa or elsewhere the reason is simple because we have a lot of pressure when you go to parliament when you open the radio or tv or have interaction with people you always hear not compliments even a very effective service delivery platform uh you hear always complaints on the quality and so on so i would say there is a general consensus on its on the need to improve quality so there is no need to go out and convince that quality is an important agenda everybody understands quality is an important agenda but the problem starts when you start to think how you would like to influence policy as uh some of you might have noticed in my observation of a health system and a medical practice elsewhere including my own country you know when something is established as a culture it's always difficult to change it whether it is right or not whether it is an effective way of doing business or not our system is always difficult to change so i think it will be critical and this this has always been my point of view and that's what we have started you know to do in Ethiopia we need revolution to bring quality uh maybe we like in Ethiopia the word revolution but what does revolution mean you know it's a fundamental change in the way you think in the way you deliver and in the way your system is organized so without this kind of fundamental change it will be very difficult to change the conversation about quality so changing the culture attitude behavior of the health providers is going to be critical and i also agree that we really need to invest in communities so so that you know while you put all the initiative and systems and practices to change the way health providers act and behave and do in terms of meeting the service standards we have but at the same time you really need to empower communities to demand quality service you know in Ethiopia for instance with our famous health extension program we have been investing to generate demand but we have not used that same platform to empower the communities to demand better service to demand quality service at the facility level so i think you know a revolution happens when you put pressure from bottom you know the bottom up as well as top down that's one the second you know for a successful rollout of quality initiatives we you need to build an implementation capacity meaning you need to have the infrastructure the quality infrastructure in place i mean in the developed economies and health systems you have quality management officers looking at the data and you know there's a care provided to individual cares in many of our economies in the developing world we are moving toward this you know some sort of health insurance or financial protection system so it is about time to build that infrastructure where you have people in each facility in each you know level of management structure people who really need to think about quality so you know if you come to my office and present to me you know how beautiful and successful a small quality improvement project is the question i would ask is has it really changed the behavior at the facility or is it by providing unsustainable financial incentives that you manage to deliver those results so without really creating that you know nucleus of you know systems that can really bring transformational change or revolution within a health facility it will be very difficult to to bring impact so it's not about the data it's not about you know the reporting systems insurance it's about individual providers meeting the service standards everywhere you go there are service standards formulated based on WHO or other you know normative agencies so the challenge is meetings or the standards so you need that implementation capacity that infrastructure that nucleus at each level of the system that can continuously look at and provide you know the inputs to bring transformational change at at each level of the system you need you know the human resource i totally agree with with our colleague from Mexico that you know the providers need to be skilled if they are not skilled if you know the quality will be compromised so we need to think about the skill of the the health workers as well so that kind of implementation capacity has to also be integral part of this this system supply chain is a big problem so you may have skilled workers you may have the the right kind of systems and processes in place but if the products are not there you know jsi has this motor if no product no service no program so i think it's really important to look at that as well and try to address those bottlenecks and finally you need the structure the structure of looking at you know these quality issues both vertically as well as diagonally across all programs and across all service delivery platforms and that's how i i i believe quite the quality agenda could be could be addressed by having a holistic transformational and revolutionary thinking without that small little improvements wouldn't really take us far and that's how the conversation the conversation should be formulated as well thank you okay well thank you for that um you spoke about many things and many levels of the system from the community on all the way to the minister's office actually um i think again one thing that i'm hearing is that and it's interesting it's a bit of attention right you're speaking uh a lot about culture standards almost the air that we breathe in these clinics and the the way the expectations that should be being fulfilled and these are things far above a specific indicator that might be collected about what was done you're also speaking about sustainability i think culture changing culture is the only way to sustain something meaningfully was the implication and i think that's an interesting and productive tension actually because the indicators are serving one set of purposes but what i'm hearing you say is we don't work to the indicators we work to a greater purpose a changed culture a changed level of performance not just and not just an incremental shift and so i think that's an interesting issue for all of us those of us focusing more on the measurement side how do we get at the measure some of those deeper deeper elements that that you're asking us to to talk about so thank you for all of those wonderful presentations um what i would like to do now is open it up to two questions um i know there's already one question from the audience about the role of the of the private sector in all of these discussions and in your health systems and what can they do to to promote quality how do we ensure and regulate them also to and to work together so i but that's one question let me take two other we'll take three questions at a time please yes in the back and please introduce yourself hello my name is um ania smith i'm from still embossed university in south africa and we've just completed a series of standardized patient studies on tv hypertension and contraception and something that really struck me while we were designing the instruments was the big gap between the guidelines that are issued in terms of quality and the underground realities because i think going through all the the guidelines um for a contraception consultation would take 40 to 45 minutes the reality is the nurse has five to ten minutes so i think my question to the panelists is is there a big gap between international but also country-level thinking around the policies and guidelines that determine quality and the on the ground realities and do we need to become much more focused in thinking about quality and what quality means thank you thank you other hands yes please thank you very much um isabel vexmuth from service delivery and safety department sorry your would you mind just repeating you're cutting out isabel vexmuth from service delivery and safety department uh i have question for mexico in fact um i am a little bit astonished because you have not mentioned you know social determinants of health you know and how what are the actions of mexico you know to tackle the root cause of the problem specifically of diabetes you know and i have not seen as well multi-sectorial you know approach in the way you have described you know how you address quality and uh i think it's uh it's very disease uh disease oriented you know approach what about health promotion what about salutogenesis you know approach uh specifically for example in the in the area of patient safety not do arm we discuss as well to change this model not disease oriented approach but you know salutogenesis model no what the country do in the reality about that you know and i am interested to know more about mexico and what they do thank you all right thank you we'll just we'll we'll get some answers and we'll come back to you anthony you're on the list no not now okay um all right so the three questions were there is a question about the role of private sector and how we work with a private sector that's one that's for anyone who would like to speak um um the the question around guidelines and the reality testing of guidelines do they really can they can be they even be done is i think what you're asking in real settings and i think the last question had a few different elements but one of the things i think is thinking about the the the disease versus a more patient-centered or preventive model but also the multi-sectoral piece i heard you really asking about ministry of health is one thing but what about all the other sectors that are implicated in the generation of disease so maybe since uh we'll start with that one since it's very specific yes thanks um well i have five minutes to say about pretty much everything i could so yes of course i didn't go into detail in terms of diabetes there is there is a national strategy and it has to do with with not just not even just health it has to do with education it has to do with social development it has to do with many different aspects and social determinants of overweight obesity and diabetes so it's not just diabetes and and quality measurement and quality policy has to do with all these different aspects that are happening in the country so it has to do with work with educational authorities at the local level so it's schools and particularly primary schools so just to go into detail with this we're we're just launching a national program to measure every single child in between first grade and sixth grade and working with them to do activity related programs within schools and see whether we have our results within one year and repeating this every year so this this includes roughly about um i think it's it's something like 10 million children in the country so the entire population that it's within schools so all this all this is online so i can give you much more details on this uh and and how we're actually doing all this tackle this now in terms of of the whether to go general or disease specific i think it's a combination and and i think i said it maybe maybe not as clear but we do have a national quality strategy which is not linked to any particular condition for example patient safety one of our basic concerns and it is not oriented to any specific condition now the problem that we have and and i'm very glad to be here at WHO is how to actually link general priorities with vertical programs that address very specific particular diseases so when you have that context and you apply that into local context which could be a country or it could be a local facility within a community it becomes very complicated because then you rely on the same provider of services to do everything so they have to report to the tb program they have to report to the chronic disease program they have to receive the vector-born disease program and they have to do all the quality of care when when if you think about this qualities at the center of all this you cannot improve in one condition if you don't do the overall picture of this so this this complicated context that comes all the way from WHO and of course is replicated throughout all our national and local resource allocation programs it's trying to link now to to quality of health care so for example if you talk about diabetes and if you're trying to prioritize diabetes within the chronic disease spectrum then quality of health care has to be there and the same with these other seven seven priorities that were established based on our burden of disease and now the other consideration of this is if we prioritize once you have the overall quality standard and overall quality policy if you prioritize this enables you to show results to the local community and empower them to actually follow and continue doing these things and as i was mentioning and probably i wasn't very clear these also have externalities or indirect effects within the facility so if you improve mercurial infection attention care within hospital it will also improve indirectly the quality of care received by a patient come with pancreatitis or an acute abdomen syndrome why because people have been training on on triage mechanisms because they've been trained on how to do a fast diagnostic because resources are available for the emergency services because administrators are sensible on what to do at every specific moment so that's what we're trying to do show results on very of these seven priorities and then trying to make this basically sum up to our national quality strategy thank you i'm just aware of the time in fact that it is very short and a number of you are heading off to other meetings in just about 13 minutes or so um so i actually wonder dr. cassel you did have an outstanding question would you add it to the mix and then we'll let the rest of the panelists comment on the remaining questions please mine's a very generic question about since you're all in power of power relations in relation to quality uh perhaps starting with the complexity of international donors and agencies who come in with tyrannical experts and lots of money and fragmentation to try and drive you in particular directions or the power or not of professional associations like whether nurses and midwives have a voice how you can actually empower some of your frontline workers and finally a really difficult one which is we're trying to encourage people to question everything and change everything at the front line but you're working within traditional hierarchies where you're told what to do yeah and that is a kind of fundamental cultural shift and how do you bring that about okay small question yeah no i thank you for that that's a very specific comment straightforward so these are the three items remaining for all of you or anyone who would like to comment again the guideline reality the the private sector role and also the tyranny of the of experts yeah okay so maybe i will speak a little bit about the guidelines and the and reality on the ground i mean you are right even in Liberia we we have similar problem like that the the the problem with that is that it's it's top down guidelines are created at the top and then they are given to the people and say look this is what you need to do so um learning from all our experiences but we're dating like brothers to take the all our approach at least a little bit pattern off so sitting with the um those who will be affected by the guidelines develop the guidelines together we think one that is critical is because they create ownership so when there's ownership even though there's the reality between the guideline and what is happening on the ground is very difficult but there's when there's ownership people identify with the document and know that they were part of crafting the document they are more inclined to implement the document rather than you know it being top down yeah i i would also start from the guideline i think i i would like to underline the need again to have an institutional approach to solving such problems you know our experience in Ethiopia has been you know if you go to a primary hospital or a district hospital or even a bigger hospital there are days where you know the the corridor the facilities overcrowded usually these are market days where people especially in rural areas come you know uh to the towns they they drop by the the health facility to seek you know care or access services in other times you see the health facilities are not busy so it is as much as planning and deploying your resources in an efficient and effective manner to deal with that so it's not that the guideline is not realistic but it is how the resources are deployed you know there could be instances where guidelines may not be realistic but if you have the infrastructure at the ground to look at what are the problems you know that the health service delivery is facing in each of the facility then they should be able to come up with you know innovations to address that problem so it's not only a question of having only few nurses or physicians delivering services but it is how you use you know the the services and the available resources the power dynamics is the question i really like because you know i think the the beginning the the initial step and the most important thing for countries to do which you know my country has done over the last 15 years is you know potentially knowing what you want knowing the problem you have the problem we have in many places is you may have a national plan but that national plan is probably developed by a consultant that is parachuting into the country without good understanding of the situation on the ground so it all starts with planning and planning means you know defining your problem and defining how you really want to deal with that and if you own that plan and if you really own what needs to be done there is no way you know money could influence your thinking because you can influence the thinking of the people with the money and that's what it has successfully done over the last 15 years you know the the famous health extension program i go back to that because you know it it is one example how when it was a starter many donors who now claim that you know it is their baby we're against it at the beginning but when you really define that this is a way the country needs to go and if you put whatever resources you have then you know donors would follow so i think that's that's the basic you know problem that we have to address in the developing world you know making sure that countries define the problem and what they want to achieve the the empowering professional associations and councils is also an uninteresting approach but you have to ensure that a proper balance is is maintained when you do that what i mean is you know again in including my own country in the beginning many professional associations where against task shifting some core responsibilities so we have to pass a law because we can't i mean that's what i advise many of my former colleagues if the professional associations becomes a stumbling block you are the government pass a law go to the parliament and make sure that you maintain that balance because sometimes you are dealing with a professional interest that is you know really eager to protect its own territory without really looking at the public health goods of you know task shifting some responsibilities and that i believe is the right approach for a government to follow but at the same time you have to also make sure that they are part of the whatever initiative we do in terms of defining the services standards in terms of you know being part of the regulatory services and so on especially in resource limited settings where physicians in no way can go to a remote place when they are against training emergency surgical officers to do the basic you know emergency surgery that's that's not empowering you know professional associations that is actually impeding you know public health service delivery so i think that's right balance has to be maintained and you know the hierarchical tradition we have in the medical system is what i i mean we need revolution to change that you know when i asked you know i was a CEO of one hospital i'm sorry if i am pressing too much you know i asked my psychiatrist at the time to work eight hours a day they have never done that you know there are only 10 psychiatrists for the entire country and you know when i assume that office the outpatient waiting time was six weeks you know imagine a patient with severe depression may commit suicide in that in that period of time the inpatient admission waiting time was more than six months so it's a system that was broken so there was a clear need for the psychiatrists to work long hours eight hours at least you know and you know putting in place triaging and so on and so forth was was really important but me coming as a CEO telling the psychiatrists who were the gods of that hospital was unthinkable at the time but when you involve everyone and start that revolution and when you have someone who backs you you know the the power that this you know within the sector then you can really make a change so it we really need to shake up the system and that's that's really you know important if we are serious about quality thank you dr cassetti for those stirring words you are a revolutionary theme i just want to give the final couple of comments to dr. Garcia Sifo and dr. Vera please to start just in brief okay thank you in indonesia too many guidelines was developed and too many train was too many health workers was trained but in the reality the health program with our health system are working in silos which is the problem the big problem problem and also for the culturally attitude toward quality is still a lack of from the professional and they don't not adhere to use the guideline that the big problem the big challenges in our country we need to change the mindset of the difference stakeholder regarding quality and safety and also how to move out from the silos into the integrity quality and the program thank you really quickly i don't know if that's possible but i'm just on the gap on between clinical guidelines and reality i agree with what's been said before i mean there's no link mexico has developed over 800 clinical guidelines all based on co-craine methodology to present the best available evidence on any topic that you want and then this is not used by clinicians because it's not presented in a way a clinician can use it so it doesn't really make sense to do this massive document with all available evidence and all graded according to to the source of these evidence if it's not going to be the right tool for the right people so that's that's something very important that that uh policy makers have to have to know i mean you have to basically bring down all these evidence to something clinicians can can use in the five minutes they might have in front of the patient and and and how this should help them to make better decisions when they are within the within this context of of uh service provision so that's that's what we're trying to change now basically modify this view of the clinical guideline like these massive documents that will lead any anything or any any provision of service to more of practical algorithms and decision trees for clinicians to know what to do if they have a doubt at any moment and then on on how you bring together all these actors so we don't have a problem with with external donals for example in in mexico it's it's easy to coordinate a few NGOs that work within mexico but it's it's a problem to actually bring consensus with all the unions for example we have over 300 000 nurses in the country represented represented by different unions and then they may protect different bits and then quality of healthcare is not or might not be the priority for everyone please see will of course be in into this course but might not be in terms of how much time they dedicate to patient or how much time they work or how to do for example transferring between uh different personal work in different times and and this bringing them all to the discussion is the key aspect of this basically they have to be part of this they have to be involved with making all these decisions and uh and this is true as well for the private sector there was something about the private sector and how do you involve the private sector well it's true as well for what we're doing doesn't really see institutional settings it sees mexico as a whole as a whole country and so all the quality policy applies for everyone from the regulation side so it applies to public social and private entities from the uh perspective side we we give resources to both private and public entities to to improve service and and we also provide all these tools and and training online training for everyone not not necessarily related to any decision study thank you um and thank you all for those comments before i hand over to dr shamside for concluding remarks i want you to join me in a big round of applause please for okay so good morning everybody i'm sure you'll all agree this has been quite a an intense discussion there are many things that are probably unanswered but that was the purpose of this morning is to actually stimulate our thinking get things moving on quality we heard right from the beginning the developmental context that fin purely put forward very clearly but he also came back right to the human moment of of childbirth i think that was a very important start to the day we heard from lara about the empty promise of access clearly articulating the need to link u hc in quality we heard a very important point from ed kelly about the opportunity to do something right and how many times you do it right a very simple way of putting it that my mother could understand i think that's probably one of the things that we have a problem with in quality is we do have a tendency of over complicating but we also heard about the humility to learn from our experiences margaret placed the local trust right at the heart of the discussion before we had an opportunity to listen to five authors directly which is a rare opportunity to listen directly from those that are publishing and of course all of those five aspects are different dimensions of quality that we need to consider there were granular details but it's an opportunity over lunch to be able to discuss a little further with them but we heard the impatience of a quality lead even while the academic colleagues were speaking in terms of andrew lee karka's question which was a very impatient question which is exactly how it should be in terms of how that evidence is going to be translated into change at the front line i'm a little biased but i i i do think it was the most important session as margaret has highlighted was the final session to understand from the quality leads themselves and an ex-minister of health what does this all mean to them as they move forward we heard very clearly the linkages between quality and resilience we heard very clearly the linkage between diabetes and overarching quality we heard a very interesting and important question that delved a little bit deeper into that but that is a really important area that we need to explore in a lot more detail we heard from somebody who is responsible for millions of people in diverse islands and this is an important point that sometimes we forget the unique challenges of small island states or small islands within a state and then we had heard the inspirational words from Dr cassetti who really emphasized the need for a quality revolution so i'm just going to add just three big pieces of wisdom that i'm taking from this this morning the first one is is a no-brainer but it's an important one to emphasize the linkage between evidence and policy so when quality leads around the world are developing their national quality policies and strategies they need to be informed directly and in lifetime from the evidence that's being generated and these quality policies and strategies are being developed as we speak so this is the right time to get the information to those that can act change the second one is the learning agenda we heard the critical importance of making sure that the immense amount of learning the breadth and depth of learning that exists on issues such as maternal neonatal and child health how is that going to be translated into learning for the wider quality community we have many mechanisms to do that there are many learning communities around the world to do that but we need some convergence around that and the final point is the point about the quality revolution i think that is an incredibly important point one of the things that we've clearly heard and i've had the privilege of of connecting with many quality leads across the world i think the time for persuasion i think we're there there is no minister of health that doesn't recognize quality as a key priority in their agenda today and i think that for us is our call to action so for that i do believe that there are a number of global momentum points that we can use of course all action is local but the fact that the global report on quality will be published later on this year it's a joint endeavor of oecd world bank and who it's an opportunity to keep that global momentum alive of course margaret has clearly articulated the work of the lancet commission another opportunity to keep the world alive and energized about quality but ultimately of course quality is local and quality is about people i wanted to thank you all for being here and giving up your time and i'm going to ask dr ed kelly to give a word of thanks before we close thank you well it's just um as the person who welcomed you in it's just to send you out um i told everyone they could bring their food in i just remembered that i had stuffed the croissants in my pocket here that i've not eaten um so please take the food out as you leave because that's illegal to bring food and drink into uh into these meeting rooms but we're really very grateful to all the all the panelists um and particularly margaret's team but also the bulletin team that helped put this together and really also to all of you many of you are very busy um i the only person i know who is not busy as anthony castello has nothing else to do except worry about quality so i'm good thankfully we were able to fill his otherwise unfilled morning um but uh clearly i think we like sham said we didn't propose that this was going to have all of these answers but i do think i'll come back to the point i made earlier that for who and under its new leadership i really do believe personally and we'll definitely make it part of our work jointly with the work we're doing with anthony's team but also with the uh folks in hiv tv malaria that this is the unifying theme across the the implementation on universal health coverage and i think that clearly there what we try to talk about here was tease out the universalities of the of the issues but also look every country is going to be particular and we didn't get into sort of uh conflict zones and fragile state issues but there's a lot of work going on there there's a lot of work going on in the link with emergencies program around how the quality approach needs to be needs to be uh modulated so i think the point was that we don't uh it's not that we have had all the answers and just nobody's listened to us till now although i do think that nobody's been listening to us until now but um it's good that they are listening now but i think we need to look at how quality in uh 2017 and beyond manifests itself in terms of achieving some of these big goals around universal health coverage achieving health security advancing the amr agenda and achieving the progress on non-communicable diseases so with that i'll thank everybody for being here and we'll look forward to being back in touch very soon thank you yep good and can i just mention uh thanks to our organizers here we have lunch outside so you're welcome to annoy all of the office holders on this floor outside there and and have some lunch with the panelists thank you i know michelle very well she's been talking to you right she told me all about you yeah you're connecting these people yeah of course yeah now we'd be easier to see what can happen i know you're busy and many friends so that's the issue i think only so many people and so many things can be tackled but if quality if we can get something with Liberia too for the commission that would be great yeah for sure very nice to see you i mean yeah now we can be in contact exactly i know michelle told me about you or like have you not met up before i was like no somehow i didn't meet you in Liberia um yeah yeah yeah yeah yeah yeah nice to see you and thank you for moderating the session yeah it's moving it's very extremely busy but we're pushing hard so i hope we'll see what the feedback is from you and from others it will be interesting to get the conversation moving huh thank you okay good to see you okay hi my i'm wonder from emerald w2 emerald in kind Yeah, it's an area where we're interested in the commission as well to understand what is the thing to think about quality and the... Exactly. So we started thinking about the idea of what framework can be developed in order to save quality and the quality that needs to be processed. And you know in a number of cases, like in many countries also because they are in trouble. So this is the priority. And we do know exactly how to do this. So most of the time quality and safety will be neglected. So this is the idea. We are now writing up like a question for the idea. And then we will probably make a little one. Program. This is the first step. Then we will put up the steps. After the steps we will look at the process. Okay. I'm going to think about the white paper. These are the questions that I'm going to answer. Okay. Okay. Good. And then after that you can make some things. When is the towel going to be ready? Yes. Okay. Go ahead. Yes. And then we will discuss the other questions. If you could give me your contact here. Sure. Yeah. I would like to hear it. I have. It is important that everybody can be in touch with you. I'm sorry, I don't have my card, would you definitely be able to find me? Perfect. Very nice to see you. Okay. Thank you very much. Thank you for having me. Thank you. Thank you. Thank you for coming.