 I'm going to share my screen and Brenny, you can see my tablet screen. Yeah, that's perfect, George. Okay, so good morning, thanks to all of you for connecting again to intense sessions. So I'm going to start with the GS1 Data Matrix Code Parsing, which is not a program in its own right like you have seen for the biomedical equipment lifecycle management or for the stock management reporting mode or the real-time mode, but it is kind of a tool maybe like a Swiss knife that can do many things. So I have prepared the power some slides, but to start with I would like to just basically demonstrate to you what this does, otherwise it's you know it might be too abstract. So I'm just going to show you the two main use cases. So I'm just want to show you that I actually have a GS1 Data Matrix Code on the screen, right? Because I can't show my tablet and presentation at the same time. And I'm going to use it right now. So the GS1 Data Matrix Code Parsing, you have it in the sandbox, right? You have received the link. I hope you have all been having fun in the sandbox already. And this is an event program for those. And you open it and click on the plus. So that's for creating a for scanning a new code. And then this is just because I have admin rights. So I'm assigned to everything. Otherwise, if you're actually going to use that, and of course your user rights in practice would be limited to a certain facility. So this is what the screen looks like. And basically what we have done, it's not customized to a specific application. So this is basically a mix of the two main use cases, which is scanning drug products and for the global asset management. So we'll just show you how it works. We'll just click on the icon on the top and then go to your screen and scan the GS1 Data Matrix Code. This is really a fantastic tracker innovation. It works always. It's very crisp. Scanning GS1 Data Matrix Code works better than linear barcodes. Normally then you have to fiddle around with the focus and the distance and the angle. But these ones you've just come close to it and it like sucks in the code and it's immediately shown. So what you see here is the typical use case of a serialized drug product. So you have the product identification number, could be a G-team. So that's a global trade identification number. Then you have the batch number and the expiry date. And you have the serialized number. So this is an actual drug product. So every secondary packaging of drug products in some countries, it's already legislation in the US, Japan, European Union, I think also in India, maybe Pakistan, several countries are implementing this. Every secondary packaging has its own distinct number. And this is also a requirement already from UNICEF and Gavi for all the vaccines and the expiry date. So maybe I'll explain a bit more later. So that's the first use case just for you to understand that you scan a GS1 Data Matrix Code. And the particularity is that you have different fields and DHIs too will automatically identify which field it is and extract the information from this string and place it in the field. And then you can use that for searching or for processing in another way. So I'm going to show you the second use case before I go to the actual presentation. So I'm just going to clear this. And I'm going to scan now another GS1 Data Matrix Code, which is the use case for the global asset management. So for a piece of equipment, typically a refrigerator freezer. So I scanned again, very simply, just with a camera that is built into the tablet. And here you can see that you have the production product number. You have the serialized number. So this is the requirement that in future, called gene equipment, they will all have the serialized number. So you will already have a unique identifier attached by the manufacturer, no need to create it. And you have the production date. So there's one field that is also that is included in this code, which is a PQS number. And we still need to iron out a glitch that it's not displaying on mobile devices, but I'm sure we will sort it. So you can see here on top PQS 003 007. So that's the refrigerator, the PQS category that this device belongs to. Okay, so this was just to show you what this is about. So you have basically see how this will be used in the world in reality. And I'm sharing my slide show now. Brenno, can you see a Google Drive presentation? We see a presentation, yes. GS1 Data Matrix code parsing. Come again? The screen is changing. Okay, now we see GS1 Data Matrix code parsing. Okay, so I call it a multipurpose application. Okay, so the first one is serialized unit packs. As I mentioned, I think several times, so you see that every product has a serial number. George, we don't see the presentation anymore. So this is George? Yes, let me turn off the presentation. No, I'm trying to share. Let me try sharing. Can you see it now? Not yet, no. It was on just a few seconds ago, and then it went away. Okay, thank you. Let me try again. It's not on purpose. I don't know what happened. Okay, it's coming now. Let's see. Now we see just one Data Matrix parsing serialized unit packs. Okay, thank you. Okay, so this is the information that I showed. If you scan this GS1 Data Matrix code, then it will parse these fields that you see in the left side. And on the bottom, you can see this is an image of an actual package that the way it is printed, the human readable interface that is printed on the package in the GS1 Data Matrix. So they are very small, but they contain all this information. So this is the first use case. So somebody mentioned, anyway, I will come to that. The second one is a Global Asset Management, which is a new PQS, or not a new, but a WHO PQS requirement. And the ideas that the manufacturers are going to furnish all the equipment with GS1 linear barcodes in that case, but in the future, this might be GS1 Data Matrix codes. But the principle is the same. So health worker, immunization worker, storekeeper can scan this linear barcode and then can then use the parsed equipment for, as I have shown yesterday, for the biomedical equipment, life cycle management, for example. So here, for example, you have a serial number here, 2281963. And you could use the serial number in conjunction with the product number as a unique identifier would not have to create it. And just to note, so this is important. So you have application identifiers, 01, 21, 11, 241. So those are basically the numbers that are indicating what follows in the field. So for example, 11 means the production number is following and it has six digits. And 241 is the customer part number. And that is used for the PQS catalog number. So that's basically how it works. So that's what I just showed. So the main functionalities is that the storekeeper, immunization worker scans a GS1 Data Matrix code at receipt or before issuing a drug product. Let's take the use case of a vaccine. So you would take the vaccine out of a refrigerator or a cold box and prepare it for vaccination. But before, you would scan the GS1 Data Matrix code on the secondary packaging. Maybe in future, this will be also printed on the violets to be seen. And now one of the benefits of having this functionality in DHIS2 is that the DHIS2 could in principle do a lot of things in the background with a single scan. So it could verify the product that this is genuine. You could use it for batch level management. And this could be running in the background. So the use cases that we have so far and there might be more in general that I will go through briefly are the product quality and verification and alert for falsified product. Then there's the vaccine by monitor. There's an electronic version for that that is being developed. I don't know if it is already available on products. You could use it for the national track and trace systems, stock management at the batch level. You could record the serial number of vaccines in the electronic medical record and for managing adverse effects. You could also consider think of using it for providing updated product information from a database. And then like as we briefly discussed for end-to-end temperature traceability for linking the products with the refrigerators. So I will not go through that in too much detail. But the basic idea for the first use case is that the immunization worker will scan the GS1 data matrix code just before preparing the vaccine. And DHS2 will parse that code and will synchronize it with a local database. And if there is a note in the database or let's say the national traceability track and trace system that this product is not genuine, then the immunization worker would receive a notification before the vaccine is injected that this is not a genuine product. And can then put it into quarantine for investigation, but in any case make sure that it is not used. So that's the first use case. Jump in if you have any questions or comments. Okay. Then there are several use cases that are closely linked to the national global traceability. So there's a big project from Gavi Global Fund UNICEF on traceability and verification system where in addition to identifying falsified medicines, you also want to track and trace every secondary packaging through the supply chain from the manufacturing plant until the end user. So the national LMIS system would have to have the capability of managing serialized products, which is quite a high bar. And the DHS2 could then cover the first data mile or the last mile. So the idea is that first of all, you can trace exactly when and in which place which serialized package was opened and used. And you can also use this in the database for what is called decommissioning of the serialized number, serialized package, because the idea of having a central database is that you basically feed it with all the serialized numbers that are manufactured and that are, let's say released. The product release has been completed at the manufacturing plant. And then when that serialized package is actually distributed, then it is also recorded in the database. And this is basically the mechanism that prevents that somebody who uses, who would falsify a product would just like reprint the packaging and try to sell the same serialized number second time. Of course, you would have to work out if you have two identical numbers, which one is the genuine product, but I'm sure that there are systems which can do that. So the global asset management, this is a WHOPQS standard that you can read. And the main use case is what I presented yesterday for the biomedical equipment life cycle management, which can be also used for cold chain management, is that you scan the GS1 code, it can be linear or in future data matrix code, and that will then allow to uniquely identify the equipment. So there's no need to have a parallel system with unique identifiers, because that is quite cumbersome. You will have to have a separate database and make sure that it is updated. I just hope that these labels are then also displayed in the prominent place and not on the back of the equipment so that they're easily accessible. And so you have the, as I showed the product number, you have the manufacturing date and the PQS number that is encoded. So that's the standard. So you're sure that every manufacturer is going to print the same information on the product? Yeah, product number and the serialized number, that was the fourth one. Okay, then the electronic vaccine vial monitor. So this is more our concept of being future ready. I'm not aware that this is already in use, but let me know if you have experience. But I guess you're all familiar with vaccine vial monitors that have to be read out manually. So there's a color change and then you have a table and you can compare and check whether that vaccine has been exposed to cumulative temperatures which exceed the limit. And if that is the case, then you must not use a vaccine. But there is a GS1 standard now and it is possible to print the label on a vial with a special ink that changes the color, but that can also, where the color change can be read by a barcode scanner. And with this GS1 standard, the color change that is detected by the barcode scanner can basically tell the health worker whether that vaccine can be used or not. And if the vial has been exposed to cumulative high temperatures, then the health worker will receive an alert to not use. So the reading of the vaccine vial monitor requires training, it's a bit tricky, and that would help to avoid mistakes. So again, you scan the code once and it gives you all of this information, whether it's falsified, whether there was a batch recall and this information in one. So it's not any additional work. That's the beauty of the system that can do many things with it. So as I mentioned, on the second day, we have this idea, but we have to study the practicability in future iteration, maybe 2024 after, to use the GS1 data matrix code for batch management. That would be the ultimate batch management tool, because every GS1 data matrix code on every secondary packaging contains the expiry date and the batch number. So you can uniquely identify every package when you remove it from the shelf. Almost all healthcare products already have barcodes, but those barcodes are just the product number. So it has no information, it's like six, seven, eight digits, but it has no information on the batch number or the expiry date and cannot be used for batch level management. We know that when we have calls, this is a frequent request that we want to manage batches, but the main challenge with managing batches is that you have to be somehow bring that information into the system. And practically speaking, that can only be done if it is somehow printed on the packaging because you don't want to have the storekeeper like having to enter the batch number every time they pick something off the shelf. This is already a requirement for all the vaccines that are distributed worldwide, at least the UNICEF-Cavvy requirement, but it might take a very long time for having all of these GS1 data matrix code, all healthcare products, because it depends on national legislation, but it's something that we should definitely prepare for. One caveat that sounds like really exciting, now we can manage our stocks at the batch level, but keep in mind that the challenge will be that if you remove 10 or 15 or 20 packages from the store, then you have to scan each and every of these GS1 data matrix codes, which is quite a lot of work. There's also a system of aggregation and disaggregation where you can have GS1 data matrix code also for larger packs and for pallets. That's a more complicated system, so that might help, but that might help if in the pharmacy you are distributing a tertiary packaging with 10, 12,000 tablets of paracetamol. But again, if you are distributing separate secondary packaging, each and every code has to be scanned. And this is something I think that needs to be tested in practice, depending on how the number of packages that the storekeeper is scanning every day, how practical that actually is. GS1 data matrix codes are very small and a bit fiddly for scanning, so this is also something that we will have to optimize. The other benefit of if eventually this is available is that if the system is recording all the batches and expiry dates that you have in the system, then if the storekeeper picks the wrong batch according to FIFO, then you could provide a real-time alert saying you have another batch with an earlier expiry date, so please put that batch you have just put back on the shelf and select another one. And of course, if you could manage your stocks at the batch level and the expiry date level, then you could have calculations of your remaining shelf lifetime and you could easily calculate in real-time the risk of batches going out of stock. Okay, recording seal number four, adverse effects following immunization. So that's an obvious benefit. Imagine that you vaccinate a child and you have checked that the vial is genuine, you have checked that it's not expired, that the BVM is in the range, but in addition to that DHS2 does feature an electronic registry, you could for every vaccination also register the serial number. That would then greatly help in case you have an adverse effect to inform the manufacturer saying we vaccinated these five children, we determined the following side effects and these were the exact serialized numbers of each of the product and that will help the manufacturer to check whether there was any issue with the batch and it might also it would also allow to actually check the track of this serialized package through the supply chain and to see whether it was exposed to high temperature anywhere. Updated product information, so with the exact product identification, you could unique identification, you could envisage that you can connect this product to a database and the health worker could or physician any type of health professional could retrieve the updated information of the product from a database, maybe even select language so you don't have to look into a long patient information in the package which comes in several languages and you could imagine that you have like a little catalog for those commonly used products that you could store that information like save it on your local database. Temperature end-to-end traceability, again we touched on that, that would be rather it was a suggestion yesterday, imagine that you receive the goods in the store and you scan your GS1 data matrix code and then you scan the global assets label that I showed before and you kind of associate the every batch, every serialized pack with the refrigerator where it is stored and then if you monitor the temperature with the future DHS tool or another tool then you can make that link and we are thinking of doing that actually with a relationship management in DHS tool then you can basically know exactly what date, what time the serialized pack was placed into what refrigerator then you need to scan it when you take out, take it out and then you know exactly until which date and what time it was in which refrigerator and then if you want to know the exact history of temperature history of that serialized product then you could go into a database DHS tool in the future and check the temperature record doing that time but ideally we are thinking of if the serialized package are tracked at all levels so including in the national system then in principle you could like piece together the temperature records from the different warehouses, let's say the national store, the regional store, the district store plus ideally doing transportation and ideally even in the code box taken to the vaccination site and you could have a complete end-to-end temperature profile for each bile from the time of manufacture until the vaccine was actually administered to a child so this would be the dream come true and something that I think I didn't mention mentioned yesterday but we are thinking of also measuring calculating the mean kinetic temperature which is a nice summary of the entire temperature exposure and that would be a quick measure to determine whether the allowed temperature exposure has been exceeded or not. Okay I think that concludes my presentation, there's one question. Okay, is this system for monitoring the availability of vaccines in whole facilities? I'm sorry if you can maybe just elaborate on the question. George I think it would be good if we just make a quick summary and move on to the performance management framework so we ensure we have sufficient time. I think for here you showed very clearly the what DHS2 can do now and what we've developed in terms of the ability to parse the GS1 data matrix and identify these different identifiers. I think what would be interesting is for the participants to use the recording and use the presentation to go to each use case and if it's applicable to also see what kind of follow-up they may want to take with us and explore further the seven different applications which you showed here because there will be some work still to be done for some of these but it's a really great introduction to this and potential use cases and that they can then follow up referring to each one of these. Okay thank you very well maybe just one last comment actually important I forgot is I mentioned this is a kind of a Swiss knife universal tool and actually what you have seen I just this is just a tip of the iceberg so this is basically seven I think six application identifiers that are used for these two applications but actually there's a whole catalog of several hundred and I think that DHS2 is currently featuring allows to parse 300 of them so there's many other fields and that means there's huge potential to use this in future it will not require any development tracker team has done an excellent job so it means we don't have to every time ask them we have a new field can you add that AI to the to the native program rule you can just basically scan the GS1 data matrix code and if it includes one of the 300 application identifiers you just give the right name to the field in the program rule then it will parse that information correctly. Okay so I'll move on to the performance management so some general notes on performance management is that okay I should maybe mention this is a concept that was developed together with WHO in 2020 for some specific programs and to identify indicators that can be used for the WHO metadata packages but then we try to like generalize those indicators rather than having specific indicators only from malaria or or TB and have a small set of simple logistics performance in metrics that can be used for any type of healthcare product so as a general comment there's a need to measure indicators and management performance at all stages of the supply network so there's a tendency to measure this maybe at the national level which is good it's also it's necessary but it's not sufficient and I'll come to that in a minute all actors should use the same indicators that's really important so that you can compare metrics so if the healthcare facility or the district store the national store are using different metrics or different scales like somebody saying understock is less than one month stock and somebody else is using under less than two months as understock then you can't compare the indicators and it limits the use and it's really important that all actors at all levels the health workers the logisticians anybody who has a stake in managing supplies or using supplies that they all know their role in analysis and metrics and taking corrective action so I guess all of you who have been involved in performance management know the blame game there's it's a fact that the supply chain is influenced by many stakeholders by many actors and all of these actors can contribute to improving the supply at the at the last mile some have more influence some have less but still everyone can contribute and it's important that everyone works together providing their contribution rather than only pointing fingers at others how they could improve now the next statement is kind of my mantra data from healthcare facility level is indispensable and it's not possible to diagnose supply chain problems from secondary data from upstream logistics levels now you might just believe what I'm saying so my story about this why I actually came into logistics was that I actually trained as a nurse and I worked for five years in hospitals as a nurse and I was then sent to the field on my first assignment to Afghanistan to Jalalabad for one year where I was in charge of managing a medical store I had no training basically no real experience I don't close to nothing about logistics but like many I have I started with big spreadsheets and basically calculating how much I had distributed to a large hospital in Jalalabad and two other facilities and my stock never balanced I always had shortages and overstock and I simply couldn't work out what I was doing I spent a lot of time with a calculator with a big sheet of paper because we didn't have computers at that time and trying to calculate from the average distributions from the medical warehouse in Jalalabad to the hospitals how much I would be needing every month and it simply didn't work out I spent a year in in Afghanistan in Jalalabad and never worked it out so after doing the same thing in Afghanistan and also not in Kabul a second year and not getting very far I realized that I need to study this and I went to a university in the UK and then within a few weeks there was a presentation on the forest effect on the bullwhip effect and then suddenly I had this enlightening moment where I understood that I was not getting anywhere because I was not using end user data so this is something that J Forester discovered in 1956 has created the whole system of science of system dynamics actually actually and the whole theory about behind that big institute at MIT and that's basically why I'm making the statement this is not random that I invented it unfortunately this was long discovered like 70 years ago almost 70 years ago but I simply didn't know that and I have studied that in the meantime so I'm absolutely convinced that if any country for any product you want to sort out your supply chain you have to have facility level data because it's a mathematical certainty that you the secondary data so the data on what you distribute from a district from a regional from a national warehouse to a facility or to the downstream level is always distorted and for many reasons you can't prevent that the only solution the only cure is to collect data at the facility level and that's why DHI is always so a great tool for using at the facility level the other important point about the about the framework very simple framework actually which is maybe not very obvious is that distinguish between symptoms diagnosis and therapy so symptoms we call the manifestation indicators simply say that there is a problem and it's important because to make that distinction because if you detect that you have shortages you have stock outs you have detected the problem so you have identified the symptom but you you stock out the shortages I show later can have several reasons so you need to know that you have a problem but it's not enough it doesn't tell you how to improve the situation so for this you need to have a diagnosis like in a patient so you need what we call the root cause analysis and that determines the underlying problem that I will show briefly and then equally important you need to take corrective action so often we have detailed analysis which identify the symptoms so we know what is in shortage the stock outs where they are and when they took place how often they took place and then there's a risk that you stay with this analysis and don't go further but you need to determine the underlying problem that is already not done that often but then the important point is you need then from the diagnosis to determine the corrective action otherwise you have not gained a lot even if you know the symptom and you have done the diagnosis but then nobody takes action or the action is not effective then you're not improving and the mantra of the whole performance management framework is that the ultimate objective is of course to improve services to patients at the service delivery point and that means to ensure that all healthcare products are available at all times and that you prevent shortages that's always the ultimate objective and not the analysis itself so imagine that you were to detect testing let's say testing a child from malaria and then not taking any action for treating the child if the results are positive would not be of much benefit you would just like have you tally sheet and say I diagnosed 99 children with malaria and I confirmed with a lab test what type of malaria it was but of course you need to do a therapy so in the healthcare system it's obvious that just determine detecting symptoms and determining a diagnosis is of little value but on the other hand the the symptoms and the diagnosis the diagnosis and the therapy also requires determining the symptoms so I'm not saying that determining the symptoms that the child has fever for example or that you have a shortage or a stock out is unnecessary I'm just saying that always use those three together they are all interrelated and indispensable and should not stop at either step but we have to go to the all the way to the therapy so for the manifestation indicators quite simple in logistics fortunately shortages and stock outs are commonly measured so a stock out is nothing else than a coverage time of zero so it's kind of the same metric it's just just another word so in fact you have a single metric that you need to measure which is a coverage time and that's there's other metrics but the idea is that one simple measurement is sufficient we have seen with our work that you often have redundant indicators that are measuring the same problem in different shapes or forms maybe it helps a bit but it's don't think it's essential so basically focus on one indicator one metric that is most meaningful to determine the shortage and stock outs and measure that consistently it's maybe more efficient than having several and using them differently it's important to determine a baseline so you need also because you want to measure the improvement obviously so you need to determine the number the coverage time the easiest way to count is a number of stock outs by facility by month for example determine a baseline from historic data let's say for six months and then that will be the benchmark to ensure that you have actually improved if you measure only stock outs just a note and not the coverage time you have the risk of gaming so that's something that is very well known in logistic social sciences and that is studied also in academic papers is that always be aware that depending on how you design your metric you have the risk of gaming so people will avoid will kind of trick the system to avoid that they that they look bad for example you could if your stock out is your only metric and all you need to do is to keep the last vaccine in the refrigerator you will never have a stock out just the last child will not be vaccinated that of course not the objective of performance management but just to tell you that every performance management system you know can be tricked and you need to be aware of that and try to hedge against it so on the current stock outs and shortage we have the three dimensions of the items so you want to know what items are in shortage in what places are the shortage and stock outs occurring and then the time so you want to know how long has that shortage occurred and then the second dimension is to look at the at the historical development so looking at development of a time so whether at the item level whether the number of items that are in shortage are increasing or decreasing the number of locations are increasing or decreasing and whether the length of the shortage of the stock out is increasing or decreasing so this is kind of to sort out the different dimensions because you can have any number of charts dozens of charts and they can be confusing to understand which of them are actually redundant or measuring something uniquely so that just allows you to structure this information and I think if you capture those let's say two times three dimensions then you have a quite a good grasp on on your on your symptom so how could you do that in THHS too so this is a beautiful map Scott Frost Patrick who is going to present shortly develop this for Malawi two years ago and I think this is like widely used it's like native THHS to analytics functionality and you can create a map here in this case you can look you would look at item one item pcg vaccine you could look at it by district and for the current month you can also do that at the facility level and the regional level of course so I'm not going to show all the options and we're still at the symptom level then you can look at one item by district sorry you can look at at the items across items so several items by district over time and then you could also of course envisage doing the same for one month but then across district so there you always have these variations of these different dimensions that you can combine with each other and different so now once you have to once you have determined the the the symptom which is the shortage as a stock out the next step is then to determine the root cause so some general notes it's important to determine at what threshold will which person take what action in what time frame and what indicator will change so that's a lot I come to that later but that's basically the objective of the diagnosis is to determine the appropriate corrective action just to mention the often overlooked elephant in the room is inventory control and stock replenishment at the facility level so that's absolutely critical I've spent 25 years many of them in the field lots of hospitals clinics and sometimes you found that the the facilities were desperate with stock outs and shortages obviously as a logistician it was always my fault and that's like our destiny but in fact you find sometimes that the very simple calculations are not correct for example the lead time is not considered correctly or the buffer stocks are too low and then it means that you have shortages and stock outs by design of course this is not always the case but it's something to absolutely consider in supply chain management unfortunately you have to look at all the aspects it's not sufficient to just reduce lead time you have to look at all the actors in the supply network all the factors in order to make sure that the goods are available at the facility level so I already mentioned that the stock out is just an extreme case where the of the coverage time of being zero and I would mention the game the gaming app so just to reiterate diagnosing a child with a fever or not diagnosing determining that the child has a fever is just a symptom but you need a diagnosis so I'm this is basically the outcome of the of the work done with WHO very simple it's actually quite amazing if you think that stock outs is a very common problem but actually that there are not that many causes you can maybe make a long list of causes but at the end of the day you can kind of cook it down to a few and just to mention this is the root cause analysis at the facility level if you're looking at the same analysis at the distribution center at the district or regional warehouse you will have other possibilities you have the whole warehouse management and so on but I'm just focusing at the facility level and fortunately it's quite simple so you basically have three reasons why you can have a shortage that is because the delivery is late what is important here it's not about just eternally shortening delivery lead times late delivery means that the delivery lead time is longer than the promised customer lead time so that means that if the district warehouse is promising delivery in two weeks and then repeated it delivers in more than two weeks then that is a delay late delivery but if a district warehouse would promise to deliver within one month and deliver in that time then that is an on-time delivery so the late and on-time delivery is not related to the absolute lead time but to the promise because if you know that the delivery is going to take one month then you can increase your your stocks accordingly and still avoid a short shortage so that's also one of the traps is to try to just reduce lead time I think this is one of the misconceptions about using drones in many places because even reducing the lead time to zero is not necessarily going to eliminate stock outs if your inventory control is not correct and as I mentioned impressed level your stock order up the levels or your reorder levels there are different names are not high enough then you will always have shortages so that's as I mentioned I think the elephant in the room that is often overlooked and then irregular order frequency in periods that's also a problem that is not visible at the at the last mile and it is again related to the forest or the bull whip effect to demand distortion because if you place let's say the facility is supposed to place their order every four weeks and once the order after three weeks and then they order one week early and then the next month the order on time at the end of the month it means that the demand in the first order is reflect three weeks of demand and the second order will reflect five weeks of demand and that's a huge difference and that is exactly what unnecessarily causes demand distortion and then which amplifies across the supply chain it makes it impossible for the medical planners at the district region national level at the manufacturers to correctly plan because they don't know whether the demand has actually decreased or increased or it's just an irregular order and then shortages and stockouts always to keep in mind that we are always talking about improving services to the patient so the effect is if there's a shortage the patient is not treated at all not sufficiently treated not treated with the right product or not vaccinated and the patient health can deteriorate or is not improved as it could if the products would be available so you see very simple not that many reasons now the bad news about the root cause analysis is that it requires quite a bit of data and that data will not be available in DHS too because you're only basically recording the stock on hand and the stock distributed consumption data but it is native in any professional upstream national LMI system so the screenshots that I have taken here is just an example of analysis from an ERP system you need to have the data on the lead time but you can see that the analysis is actually straightforward so for the first root cause late delivery all you need to do is basically compare the promised delivery dates that is the vertical red line with the actual delivery dates and then usually you will set a threshold of 95 99 percent and you will determine let's say 95 percent of the of the line items are delivered within the promised lead time you're probably doing a good job 98 percent would be better 100 percent is very difficult to achieve but it's fairly straightforward and the corrective action is then that's also important that either reduce the lead time so if there is a discrepancy so let's say you have only 80 percent of the line items are delivered on time then you have two possibilities either you reduce lead time or you change the inventory control parameters it means that you increase the the lead time in your calculation or to put it simply you just increase your buffer stocks and you will also not have shortages even if the lead time is longer inventory control as I mentioned so this is an example of of an ERP system where let's say LMI system where the user is simply told which reorder levels need to be increased which ones can be decreased and which ones do not have to change so this is something that should be reviewed regularly and you should have a sound algorithm because obviously for all those items that you see if the reorder level is too low you're going to have shortages and stock outs by design regardless of the efforts of the upstream supply chain and then the third root cause is possible root cause is that irregular irregularity of orders so you can see at the bottom this is line chart this is also quite simple to measure so you just determine a window so let's say a facility supposed to make the orders on the 23rd of every month plus minus two days you have a green line if the ordered on time and you have a red line if the ordered too early too late or if they made additional orders again which causes demand distortion so you will need an LMI system in order to determine the root cause analysis but once if you have implemented the system then the analysis again is fairly straightforward and it's our conviction that anyway if you want to improve the supply of vaccines or any other kind of healthcare products in a country you will eventually have to have a national LMI system we realize it might takes a big effort and might take years to implement but absolutely convinced that just using paper and excel spreadsheet will not do the trick yes sorry there's just one I think there's only one second there another root cause for stock outs is of course short remaining shelf life and that you you have enough stock but it expires before it can be used so or that or that stock was damaged so you could have a perfect supply chain but then your cold chain is broken you have vaccines is expired you also core shorter so that's kind of the other path that I think is quite obvious then the corrective action that's the most important point symptoms diagnosis now you have to somebody has to swing into action and the therapy is basically addressing the root cause as I mentioned inventory control is to be kept in mind so as I mentioned before I will not go into detail here but this is an example just to show that it's not very complicated and that this is actually practical to use you have to determine at what threshold you're going to take action so you could say you have seen that if the coverage time is only one month are you going to take action hopefully you're not going to wait until you have a stock out but anyway this has to be determined because otherwise you're measuring a metric and you have to have a clear indication when do you take action like you have a refrigerator and you know exactly if the temperature goes above 80 degrees too high you have to do something this is also really important which person so as I mentioned you have different actors in any supply chain so here you can see you have logistic services you have the healthcare facility let's say that's the management level you have the storekeeper the pharmacist and you also have the health professionals all of them have a stake all of them have a role and depending on the problem sometimes all of them have to take action so you have to determine who takes what action then you need to determine how quickly that should be done and then you need to determine what metric will be used to to measure your success so I will not go into details but you can see here basically of course shortages stock outs is the issue so you can see here what is the action that impacts for example lead time and how do you measure it what's target and what is the corrective action for the logisticians and the same at the facility level you have here the range that that will determine whether you need to act to take corrective action and then the corrective action so very simple frame framework but again I think we should measure a few metrics and then focus a few simple metrics do it systematically regularly and then make sure somebody takes corrective action so this is just another example of the same approach for for overstocking and product quality so these are kind of the three issues that were determined by the working with WTO obviously stock outs and shortages that's the top priority then overstocking is also an issue because you're wasting products and wasting valuable funds and then the product quality of course always on top of the mind and you have also a small framework for those so I think we think those three basically determine allow to cover a good range of of issues so I think I can close with that and we have a few minutes for taking some questions or comment or commenting George I think first of all a big thank you for that and for sharing your knowledge and insights you had first a lot of respect for your time spent in Afghanistan and then for your nursing background so those are the first reactions but then they're asking to share maybe some of this content and I said that you both studied this and wrote your PhD on this topic and you have also a book on the topic and I wonder if we can share those on the drive with the participants so they can also reference to it and then I don't think we have a very specific question but multiple remarks on the the relevance of the topic I wonder if we can or if you can comment on how this then connects to the different features particularly of course the the stock features and how this can support implementing this kind of performance management framework what part what piece of the puzzle is the DHS2 stock management facility because you're showing something that goes much broader and much beyond but connecting it to action and a framework that should be implemented at a much broader level so maybe connecting that to what we're then proposing in the DHS2 features and actually one thing I did have a question from Nora who asked how the stock out is measured maybe we can connect that to the DHS2 management as well but if you could comment and connect those that would be great okay just while I'm going to open DHS2 to show the the dashboard that I showed briefly so I can show how that's in practice I'm just making a quick comment somebody scanned the QR code in in the tool and didn't get the result so maybe we can discuss bilaterally but do keep in mind the GS1 data matrix codes are completely different from QR codes so the GS1 data matrix code parsing you can use it exclusively for scanning GS1 data matrix code you can scan QR codes in DHS2 natively but not with this tool because it simply works differently. Then I'm going to log into the actual developer sandbox I hope I'm in the right correct database but basically you will see the same thing if you connect in the sandbox okay so as I presented for the for determining the symptoms it's only part of the framework but it's essentially you have to also detect the system you will basically that information is actually available in DHS2 so I briefly showed the dashboard so any any implementation of DHS2 or any implementation of stock management will always require that you measure the stock on hand at the end of the month if you're using the monthly reporting mode and this is basically how you would do it so you see here the folic asset had a stock on hand of zero in October November January and February and May so poor performance so you measure the stock out simply by determining that you have zero a quantity of zero on the shelf and this is done at a time of your monthly stock reporting now I realized that this is not a perfect reflection of the picture because you are measuring your basically stock out metric once a month only so this item could have been in stock the day before it could be have been delivered the day tomorrow so if you have a stock out in October 2021 of your folic asset you don't know whether that stock out lasted one day in theory even one hour or it lasted for six months but it is still an indication if you do that every month then you can manage measure the number of stock months that that item was out of stock at the end of the month the advantage of this simple measurement is that it is very easy to measure and it doesn't require any additional work so when while we work for WHO we actually I actually had a slide I think there's a dozen different ways of measuring stock outs and shortages but as I mentioned the easy way is a sustainable way so the stock discrepancies yes then if you are measuring the stock outs at the end even if it is at the end of the month only it is still statistically significant because then you can count the number of stock outs that you had at the end of the month and you can see whether the number of stock outs increased or not so even if you were to say well one stock out it was just bad luck that that day that item was not in stock if you do this over a long period of time then statistically the just counting the number of stock outs will still be relevant and the coverage time is simply calculating the the stock on hand divided by the monthly demand or consumption then you can decide whether you use the demand from the previous month or from the last three months or last six months that's kind of a choice to make and if you are yeah then for the for the coverage time what you can also do is you can count the the number of items that are out of stock that have zero to one or one to two months stock and you can create this instagram like the distribution and that gives you a good picture of your stock this is something we have used in our ep it's very very telling you can just look at the single chart even without looking at the details and you can see whether your stock is balanced or not because ideally all your items should have three to four months stock you should not have any shortages and you should not have any stock and then of course there's other ways of measuring stock outs and shortages so one very common way of measuring stock outs maybe the most common way is to is to count the stock out days so you count every day for each item whether it was in stock or not in stock this is a effective way but I think it's it's a lot of work because you have to basically tally your stock out for each of the item and you might have 300 400 500 in stock and then you must have to determine exactly when it went out of stock when it was replenished how many days it was out of stock and that is kind of very time consuming the good news is that if you have a real-time stock management system as I demonstrated on Tuesday then that system is recording each and every transaction it means that you know exactly the date the time even the minute when the last item was taken off the shelf and you also know the date the time and even the minute when the stock was replenished and this is the idea that if you have those data then you could actually calculate the the stock out days or even the stock out time in in hours automatically in the system without having to do any additional calculations okay I'll explain yeah I wonder if Nora wants to comment I mean you gave multiple answers to how they can be captured and also some implications do you want to to is that sufficient or do you have any other reflections let me see we can unmute Nora if I can find her quickly enough here yeah go ahead Nora okay we can't hear her we'll assume she's satisfied with the very comprehensive answer all right I don't see any other questions or comments I think yeah thanks again George for sharing this thank you for sharing the in-depth analysis of what you can achieve with the data being collected the importance of this consumption level data so the facility level data and how that can inform first identifying and then correcting both stock out and stock coverage and overstock Nora I think yeah that's great you can discuss with George bilaterally after so then I think that's good George if there's no other comments I think we can maybe end here and we can take our break slightly earlier than we're just eight minutes ahead of schedule we can end here and then we can come back in 22 minutes so quarter to 12 also time we can come back and continue with the next session on analytics we'll share as much of this content as possible on the drive including the the research that George has done on the topic and if you have any questions again feel free to reach out to us on Slack and then we'll take then the 22 minute break now and back at quarter to 12 are you Scott great thanks so much Bruno um can everyone hear me is it okay just a loud and clear loud and clear great well a very early good morning to all of you from the US where I am this week I'm usually based in Oslo with Bruno there but I've had to come over to the US for to attend a wedding so you got me up and in the middle of the night but I'm always very happy to do analytics and talk about analytics and also as Brenda pointed out quite flatteringly actually I think it's great to be able to talk to a crowd of folks who are interested in supply chain logistics which is a topic that is near and dear to my heart as well so Brenda's asked for me to do a very simple thing which is just give a quick overview of dhi's to analytics I'm sure you've seen little bits and pieces as you've gone through the academy this week on some of the highlights of dhi's to analytics I of course in this next hour don't have time to go over everything so I am also just going to give you a highlight of the core analytics applications I'm just going to do this all with the demo I think it's better to see how it works instead of just having to be told how it works through powerpoint presentations if you have any questions at any point please do just I don't know what's the best method of what you guys have been doing but I'm fine if you just put it into the chat channel here or just unmute yourself and speak up it's also totally fine okay so what I'm going to do with analytics is kind of touch on three applications the dashboard application the data visualizer application and then I'm also going to take us to the WHO data quality application as well the best place to be able to explore dhi's to on your own is our website play dot dhi's to dot org this is where we have access to all of our demo instances and we have actually as you can see here three versions of dhi's to so we have the latest version which is dhi's 2.38 we also have 237 236 and 235 and just to reiterate maybe you haven't been told this yet is that we support three versions of dhi's to at all times so we release two versions of dhi's to every year so essentially we're supporting 18 months of dhi's to versions at any given time the latest releases 2.38 and actually in just a few weeks we will be releasing 2.39 so 238 won't be the newest anymore in a couple of weeks but as it currently stands it is so i'm going to click on 238 and i'm just going to go ahead and log in and see my i've saved my login credentials here already i'm just going to make this full screen there we go and hide the chat okay so when i log in and this is a little bit typical the first thing you see is i have dashboards at the top of the screen here i have a list of all of my dashboards and you can see you can have many dashboards actually right now i've landed on delivery you see delivery here is in green that's indicating that it's selected and that's the dashboard and that i'm seeing if i click over to antenatal care you then see that my dashboard has changed and now i'm looking at the antenatal care dashboard and the dashboard of course has a series of different types of analytics so we have a sink we have a text field here we also have a column chart a map another column chart a stacked column chart and there's various chart types here all right let's just talk about some of the features that i can or some of the tools i have available to me on the dashboard itself then let's start from the right to left so next to the dashboard name here you see i have all of these buttons i'm going to start with the more option here if i click on more then i have quite a lot of functionality available to me the first one that's useful to point out is i can make available offline so if i click on the make available offline button then you'll see that the dashboard is reloading and it's reloading so that it is being saved offline so you see that i have antenatal care here this icon indicates that the dashboard has been saved offline so if i navigate to another dashboard it would not load if i was offline but this dashboard is available offline that also means that this dashboard is available to be viewed on a cell phone as well so if i am looking at this dashboard through my mobile phone's web browser so if i'm looking at my smartphone through either ios or android smartphone through the web browser looking at this dashboard i can save that dashboard to that web browser and be able to view it offline so even when my phone is not connected okay so you can see that this is available offline if i click on more again i have a couple of options i can remove from offline storage or i can sync offline data now i can also un-star this dashboard un-starring means that the star that was in front of it has gone away and the star is useful because it brings the dashboards that are starred to the front of the list so if i click on another dashboard with disease surveillance for example and i started that dashboard you see that it's now come to the front of the list if i un-start it goes back to its original location the point is that the starred dashboards make it if you start a dashboard that makes it very easy to find it in that's in this list if i minimize this list you'll see that only the top row is visible and then i can have all of my star dashboards here displayed on the top row all right a couple of other functionality here i can show the dashboard description i'll show you in a second how i'm able to write a dashboard description but if there was one then it would show up here you see there's no description available for this dashboard as it currently stands we'll change that in just a few moments i can also print the dashboard so printing the dashboard is a very useful feature for a lot of users we appreciate that there's a lot of times in which looking at a dashboard on a screen is not the best way to present it maybe you need to present a dashboard in a meeting or present a dashboard in a in a report and the printing option gives you that ability so if i click on print you see i have two options i have dashboard layout and one item per page if i choose dashboard layout then what it's going to do is it's going to render the dashboard automatically on an a4 size piece of paper landscape orientation the first page is the title page so that's the title of the dashboard and then you see i have my dashboard items each dashboard item here is in the order or in the orientation or configuration that you see it on the dashboard itself and you see as i scroll i'm seeing the first row of dashboard items on the first page and the second row of dashboard items on the on the second page the reason that they're showing up on two different pages is because dh is to is automatically put in this page break here between the the second and third pages and usually scroll down you see i see all of my dashboard items rendered on a piece of paper now if i click on the print button in the top left or sorry excuse me top right corner then i will have all of my various it will load the preview for me and you can see i have all of my normal print option so i can print to a printer or i can save it as a pdf if you save it as if you save it as a pdf then you can you'll have it available to you you can attach to an email or or print out as a report all right one more print option to show you i go to print again and then i go to one item per page then it's a very similar situation but instead of multiple dashboard items per single page of printing i have a single dashboard item per page so you see this is the first dashboard item available to me to print second dashboard item and it's just showing up in a single page and of course if i go back to my print options i have the same options as i did in the previous print so that's everything in the more menu now a very important functionality in dhs2 dashboards is the ability to add a filter to your dashboard so we appreciate that that you may be looking at a dashboard there are certainly times and situations where you want to drill down the hierarchy for the dashboard or you want to change the period of the dashboard so that the amount of time that's being covered on the dashboard so if i go to filter this is where you can do those things if you see that we have all of our dimensions if you're familiar with dhs2 data visualizer application or any of the other applications for analytics you'll appreciate that we have all of the same dimensions in those applications as well let's just do a few things for example here let's say i want to see my data only for the last three months for the entire dashboard if i click confirm then you'll see that everything will reload it looks like there's no data for the last three months actually here let's say somebody has gone through our demo database and deleted some data recently let's just remove that and instead of last three months let's change this to years and let's say last year it's still no data no worries we are prepared for this so the fun thing about those demo databases that you're seeing right now is that they are available to anyone in the world so there's a lot of people in there playing around sometimes things get messed up that's okay i have access to other databases that i can demo from important to note that you are available or you are allowed to do any number of changes in those demo databases that you want in the play.dhs2.org you can you can change things you can change the language you can delete data you can import data whatever it is you want to do but it is important to remember that the any change that you made you make will be lost by the next day so those demo databases reset themselves every night okay so let's get back on topic take a look at the filter i go to my period dimension i'll say last three months click confirm and you see that now my dashboard items are reloading with just the last three months every single dashboard item has now had its filter up its period changed to last three months and let's say i want to zoom in to just a single district here i'm currently looking at just the entire bunch of celion if i go into bow then you'll see all of my data again reloads just with the bow filled org unit this is bow and then you can see that i have my two periods or sorry my two filters displayed here at the top and as i remove them everything reloads and i'm back to my original dashboard all right of course you have other dimensions as well so you have like things like area diagnosis donor these other dimensions are defined but in your dhi's two instance so different countries will have different dimensions here um it's just however your system administrators have defined these additional dimensions all right the next thing is that you are able to share a dashboard so if i want to share this dashboard with many users um say i'm making a dashboard at national level and i want to share that dashboard down to the district level users then what i'm able to do is search for individual users or user groups just for the sake of demonstration i'm going to search for admin and here we have a user group called administrators i can select their level so i can give them view and edit permission give access okay so now i have given this user group access to this dashboard um you can see that i also have two other users groups assigned so i have external users no access and all users which is everyone who's logged in can view and edit the data now um uh an important consideration when um in the use case that i was just referring to which is when i make a dashboard at say national level and i want to share it down to the district level is that the people at district level will need to have the authority to not only view the dashboard that you're sharing with them and which you do through the sharing settings here but also the authority to view each individual chart map or table that you put on your dashboard as well um and we have a functionality here called apply sharing to dashboard items where you're very easily able to apply um kind of cascade sharing so if you share the dashboard then all the items on the dashboard also become shared automatically um to a user or user group and you use that functionality here so apply sharing settings to dashboards um and you see that i have an icon here saying all 11 items in this dashboard will be potentially updated and sharing setting with one user uh group which is that admin user group that i just added so that's why we're seeing this public access is not affected so if i click on apply sharing to dashboard items it says that no at dashboard items were already updated um because this dashboard was already shared publicly so when i just by adding another user group it did not mean that i changed any of the sharing settings because all the sharing settings were public already okay and the next thing that i want to point out is the edit so the edit is where you actually go in and you're able to build the dashboards um and dash the dashboards and dh is to are highly configurable customizable and so you can do a lot of different things with it you can change the size of all of your dashboard items you can see that i'm doing that here by clicking in the bottom right corner and dragging around i can also change the dashboard title i can change the description if there is a dashboard code i can also add the description so this is where we didn't see one before but i will add a description now i think a description which i'm going to spell description there we go um and i can change um okay so if i save these changes then i go to show description you can see now my description is there now if i wanted to make a new dashboard i can click in the top left corner this plus sign here and here i'm able to make a new dashboard very easily so let's just make one quickly for example i'll call it test i'll give it a test description and i can change my layout or the flow which i want to add my dashboard items so here i can change it to free flow or fixed column let's change it to fixed column let's change it to four columns save this layout and i can say where do i want my new items to come do i want to just come at the end of the dashboard or the start of the dashboard for the sake of demonstration i'm just going to change it to the start of the dashboard and now i can start adding my dashboard items so let's make a dashboard item that has a bunch of antenatal care a and c stuff right so i searched a and c and you're seeing that i'm seeing all of these dashboard items here you can see the different visualization type that's illustrated here so this one's a bar chart this one's a line chart this one's a stacked column and as i scroll down you can see a pie chart and some maps and even some reports and stuff so let's just add some dashboard items now so i'm going to click insert where they're highlighted and you can see that they're coming they're automatically loading let's add a pie chart let's add a map as well okay and you see that now i have four columns let's add a couple more just you see that i'm just continuing to add and let's add a couple of reports all right okay so you can see i have added some reports these are hyperlinks to some standard reports which are the kind of html reports or text reports i have some dashboard items i have these maps charts tables i can also drag these around so i come to the bottom right corner so if i save my changes then voila that's how easy it is to make a dashboard so you can see them all of my dashboard items i have four four columns only three in this column or in this row excuse me okay so now let's go in and take a quick look at the data visualizer application so i'm going to start with this chart and i'm going to click in the top right corner and you see that when i click in this three dot menu here i have a couple of different options the first one is i can view as table so if i click this view as table then it'll automatically convert the data to a table sometimes it's useful to see the data from multiple perspectives so or just see the actual raw numbers and which is what gives you here instead of a chart so i can also view as a map so here it's going to load as a map well i don't think it actually will load as a map because it it's coming at national level but i can change it back to a chart i can open the data visualizer application which something would do come back to i can show details and interpretations the details and interpretations functionality is essentially the ability to have conversations around the data in dhs2 for the second time i won't do a lot of demonstration on this there's some really good youtube videos available they go over this in a lot of detail but it essentially is a basically a messaging functionality kind of like facebook messenger built into dhs2 that allows the user to have allows multiple users to have a conversation directly around each dashboard item i can also use yep you just read my mind i was immediately about to ask you to show the interpretations uh because we just came from the performance management framework presentation which george presented so i think that's really interesting that uh based on the visualization that you can actually make these comments and then tag somebody who would be a person responsible for conducting an action so you have this link between the visualization and the data and then the corrective action that will fix for example a stock out or a stock a low stock that needs to be replenished so just this thinking between having data and indicators and then making that actionable and connected to a corrective action which will solve that problem yeah absolutely okay so let's go let's go over that in more detail then um so i've clicked on show interpretations um here and you see that below the chart comes this oops sorry let me just get back under comes this box here so first of all i have an area for my description uh and you see below the which there is currently no description written for this uh chart it's usually best practice to write a description um uh the description should communicate what the chart is showing so the different types of indicators um maybe if there's a standard operating procedure or time at which the chart should be used for maybe routine planning cycles or different types of meetings um then you can provide that kind of standard operating procedure here in the description but the point is that it is very much best practice to include a description for each um dashboard item the point being that you don't want the people who are using your dashboard especially if you're making a dashboard that's meant to be shared with many users you do not want those users to have to guess what's being shown on the dashboard um or any and on each uh individual chart as well you want that to be communicated very very clearly so here you have a um no description which is not best practice we can i'll show you how you change that in just a few minutes um under that you have some additional information about the dashboard so owner created last updated on in its sharing settings you also have this bell icon this bell icon is an indication to you um that you will receive a notification whenever you um there are any new interpretations made for this dashboard item so if i click on the bell icon now you'll see that it's become a solid bell um it before that it did have this plus icon if i click on it now then it's indicating that i am subscribed to this dashboard item and that means that i will receive notifications whenever there is a new interpretation for this dashboard item right for this chart so those interpretations are here below so you see interpretations and you see that there has been one interpretation already made by this uh by tom here and tom is asking if the coverage is for this district is below target and if i've identified any bottlenecks so i can reply to tom just like you see kind of an in any kind of social media platform i can do an et symbol and i can start writing tom and you see that i have tom here showed up his username is system so then i've just tagged tom to uh to my comment i could say everything is fine we looked into it okay so and i can even give him a couple a thumbs up or a smiley or a frowny emoji we don't have all of the emojis in the world in dhs to you know dhs to even though it kind of looks like facebook messenger what i'm doing right now is not meant to be a messaging service it's it's meant to be a professional platform to visualize and utilize data so we don't have every emoji in the world here available to you um but uh just a few to help communicate i can also make things bold italics and include hyperlinks if i like all right save interpretation all right so now you see that my interpretation is here where i responded to tom now let's log out because right now i'm logged in as john see that's my username john let's log out as john and log in as tom and see what tom receives in terms of notifications so i will log out and then i'm going to log in now as tom all right so tom has access to the same dashboards or similar dashboards as john did now tom we know tom was just mentioned so where does tom receive the notification that he was mentioned in an interpretation well he can come up here to this messaging application the messaging application is up here in the top right corner um and if i click on the messages then essentially the messaging application dhs to is essentially is basically like an email service built into dhs to so here you're able to send messages between users you're able to receive messages automatic messages from the system as well from dhs to itself uh show you some examples of those so if i go down to system you can see that tom has a lot of unread messages so i'm sorry maybe unfortunately this should have shown up right here i'm not sure why it didn't but you can see that there are messages from um other users here um that that tom has received um so conversations just like you see people you can see here there's also notifications on where um tom has been mentioned in the past so you can see you were mentioned in the following comment um and you can see that here's the comment itself and it actually gives you a link to the um analytic that you were mentioned in there's also other messaging services here so you have the validation so if there's if um you have automatic validation notifications turned on then dhs to can be performing data quality checks running validation of rules um at whatever frequency you've set them to run and pushing you pushing to you automatic notifications whenever there are alerts detected so dhs to is very smart in that way in that you can automate a lot of data quality checks a lot of data checks um and and they can send the notifications directly to the user so here we have a um validation notification and it's saying there's a pop this one is for detecting possible malaria outbreaks so you can see that if i click on that then it's saying here's a number of malaria cases exceed a threshold for this uh in galahund chc for this period we also have a ticketing functionality in dhs too i won't go over it in too much detail but essentially for system administrators it's an ability to um uh kind of monitor any kinds of um issues that are being reported from users at lower level so if someone for example you see here like how do i create analysis on anatom care like you know if a user at lower level has an issue then they can send a ticket notification and then and then the system administrators will be able to follow up on those tickets so you can see i also have things like status priority assignee so um i'm able to to uh kind of monitor the outcome of each one of these messages that come through and then of course i have my system generated so this is coming from like the dhs to server and it's telling again i'm locked in as a system administrator so uh this is dhs to telling the system administrator that some things have not worked or that there's some kind of issue with kind of the server most of you are not system administrators i assume so you probably won't have to worry about the tickets or the system at any point all right i'm going to log back in now as john so the point again with the interpretations is that it allows you to have the conversation directly around the data and i can also show you that you this in the data visualizer application as well so the data visualizer application is the main application that we use to make all of our different charts in dhs too we have other analytics applications as well excuse me we have other analytics applications as well um except um that and we don't unfortunately we don't have time to go through those um but they perform kind of more specialized analytic functionalities for example viewing tracker data or line listing tracker data um but uh for the second time we will just focus so here on the data visualizer application in the maps app as well so here in the data visualizer application um the first thing to point out is in the top left corner is our chart type selector so here i can change my to my various chart types you can see we have quite a lot of different chart types available in dhs too for example i can change this to like a stacked area real quick um quick update so now you see it is represented as a stacked area i can change it to any of these different chart types just varies by one click to point out that we don't have every single chart type in the world available to use our for you used to using like things like power bi or tableau or um maybe a statistical analysis software like r or stata or sass or spsf or something like that um we they have all the chart types in the world available to you in those software we don't include them here one of the reasons that we don't include every chart type is because most chart types are not used most of the time most of the time people are going to use very simple chart types and they should they should use things like column charts or line charts pie charts these are very simple and easy to understand charts we don't want dhs to dashboards to take a phd to interpret them we want dhs to dashboards to be interpretable by any user at any level um so therefore we include very simple chart types here in dhs to um to to use um so i'll change that back to a column chart um and you can see that in dhs to just like i showed you on the dashboard we have our main dimension so here we have data period and org unit um and you can see that each one of these are already selected so they're showing up here in my layout menu my organization unit is in series period is a category and my data isn't filter i could change any one of these if i wanted um what i'm going to do is instead of changing one of my main dimensions i'm going to add a additional dimension so let's change add an additional dimension for a facility type okay and so when i click on the facility type dimension i have these different facility types available to me so chc which is um community health center we have community health posts we have clinics hospitals and mchp which is just maternal i'm not really sure what that facility type is but it doesn't really matter for the second demonstration i'll just go ahead and turn them all on and i'm going to add it to my series and what happened so now you see that i have all of my different health clinics here at the bottom and they're showing up as these these um columns so instead of disaggregating by uh districts which is what i've had been doing i am now disaggregating by facility type so i can see kind of the coverage for the last 12 months by each different facility type all right if i don't like that layout i can do a few things here let's say i want to add my facilities to an additional category dimension and let's move our org units back over to series let's see what this looks like okay so now it looks quite a lot different so we have each month here and then we have a desegregation by each month by each facility type so you can see that i have every single month each facility type um kind of repeating it's hard to see trends or do a lot of analysis here so what i'm going to do is strike my facilities in front of period and let's see what happens now well all right so now we have every month trend by facility type again a little tough to see because i'm using a uh stacked uh or sorry a column chart let's change this to a so we're looking at here coverages let's change this to lines all right that's a little bit easier to understand so you can see each facility type and it's um a and c three coverage over the last 12 months so you can see some facility types are having these big spikes and going all over the place others are pretty standard over the last three months you can see some big data discrepancies with the clinic so i think this is a fairly easy to understand chart type all right now let's change this over to a um let's go back to column click update again let's remove our facility types so what i'm interested in doing is click over here click remove click update all right so what if i wanted to apply a legend to this chart type so i have if i go to my options menu and you see i won't have i won't go over each one of options there's a lot of different options and additional functionality here in the options menu i'm going to go to legend i'm going to go to legend type and i'm going to choose um uh use pre-defined legend or i can say i can select a legend let's select a legend and choose antenatal care coverage show legend key click update all right and so now you see that all of my bars have changed color to reflect the legend so you see i have my antenatal care coverage legend over here on the right side of the screen and each one of my bars now is reflecting that now i can't exactly see which bar is for which health or for which each district so that's a little hard to see here i can say turn each one of them off and so that i'm only looking at one for example and that makes it a little easier to interpret um another very common um chart type at which we use legends is the single value so if i click single value click update here you can see that now i have a i've produced it that that chart into a single value item and that value is going to be the average of antenatal care coverage across the districts that i've selected the six districts across the period that i selected which is the last 12 months okay a lot of people ask about these green dots here on the left side of the menu under the your dimensions essentially what these green dots are indicating is that there are data um these dimensions can be turned on for the data that has been selected so you saw that in just a minute ago when i tuned on the facility type you see facility type has a screen dot that's dh is to just indicating to you that based on the data that you have turned on these additional dimensions could be applied and you can see that not all dimensions apply to all data okay so let's navigate back to our dashboard i'm not going to uh save these changes i'm just going to go ahead and leave this um well actually just for the sake of demonstration if i go to file then i have all of my save options here so i can save save as rename i can translate it to another um um language or i can get a link to it if i wanted to post a link to it and say uh into a message or something like that and i can do that here i am let's just go ahead and save it and then we can see it on the dashboard as well so navigate back to the dashboard and my dashboard items are loading and yep here's the chart that i just saved loading here on the dashboard now let's go take a look at a map i'm going to go down to a map here and say view open in the maps app and so dhs2 we have an application that allows you to build um different kinds of thematic maps in dhs2 um so here you see we have an ipt2 coverage map showing individual um districts i believe i could go down to say individual facility level if i want let's just do that for the sake of demonstration here we have our data type selector so um here i can choose what data i want to view on the map i can change my period so we're looking at this year i'm going to look at work unit i can see right now my level is chieftain let's just go down to individual facility you can see i don't have any filters applied and my style is an automatic legend let's change this to a pre-defined legend and let's go to our immunization coverage legend update all right so now we're looking at individual health facilities each facility is a dot on the map and we're looking at our a different legend so we have our predefined legend from red low to high very which is that light green color and we have invalid so we have anything that's over 120 percent is showing up as a gray dot so you see we actually kind of a lot of invalid um facilities here for this ipt2 coverage all right let's add a couple of additional layers so if i go in the top left corner i can choose my add layer and you see i have lots of different layer types available to me i can add thematic layers so this is these would be layers coming from any kind of um routine data that you have coming in aggregated data would be your thematic layers if you have data coming in for individual events you can add an event layer if you have data coming in from any kind of track or program you want to see individual tracked entities you can add that layer here we also have facilities and org units let's turn on org units because we don't see any borders on this map so it's a little bit confusing let's add our chieftain border and this is at district as well and those layers so you see our borders come now on the map and under this top row in the second row of layers you see i have several additional layers as well i have my population layer my population by age building footprint elevation and precipitation let's turn on the population by age and this population by age data is coming to us through google earth engine so this data is actually produced by world pop it is not census-based population data it is um a they employ different methodologies to get the population what they're doing here with google with a world pop is they are looking at a household enumeration looking at population density and then extrapolating a population number based upon that based upon the the the number of structures in that and the household that are that are known okay so i'm going to go to my groups and let's say we want to look at under five male and female for this population data which is usually our target population for immunization anyways we have different aggregation methods i'm going to turn off the mean excuse me i'm going to go to my org units and let's just do this for just one district because i don't want to load the entire country click add layer it's just going to take a second load right yep here it comes let's turn off our ipt2 coverage because it's a little bit in the way right now so turn that off and let's turn off our district so okay so that makes it a little easier to see so here's our population you know and you can see that we're looking at for the entire district as i as if i click on the population if i click on the district then it shows me the population for under fives for the entire district which is saying it's 125 000 okay one of the new functionalities we have with this which is pretty cool is that we can actually look at the um population by catchment facility so i'm going to go back to my population by age layer and i'm going to go down to choose associated geometries and i'm going to choose catchment area and i'm also going to select instead of district i'm going to select facility so in dhi2 you can have multiple what we call geometries for each org unit so a health facility is represented as a point in nearly every country and you saw that earlier looking at the ipt2 coverage map but we can also associate facilities as shapes which represents their catchment area and of course all facilities have a catchment area and these catchment areas are not being produced in dhi2 they're just being displayed in dhi2 so the catchment areas are actually being produced by um there's various organizations that will work with countries to produce catchment areas the ones that we work very closely with are grid three and um and cross cut um and those both those organizations have the technology available sorry my voice have the technology available to produce these catchment areas and they also um they produce these catchment areas automatically um so they're kind of like machine learning generated and then they're based upon things like um road condition travel time geographic coverage land land coverage population density household location they factor in a lot of different data sources to produce these catchment areas um and then they're able to be imported into dhi2 displayed on a map and of course in cross cut or grid three you're able to edit them so here they're automatically generated but the what's automatically generated may not be accurate to real life so um uh it's usually a good place to start at the very least and then you can go in and um modify it um as is necessary in the cross cut or grid three applications Scott just to uh one second because we have questions coming in through the slack channel so just to confirm that there's no calculation or formula that you need to make within dhs2 this is something that you can activate or deactivate here in this uh as a layer within this map function just to be explicit about that sure yes a good question so you have to have catchment areas um you cannot generate the catchment areas in dhs2 so you have to have those automatically but once you or you have to already have those produced but once you have those then all you have to do is upload those two dhs2 and everything that i'm showing you now will be done automatically there are no additional calculations that are required so everybody should have access to the population layer that i'm showing you now um and once you upload your catchment areas then dhs2 will do the hard work for you which is kind of run these numbers so if i click on this health facility for example you see that within this catching area for this health facility i have 2,422 children under five hopefully that interests the question and so that number that number that 2,422 is being produced automatically by dhs2 based upon where the the borders of the catchment area but then to access that so just to access that there would then need to be a a work together with one of these partners grid three or cross cut in order to ensure that the catchment areas have been made and are available for the country that you're working from that's exactly right yeah and they're really great organizations they're really easy to work with um and you can go to crosscut.io maybe someone can drop that in the chat that's the link to crosscut um and and they will work with you to help make these catchment areas available and they can even upload them help you upload them to dhs2 as well and i'm showing you here these are pretty rough catchment areas you can see how the artificial intelligence has kind of put this jagged border of this catchment area here in real life the catchment area may be different it probably is different and so in the crosscut application you're able to go in and smooth out these borders and kind of drag it move it around to represent what's more accurate you know what true to life but yeah you'll have to work with one of these these other groups crosscut or world or grid three to produce the catchment areas it's not a something that we can do in dhs2 okay thank you scott and i just copied that reply into the the on slack and to say that we have seven minutes left in the presentation and then we can take 15 minutes for question and answer before ending the day so uh if you want to wrap up we can just take a few questions and there is another two questions actually repeated so i can i can go to those once you're ready okay you sure i'm i'm getting pretty close to uh done here so let me just highlight a couple additional things so um let's turn our let's add an additional layer as well so one of the new ones which is really cool is the building footprints layer the building footprints layer is coming through um you can see the source here it's coming from nasa the usgs service the us geological survey service and google earth engine also the jpl caltech jpl caltech is a um is a research group based in the us california studio technology and the nasa jet propulsion laboratory so we are not producing this data ourselves it's coming from from these partner organizations um but they it's a really good data source i will just turn it on just for bow just for sake of demonstration and click add layer and here you see all of my building footprints are coming let's turn off well let's just zoom in here and you can see that as i zoom in these orange areas are becoming improved resolution let's turn down the opacity of our population so that we can see these and as i'm zooming in you can see that i am able to see the let's just turn that off entirely the individual footprint of each building um and you can they're showing up here as orange you can see that the open street maps layer that i have underneath it um this open street maps layer has slightly different building footprints i think i would um if we go to the aerial i think you'll see that it lines up a bit better yeah so you can see that i can see the individual um outline of the individual buildings and if i zoom out i can click on it and i can see my total count of buildings in the district and so this data set is very very good for urban and peri urban places um it's less reliable for and this data is is produced all through machine learning and artificial intelligence um there's not like an end of a real person that's like hovering over the entire world and drawing a little box for each individual building that they see on the map um it's it's automatically generated by the computer um but the the fun thing is that if you're planning outreach campaigns or you are um uh looking at doing some kind of outreach service then you're able to see for example in this if we turn all of these we turn all of these layers back on we're able to see and it's a little bit yeah we're able to see for example where in this facility here that has very low ipt coverage say i'm doing an ipt outreach campaign i'm able to see that this health facility has very low coverage for this catchment population and in this catchment population i'm able to see where the buildings are and where the population is so if i'm planning an outreach campaign i can say target this area over here because i know there's a lot of people there um and there's there's buildings there so that usually means there's there's people living there and um and do my outreach so the point is that you're able to layer multiple um you're able to have multiple layers on a single map so you see in this map we have four layers our footprints our population by age group our our ordinance layer and our coverage layer you're able to kind of visualize all these at one time and help guide uh various programs activities that you that you're planning to do um yeah so i think with that i will reserve the rest the time for any questions that are there let me yeah for the google earth engine this data here that you said generated with the ai machine learning um this is immediately available with a specific version of dhs2 does it also require engaging with a partner to ensure that that data is connected for the country you're working in yeah good question so this building footprints layer is available starting in dhs 2.37 or newer so it's not available in any version of dhs2 that's older than 2.37 um so you'll have to have a relatively recent or newer version of dhs2 to have it once you have that version of dhs2 all you have to do is um is have a subscription to google earth engine which is free available and then you're able to connect that to your dhs2 instance and then you'll have these available um there are some really good tutorials online for this so if you just google dhs2 google earth engine then we have a step by step guide on how to get these layers available to you in your dhs2 instance um it's very very simple it's like just a few clicks to be honest um but it is something that would have to be set up you don't have once you make that connection between google earth engine and your dhs2 instance then you're done you don't have to do any additional um configuration or calculations or anything oh uh these will automatically come these layers here the population population by age building footprints elevation precipitation temperature and land cover these these are all supplied through google earth engine all right uh and i'll post i'm capturing the replies from scott and i'll also post some links to these resources that he's referring to um so this will all be within the the questions channel now scott we do have two questions already from the start of the presentation so from different people so one to explain sharing the dashboards to external users and use cases and then from another user that was from wahid and then from koci bilali is it possible to share the dashboard to non dhs2 users one scenario we have faced previously is having to give access to dashboard uh to the dashboards to stakeholders who don't have dhs2 accounts we ended up with a work around of exporting via apis the data to power bi and creating visuals there then sharing with stakeholders if you can talk about a work on that that would be great sure so the answer to both the questions is currently no it is not available to share um dashboards outside of dhs2 meaning you're not able to share dashboards to people who don't have login credentials to dhs2 um there are there is an application there's a wordpress application that is a connecting to dhs2 so if you have a wordpress website you'd be able to connect that to your dhs2 dashboards um so the the sharing settings so you just have the all users here you don't have that's the highest level of um permission that you can give so that's just all logged in users um there's no external access um we are the the only ways then to share the dashboard outside of dhs2 in any of the current versions of dhs2 is to go to the print option and you can obviously if i go to like print then i can save it as a pdf and then i'll have as a pdf and i can attach it to an email or something like that and that would be how you can share it more externally now we are working on public facing dashboards in dhs2 um it's not as it's it's actually a fairly complicated thing so there's somebody is making some kind of noise and we can make sure you're muted okay um you know making a public dashboard in dhs2 is something that it should not be done um it trivially it should be something that you take a lot of effort in because dhs2 in most countries has a lot of individual patient level data um and what we are currently and you don't want to just make it very easy to share that data um there's a lot of situations where you could have private data you could have a lot of data security breaches um so what we're doing right now is we are working on a way at which you would be able to produce a publicly available dashboard um in a very secure way so that the users of that dashboard are not uh don't have any more access to um to dhs2 than just what they're seeing on the dashboard um you can of course connect to things like power bi and tableau there is a little bit of a security concern there because then you are kind of introducing a back door to dhs2 so you are making those api connections um you probably have embedded some kind of login credentials into your power bi or tableau api connection to to get that data and when you do that a user through the power bi or tableau would be able you know not just any normal user you're talking about someone who's like a computer like developer um someone who's you know a very advanced um user uh might be able to figure out how to get into dhs2 through there um and in which case you uh you have security vulnerability um and so we take security extremely seriously and um that's why we don't have a generic functionality for how to produce external dashboards yet now i will say this is a high priority for us we are working on it we have um we actually are actively working on it we have software developers writing code right now to be able to do it um but it will um and so hopefully in the next like maybe year we'll have publicly available dashboards kind of like exactly what we're doing tableau and power bi but we will be able to do it in a secure way which is the most important consideration for us so it's still to come and the only thing that we really have available right now is the ability to print the dashboard in which case you would be able to save us a pdf and attach it to an email all right that's great thank you scott we had a few other questions also about linking to the uh these layers and i've shared the uh there's a github uh uh recipe or or or guide on how to connect that um i also have a question about the different versions of dhs2 and the features which you explain and then i'm also linking there to the the release notes for each version so in the documentation as scott explained we support the three previous releases so 236 237 238 is clearly listed there so i'll put a link and you can look at each new feature for each release and the difference between them um i think that covers most of the questions i don't see any hands up here in the chat um but i think this is really great to know what possibilities there are to build different visualizations and then to build dashboards to target those for different users at different levels if you're a district level manager or working at a central level if you're at a facility collecting data on a mobile device you can also have these available to you even offline you can make the data that you're collecting also relevant to that same user so i think these are really interesting features uh the interpretations i think are quite interesting in that you can have uh visualizations you can have dashboards and actually comment and contact people who will then be making this corrective action and fixing uh an error and not simply having uh analytics for the purpose of reporting and of course that's important but also making it actionable and to actually improve the processes again linking back to the performance management framework which george presented in the previous section so i think for us um some of the work that we can do and now speaking for myself for george within the lmis team is really connecting these features and making them very um relevant and specific to the lmis use case and really bringing that out and showing how that can support decision making and the work of supply chain management and how these different features can actually help and improve your work so i think if there's no other comments i think can we show pictures on the analytics dashboard um no you're not able to upload photos or images to the dashboards um that's another thing that we've been working on um but as it currently stands no you're just able to you're able to have text items you're able to have hyperlinks you're able to have all of these different charts and maps and tables that you see but you can upload photos yeah and then if it's available on the mobile and of course we said that that it's available it is available on the mobile also offline if you make the dashboards uh offline available offline through the setting which scott showed at the start so that's a short answer on that one but i it's available it's available through the web browser on your cell phone so it's not like there's a separate dhs2 dashboard application you just have to use your phone's web browser so if you're using if you're having an iphone you use safari if you're using android you're probably using google chrome um and or internet explorer something like that so you just go to your dhs2 instance url you log in like you usually do and then the dashboard is automatically optimized to fit on your cell phone screen um and so and so you'll see that and then you can have the exact same functionality that you see that i have here on the web on my um on my laptop and which includes save offline and then i will be able to um and if i do that on my cell phone then i'll be able to take the dashboard offline with me when i lose like mobile data connection or something like that all right that's great um i the questions are now they were pouring in so is it possible to show the district names on the maps yeah sorry i didn't demonstrate that it is possible to show um labels in the maps app all right so if we go to style sorry if we go or units i'm gonna choose labels i need to add a or unit layer of course districts style labels there you see the names are there okay we see the names are still there all right this is great um good i think we're now we've rounded out just about all of the questions yeah the benefits of offline dashboard functionality of course it's having the ability to view your data when you don't have an internet connection if you have a low connectivity site you can still have this data available um one thing we're looking at is of course the um when we get to the point of implementing the transaction based stock tool and then having transactions uh being captured without the use of a physical stock card having the that visualizing the line listing for example offline would be your method of seeing what transactions have been made right from your store uh from your uh from your store so that's that's one of the key points is it possible to add values on the map um yeah so it's a good question uh if i go to my uh style then well first of all um i can change it to a bubble and the the you're not able to turn on uh data values that's again something that actually we are actively working on we're not working on all the questions of our our actually things that we truly are working on um i would tell you honestly if we weren't working on it but it actually is something that we are adding um the the ability to show the values um so you're not if you click on it then you're able to sorry i need to turn that one off if you click on it then you're able to see the value but the value is not just hovering here on the map or if i even if i hover you see the values coming but no you can't statically statically display the value currently okay and then maybe we can take one final uh question as we have two minutes to go and i want to be respectful for to people's time and maybe you have meetings but one final one can we use the functionality of the offline dashboard for all the dashboards or is it only applicable to one dashboard no you can use it for all dashboards so any dashboard that you see on your dhs instance you can make available offline um at any point that you want okay this is great so i think we can end there if there's any more questions feel free to continue to ask them on the slack channel of course um i think one thing that i haven't been asked yet is the word of the day and for today it will be guacamole there it is on the screen you can use that for your attendance i want to say a very big thank you again to scott for sharing with us uh insights on the analytics uh and of course all of the groundwork on the lmis use case and really getting us to the point that we're at now if you have any more questions to scott any more recommendations and ideas feel free to share with us and we'll we'll pass it on to him as well um i don't know scott if you have any final message or final word to the group i think it's been a very active very engaging academy so far lots of questions lots of activity and lots of interest so uh if you have any any final words if not we can also ask alice if you have any final comments before we close off sure i mean i just wanted to say a huge thank you to bruno and george um when i first did this academy it was you know we we didn't have that much to say to be honest dhs2 wasn't uh as organized in the supply chain space and over the last couple years bruno and george have just done an absolutely tremendous job on uh getting dhs2 uh in position to support supply change use cases and not just in kind of a um and an amateur kind of way in a very very professional very organized very structured i mean these guys nowadays are giving who guidance you know they are they are working with dozens of partners many countries and really showing a lot of leadership in the space which is which is incredible to see so um i you know i i felt very safe not being part of this academy because i know it was in very very good hands um this week and allowed me to travel back to the us and visit with my family which has been wonderful and so so it's just uh really been an honor to be able to work with them and uh and i hope that you all are um have the opportunity to continue to work with them because they're really great guys and they just really know this space so very well and uh they have a really great vision for dhs2 and how it's going to how it can be used in supply chain um and as well as health coordinating with so many partners uh it's kind of started to get pretty hectic for them i'm sure okay so that's it for me uh and again if you guys do have any questions you can find us on the community practice for you know general analytics issues and of course i'm sure bruno and george would be very happy to pass things along as they as i said i think i'm going to go back to bed thank you scott a big thank you and we'll be in touch i don't know alice if you have any any also comments i see you you share the attendance and the feedback form the word that is on the screen anything more we need to remember no i think that's it recording would be available in a couple of hours yep and then tomorrow we'll start back up again 10 a.m we'll focus on uh assessment planning implementation we'll have a specific session on tracker uh sorry on um mobile on mobile from uh hi me another uh dhs2 expert who will speak about that and though the implications of implementing uh uh the uh the mobile solution and then we'll have a final wrap-up uh going through some mentee questions and ensuring that all of the key points were mentioned to those who have uh today as their last work week and tomorrow is uh is a weekend we will have this recorded and you can just reach out to us directly for for the word of the day for that so if you're not available tomorrow because it's a weekend where you're at in the country you're at just reach out to us and of course the recording will be available for you to watch on sunday then all right thank you everybody and bye for now see you tomorrow morning thank you bye