 So I'll be talking about how tracker and aggregate data can be linked in the tries to some different approaches. And I'll start with sort of general introduction to different ways different approaches of integrating tracker and aggregate data in the tries. Sort of bit broadly and then focus more on specifically how data can be moved from being tracker data into an aggregate system. So actually exporting data from tracker and saving it as aggregate data values. And in particular, different considerations, I need to have when doing this. Then we have Eric from his Puganda will be talking about how they have approached this issue of linking tracker and aggregate in their TV leprosy tracker implementation. So we'll get that use case presented and towards the end we'll see if we have time for some questions. If not, we'll follow up in the slack channel. So, to begin, I'll go through a few different ways in which you can link tracker and aggregate data in the tries. And I think that when we talk of aggregate data in the tries, we often assume that's the HMIS that it's the aggregate reporting system as such. I think in this context, we have to think of aggregate data also as being a function of the tries to aggregate data model which you can use even if you don't have an HMIS. It's reporting system, even if you're not actually entering aggregate data. So it's sort of a separation between aggregate data as such in the tries to and having an aggregate HMIS reporting system. We're going through three different approaches for linking these two types of data in the tries to briefly before focusing on the third one. So the first one is perhaps a bit obvious. But in the tries to, you have the possibility of opening the analysis tools and making charts that actually combine aggregate data elements, aggregate indicators with tracker tracker data. So whether it's using the data visualizer or and combining different types of indicators there in one table or chart, or if it's using the different event reports event chart apps for producing outputs with aggregate data and combining them with tracker data and combining them with aggregate data in a dashboard. Those are just two examples. So on the slide here, the example on the right, which is from our demo database so it's not real data but showing how the first three rows is aggregate data coming from routine reporting of covid vaccinations. And then we have adverse events following immunization data coming from tracker combined in one report. That's one example. The other example at the bottom shows how you could have, for example, line listing with anonymous data. In this case, cause of death data in a dashboard next to aggregated data. So that's one approach. The advantage here is that it's easy to set up you just use the analysis tools. It works well if you have data from tracker and aggregate that are sort of complimentary and it makes sense to bring them up together. You have the possibility of including sort of any level of granularity you want from the tracker side. The disadvantage here is one that due to the way the data in each IC structured, the way that sort of dimensionality in the data is handled in aggregate. So there are limitations to what you can do. And of course, a big assumption here is that you have the same data in the same database. And as Marcus talked about this morning, that is not compatible with the way that works in tracker. So you wouldn't be able to have one table combining program indicators and aggregate indicators with the age disaggregation as a separate dimension, for example, so there are limitations to what you can do. But this morning that's not always the case, not always recommended. A second way of combining tracker and aggregate data is through aggregate indicators. So in the aggregate side of the choice, the aggregate indicators, which we define essentially setting up a calculation of numerator and denominator. Those are illustrated on the right there. Those can draw in data both from the aggregate side and the tracker side. So you could have, for example, a case where you have immunization registry and tracker, where you can get the count of children, given a busy G dose in this example on the slide. And then you can pull in your population denominators from the aggregate side as one example. You can also do this to add up service data say for example you have a certain type of facilities using tracker for maternal health. And you have other types of facilities reporting aggregate. If this isn't the same details to instance, you can add those two numbers up. So we have ANC first visits coming from tracker in some facilities and see first visits coming from aggregate in other facilities, adding up to total number of ANC first visits. So in some scenarios, this may be useful to add up sort of complimentary data. Again, this is relatively easy to set up within the choice to using the built-in functionality. It can help potentially hide some of the complexity to the end users since they get an aggregate indicator which gives them the total they want without having to look for data in different data sources. Some of the same disadvantages apply here as for the first approach. There are limitations again on these aggregations having separate data dimensions for things like age breakdowns and sex. It can be difficult to manage. If you're using this to add up service data, you need to be very sure that you don't have any overlaps. So you have same facility reporting through tracker and aggregate, then you will be double counting. So you need to be very careful if you're doing this in that way. And again, it requires that the tracker and aggregate data is in the same details to instance. So in one of the example indicators with the BCG coverage, that may be as simple as having some aggregate denominators in your tracker instance. But in other cases, it may be more complex. Then the last approach, which is probably what many think of when they think of linking tracker and aggregate data values, is to actually take the tracker data and aggregate it and save it as aggregate data values. So then this way you can use the tracker data to feed into the aggregate reporting system, the HMIS. And there are different ways of doing this. It can be done sort of ad hoc on a need spaces we need to do. You need to do some particular analysis, or you can set it up as an automated process for every week, every month, you take your tracker data. And you save it as aggregate data values. What is a bit different with this approach is that not all the functionality is built into DJI's to itself, so you need somehow to use tools outside of the choice to do to achieve this. So whether that's manually querying the DJI to API to get the data out and importing it again. There are several applications in the DJI to community which helps you do parts of this data transfer. It's also possible to set up the predictors functionality of DJI's to. But there are some issues there with the periodicity, etc. So doing this is sort of two main use cases one is, as you see here, you have your tracker data, you want to use that to populate the aggregate reporting form so you want to automate your HMS reporting. The other main use case is that you want to leverage some of the analysis functionality of DJI's to which is applicable to aggregate data but not to track your data. And coming back to this example of the dimensionality you see on the left here you have lists of new relapsed TB cases with HN6 disaggregations. These are program indicators coming from a TB tracker. But it's just a long list of figures. If you want to be able to produce a chart like the one on the right, where you have different series for the six where you have different categories in the chart for the HDS aggregations. And you have the possibility to pivot this around. Then you would have to turn those program indicators into aggregate data values first. So to summarize the advantages here, as I said you have the possibility of analyzing data with additional dimensionality, you can use it to automate some of the aggregate reporting. But in addition to this, there are also less sort of obvious advantages. One is that it can help with the performance. So there's been some examples now that in large tracker systems, some of the program indicators are very heavy for the server to process. So in particular cases where you sort of you're adding cumulatively lots of tracked entities. So one example is you want to know the number of HIV patients currently on treatment you need to then make a query of all patients that's in the database. Or if you have, if you want to look at people vaccinated, and you need to look at all tracked entities up to the current date to look at how many has been vaccinated. The next day you need to look at all those same figures as well as the ones added in the last 24 hours. So this can be very taxing on the server to make these queries. But if you're able to save some of these values as aggregate data values so that you don't have to redo the calculation every time you carry the program indicator. It can essentially reduce the load on the server. This approach is also helpful if you have sort of a hybrid or gradual implementation of tracker, which is often the case that you have, for example, you roll out tracker in region by region. So most of the country is using aggregate reporting, and then parts of the country is still as introduced tracker, you want to have all that information in one place. So being able to take your tracker data from the regions using tracker and aggregating it up so that it becomes compatible with what the rest of the country is doing is very useful. Of course there are disadvantages compared to the other approaches this is more complicated to configure and maintain. So you need to have some sort of tool or script outside of the choice to actually do the extraction and import of the tracker data. You need to have a mapping between your tracker and your aggregate variables that needs to be maintained every time there is a change. If you have two separate instances of the choice to you need to make sure that the same organization units are present in both. So you suddenly introduce potentially a whole issue of having a master facility list or synchronizing your organization units. So keep in mind here that these approaches could be seen as complementary, and you might have to use several of them for different programs or different purposes, maybe at different stages in implementation. And to sort of address the information needs of different types of users. It's very important to keep in mind here how the choice to is actually deployed. In particular when you have both tracker and aggregate instances. So, coming back to what Marcus was talking about this morning. You may have within the country you may have. You only have tracker you don't have any aggregate instances. So you have tracker only as on the left. You may have tracker and aggregate reporting systems combined in one the choice to instance, or you may have separate instances for tracker and aggregate. In all these scenarios, there may be need a need for linking tracker and aggregate data in different ways. So if you have an instance with only tracker data and you don't have any parallel aggregate systems, you may still want to link in aggregate data. You may have tracker instance, whether it's to get the targets for planning, whether it's to get denominators to be able to calculate coverage indicators, whether it's generating aggregate data. In order to do additional analysis, you can only do in aggregate or for efficiency reasons, as I mentioned that aggregate data analytics. This is more efficient less taxing on the server. If you have a combined tracker and aggregate system. Many of the same reasons for combining the two applies here. This of course gives you more possibilities of integrating the two data sources in a single database. Let's see if it becomes relevant to look into automating HMI reporting to replace manual aggregation of individual level data. And again, much of the same applies if you have separate tracker and aggregate systems. In this case, actually moving data from tracker into aggregate may also be necessary if you want to do sort of cross program analytics. So that's another sort of separate issue here. You may have a tracker instance where you have perhaps TB case reporting. Then you have all your HIV data is being reported. So aggregate monthly forms, and if you want to look at TB and HIV together, you need to bring the TB data into the aggregate instance to be able to do that kind of cross program analytics. So this is an attempt to sort of summarize depending on how your tracker and your aggregate systems are set up, which are the approaches are feasible. What does it take. So if you have an instance with only tracker to be able to show tracker and aggregate together in dashboards to be able to use. Combine data and aggregate indicators you need to somehow get aggregate data into tracker first so it's not out of the box. It may still be relevant to use the third approach of actually creating aggregate data from your tracker data to be able to do analysis to improve performance. If you have everything combined in one instance, then all of these approaches are sort of potentially useful out of the box. If you have separate aggregate and tracker instances, the same applies to if you as if you have only tracker you need to somehow get your aggregate data into the tracker instance if you want to use the combine aggregate indicators, or if you want to show tracker and aggregate data together in charts and dashboards. So, I think after this introduction, we'll do a short mentor meteor. If Alice can help with the part one. Sorry, yes. Let me share my screen. Next question. Oh, what happened. Sorry. Next, next question. So we'll focus now on this third approach and looking at both how this is done and also some of the things you need to consider when you're doing this. So this is now. In many ways, the use case we're focusing on the most having your aggregate data, turning it into aggregate data value. Which, in many cases will feed into the HMIS reporting, but not always. So in terms of, I won't go into the full technical details on this but just showing a bit on the process of doing this technically. There are alternatives to this. Using different kinds of apps and middleware to transform the data, perhaps do the mapping between tracker and aggregate outside of the tries to what I'm sort of showing the conceptualization of here is using built in features of the tries to as much as possible. And using the API endpoints of the tries to do sort of export directly program indicator values as aggregate data values. So that as much as possible. All this process happens inside of the tries to pull. Here we have our tracker data stored in the tries to get this data transformed into aggregate data. We define program indicators, for example, program indicators that count. How many children were enrolled in the immunization registry program that had those of BCG and were zero to 11 months, for example. So this defines based on the tracker data, the aggregate outputs that we want. And then query the tries to the device to API for this data, we specify what periods do we want to want this monthly do we want it weekly do you want it quarterly. And for what organization units. And then the tries to will give us what is called a data value set. Which is a format, typically Jason or XML file with the actual data values aggregate data values exported from program indicators. This format is what the tries to uses when importing aggregate data values. So once we have this data exported from the program indicators. We can import it again into the tries as aggregate data. And this could be in the same details to instance, or different details to instance. Doesn't matter. But as a big but here is that you need to have a way of identifying both the data so there needs to be a code on the program indicators and the data element so that you have a way of mapping them. And the last organization units. So, the key components in this tracker to aggregate data migration are the program indicators that define, define the aggregate values that you want out. It's the aggregate data elements, which is the target where you want to save your aggregate data values. So the API that allows this data to be exported and important. And all this data needs to have codes or identifiers that they can be mapped to each other. And of course, this means that if as part of your implementation, you want to be able to generate aggregate data for aggregate reporting you need to make sure that the way that you set up your tracker program. Makes it possible to generate these aggregate data values, which I think is something also Brian touched upon in the previous presentation that you need to make sure that you're aligned. What you can get out of your program with the outputs you need to do aggregate reporting. As you can see, I will speed up a bit now to make sure that we have time for Eric as well. There are a few different alternatives. When you actually do this extraction of tracker data and import of aggregate data, if you have only one instance. Then of course you would do the move the data within that instance. This is where you have separate tracker and aggregate instances. There are essentially two options. You could either extract your data from the tracker instance and import it directly into the aggregate instance. Or you could first export and import the data in the tracker instance, meaning that you actually have your aggregate data values in the tracker instance as well. And then move the aggregate values from your tracker instance to your aggregate instance. So you're doing this in a two as a two step process. There are some advantages to this. In particular, that this means that any users you have in the tracker instance will be able to use this. These aggregate values with the benefits of the analysis tools for tracker. It makes the actual first part of the process of generating the aggregate data bit easier since you don't have to worry about organization units not being the same in the two instances. The downside being that you introduce a second step where you do have to take organization units into account. So this is a very having this as a separate slide just to emphasize how important this is. In some implementations where the number of organization units is not too big and the changes aren't that frequent. I think it's doable to manage this more or less manually to have procedures for making sure that if you add the facility in one instance, it's also added to the other. And if there is an issue when you're moving data you get the notification. But if you have tens of thousands of facilities you need some sort of interoperability setup that helps you manage organization units across instances. So I'll finally look at some of the things you need to consider. If you're doing this data integration so I think there are lots of things here you need to make a decision on think through how you want to do. But there is not necessarily correct answer. So what I'm doing here is for the most part outlining all the different things you need to consider in order to link aggregate and tracker systems. I'm just reemphasizing this point that you may want to produce tracker data both to be able to automate your HMIs reporting, but also to be able to do additional analysis. In terms of actually migrating the data, examples of things you need to consider is how often do you actually want to do this? Should this be done every day? Should it be done at the end of every reporting period? Should you then, when you're migrating data, do you do that for the last six months to make sure that any previous data that has been updated on the tracker side is always up to date in aggregate side? So how do you align this transfer process with procedures in HMIs? For example, in many countries, aggregate data is verified and then locked after a certain period of time. What if your tracker system, if data capture is retroactive, what if it doesn't meet those deadlines? How do you deal with that? One set of questions you need to ask yourselves. So much related, how do you deal with errors in the data? If you have data that's already been migrated and then someone does corrections on the tracker side, how do you expect that to make it into the aggregate? What if you identify data quality issues in your aggregate data? How do you go back and deal with that? Or do you correct them, adjust them separately? How do you deal with completeness and timeliness of reporting, which is key data quality metrics in HMIs, but isn't really concepts in the tracker side where you're dealing with individuals? How do you ensure that users have access to the data they need and no access to the data they shouldn't have access to? If you have separate instances, how do you make sure that users have access to both? Who are the owners of the data coming from tracker submitted to HMIS? Perhaps the tracker data is only subset of an aggregate form and who is responsible for what is being reported? So there are a number of issues related to access and ownership. And finally, related to sort of quality and ownership. You will often need to have some sort of transition period where you're doing parallel reporting, comparing the numbers, looking at sort of discrepancies. Because I think the chances that you will have identical numbers coming from your tracker and manually aggregated figures is very low. And then you need to discuss why are they different? Where does the source of the problem lay? And then use that information to decide when it's time to stop with the parallel reporting. Finally, as I mentioned, some of this needs to be managed outside of the GIS2. So as you're planning and implementation, if you want to do this, you need to also take consider that you need people who can manage these things, who can set up the integration or the migration and maintain these over time. In these metadata packages, which have been presented previously. Some of these mappings between tracker and aggregate are included in the packages. So you still need to develop the actual solution for moving data, but the mapping between the tracker side and aggregate side is included in these metadata packages. So I think the second meant different now because I see we don't have time and leave the word to Eric to present a real world case from Uganda. All right. I think it's good afternoon everyone. It's good to be on the call. My name is Eric, I work with his Uganda and I support project implementation, as well as phis to implementation. I will share with you this use case for an instance called the national national TV and the process. Electronic case but surveillance system. Which from which of course we, we have been implementing the project for about close to a year now. And we have gone through a couple of experiences and we'll be sharing those. Okay, from this presentation will focus on the linking aspect, which is what all have has been sharing. And we'll just skip through a couple of other things. I wanted to quickly give sort of a background, a quick background on where the project started objectives approach the models. And then we can lastly discuss a bit on the, the linkages, but I was also thinking, we could maybe take a look at some of the, of the system screenshots that's what we were using. We currently have as a country. And this is Minister of Health, we currently have a national HMIS system that captures aggregate information from within the Minister of Health, they are departments that do some kind of parallel data collection and reporting at the national level for the interests really that we'll be discussing so the national TV and the process program is is one of those kinds of department that are unique and they're kind of they want the data in different ways. So they needed real time integral and visual level data for purposes of reporting, but also the ability to track patients progress along the continuum of care. This is really like real time, focusing on other areas like transfers, appointments, tracing contacts and treatment outcomes. TB is a very, very unique and interesting disease. So this really necessitated the implementation of this instance. Lastly, quicker, better tailored response for for the program to really respond to challenges through data systems. This, this project is supported through the national TV and the process program, which is based in the Minister of Health. And of course, so many other partners are very interested in this data so they've been part of the whole process. The objectives were to improve TB and the process treatment outcomes of individual patients through one case based monitoring. Like I said also earlier patient transfers across both accredited and non accredited treatment units, but also improve surveillance across and of course for decision makers to get access to data and and control the spread of TB. We, we've gone through the normal process of documentation and having people participate in the in the requirements collection process. And this really helped us to understand what is required. For, for this instance to work, because again, there was a contradiction between having this case based surveillance tool alongside existing electronic medical record systems. Remember, both collecting individual patient data. And so we had to draw a line there. So we had quite a long time of discussion, picking data requirements and then judgmenting. Like Olaf has been presenting hours was really a bit more of a combined approach to have both tracker and aggregating one system. Again, with the aim of being able to pull this data and send it to the national HMIS that currently has aggregate so the vision really was that for TB accredited and non accredited sites. They would not have to manually fill in the TB summary report that goes to the HMIS on a quarterly basis. So we chose to have a track and aggregate combined approach so that once we have the tracker data, we're able to aggregate this and then send it to the HMIS system. Additionally, the other point really was that we needed to migrate some data from existing parallel system, one of those national TB best system was collecting data on MDR TB, MDR is much drug resistant TB. So we needed once there was this national aggregate system, we needed to pull all this data that had been reported in that system and bring it in one central reporting platform. Then there was also the issue of integration, where we needed to have modules within the surveillance system that would pull data from other existing system that facilities like lab, like other EMRs and so on. And then of course overall support training and capacity building of different teams. So the data model was really more of a simple one. For TB we focused on enrollment treatment, lab adverse events and then treatment outcomes. So that's really generally I think I want to go into the details of that. On the integration and linkages side, we you notice that this basically brings together different data. So I talked about the MDR instance that had legacy data. On this side we have the Minister of Health DHIS2 or HMIS, but from the facility side we had, we have different EMRs that have currently been reporting data in a parallel way. And we have lab systems, we have contact tracing systems, and then we had direct data entry based on the facility registers, patient cards, and so on. But also data reported using mobile applications through SMS notifications to clients and other providers. And then of course the analysis. So it was important that we consider a more of a flexible approach that could address bringing together the needs both at the facility or the district and also at the national level. So on the specific on the linkage part and within the system, we looked of course we had the programs designed in the tracker capture data. And like I said, we have the aggregate side which had both the quarterly aggregate report from the HMIS and also another monthly aggregate report also from the HMIS. So once we receive data from on the tracker side, we build program indicators to pull data through a web API and then populate this data on the aggregate side. Again, so here it's important because there's a lot of mapping required. So we had to design the program indicators to look exactly similar with similar names as the data elements so that it's much, much easy when you are mapping the metadata. Then additionally, you his Uganda in the past has developed this data import wizard app, which was presented also in the 2019 conference 2018. That allows program indicator to data set mapping and then supports automated import and export of data through the web API. Yeah, so the program indicators again saved automatically and data is uploaded and the mapping can be kept within. I will quickly give some screenshots on how this works. Payload is also downloaded for verification. Again, where mapping is very is involved, you need to make sure that the program indicators are sending the correct data to the correct data element. So we had a lot of verification to make sure that every every field on the aggregate side is linked to the correct program indicator and this was of course a bit manual but very helpful. Still on the linkages, yeah, we we we've had to put together and develop a web service to support receiving data from other EMRs. Like I mentioned, we have some system based on open MRS and we have also a lab system. There's another one called clinic master and all these are within the facilities that we are implementing the system. So we've developed again a web service capable of receiving a fire message. Converting it into DHIS specific format and then receive the data on the on the tracker side. So this is this again is one of the key components that probably we'll discuss another time on the data migration. Like I mentioned, there was a system that was pulling and picking data on drive resistant TB. And this system, we use the data import wizard to link directly and then import the data automatically into into the tracker. So some quick lessons learned linking data requires creating mapping program indicators to data elements and this is one of the options, of course, we chose all I've mentioned there quite many others including using the predictors, but we felt this was quite straightforward and much easier to manage. Of course, it means that if you have a matrix that has age and sex disaggregations, you have to sort of create a program indicator for each of the fields that you want to push data to. That we did minimize on on that manual work you by use of a spreadsheet and then creating the program indicators. Sort of using the spreadsheet to pull through and then manage the different categories of the data elements. We've also noticed that sometimes there's been some data mismatch. Where by when you run a line list in the report in the event report. And you will likely going to get a different figure on the aggregate side. So we, we still trying to find out what's happening, but in most cases it's related to analytics, because analytics do not run every hour. So users will want to see their data on the aggregate side. When they do that, that's sometimes it does create some data mismatch because you have tracker data and and then you have analytics on the aggregate side. We have also seen that the linkage of tracker to aggregate minimizes the duplication of efforts. I mentioned earlier that we are pushing the aggregate report to the HMIS, but this for facilities that are able to report. They don't have to again manually go and enter this report in the national HMIS system. I also made a comment earlier on this point that most users are more comfortable with aggregate analytic features. And this is because of course they have been using the HMIS for aggregate reporting. And so when you bring the tracker side, there are a few more steps to get a report. And so it becomes a bit tricky. And so you won't really get so many people using event reports to do the analysis or even the event visualizer. At least for what we have seen in the facilities that we have been supporting. And to be precise, we have about 1,700 facilities in the country that are reporting on TB. People don't do reporting using the event report and event visualizer. They prefer to use the pivot table and the data visualizer and so on. So yeah, again, the last point here is that data has to be kept in sync or is I think Olaf already mentioned this. So I wanted to quickly do a small short demo. So this is the data entry layout. We have four programs. Yeah. So for the application that we've been using to do the input from program indicators to the report, you can see here that we normally upload directly. So specify the URL, put the username and password. And then here we will have to specify that we are moving data from indicators to a data set. And we can have data mode for different options where we can move indicators to data sets. We can do data elements that are set. But in this case, we also use program indicators to that I said so once you select that option. And we're able to sort of the app pulls both data on organization on open it for example, and you'll automatically map. The facilities technically these are the same of units in the same system so we wouldn't expect to find anything that's not mapped. But if it was from another system, you would have an option to see what is mapped versus what is unmapped. And what's not mapped will basically list and we are moving again here will be moving data. At facility level so each facility should be able to map to one another. The second screenshot again shows the program indicators and the data elements. So here on this site where we have the source, you find that these are the actual program indicators we've created. And in the destination, we have the data elements on the aggregate side. So normally, the reason we did that was for the app to automatically pick and match each each program indicator with the data element, either by ID code name, or we can even match them using a manual approach. If it is anywhere where it doesn't match, you can manually search and then it will bring back a list of bring back a list of the available options on the data element side and we can pick those and map them. Lastly, we are able still also to map to pick the periods for which we want the data to move. Again, because our aggregate data set is quarterly, so the periods here will automatically show on a quarterly basis. And then we can also determine the determine the records that we want to push per page. And ultimately, once data goes in, once data has been pushed, we'll have, this is the national data set, and this is how it looks like it will be populated and for each different cases and categories, the data will be aggregated and pushed in automatically. I think that's mostly what I wanted to share on this over and probably we can have some Q&A on this. Thank you so much. Thank you very much. That's really interesting. We had at least one question from the chat about the name of the app. Can you remind us? Thank you so much. The, so the one I'm actually showing is called, we called it a wizard, but it's part of the data import, the data import wizard, which you can actually find. Sorry, let me see. If you go to the apps and you look for data import wizard, you should be able to find it. So this is the app. Are you still able to see my screen? Yes. So yeah, so this is the app developed by Uganda. You'll find it there. You can take the latest version and it should be able to do most of this. The current stable version supports 2.9 and above. Great.