 So really when we're trying to implement this in practice, you know and what are some of the considerations that we have to make that are a bit different than when we're thinking about this and kind of a More less dynamic environment, right? So, you know, there's a couple things to think about here, you know obviously scaling this up and ensuring you have the resources to do so within your country in context, but also, you know scaling this up in terms of how many people are going to be accessing the system, how many Entities will you register in your program? And then how does that impact, you know what you do in terms of training performance on the system that kind of infrastructure required to get this all to work? So this session is kind of a summary session and what we're also going to do we're going to hear from history wonka Priyanka and Pramo who've been part of this as well as Pamod Scaled up of a very a significantly large tracker system in response to COVID-19 and Just they're going to walk you through some of the struggles that they've had to overcome You know the realities of making this work in a real setting when you're dealing with large amounts of you know Millions of records essentially, right? And that it can be challenging in its own way so There's actually a whole academy on this so we're just talking about this for about an hour and a half today But there's a lot of different considerations So I've linked to some different material and I'll go over that as well Okay, so when we're looking at tracker implementations You know we compare these kind of two concepts the coverage of the the program that we're implementing and I have health examples here And we've used health examples throughout That's just more because that's my background is what I'm comfortable with but there are other ways you can use DHS to For education is a good one That's a kind of thriving kind of new use case new community of people using DHS to to track You know children through their their schooling so You know I have health here, but you can think about this, you know more generally to other concepts as well All right, and you also compare you know the coverage in terms of you know How how many people you're trying to cover with the program with the complexity of the system you're implementing right and what are the trade-offs there So you know you can think about this in terms of you know Maybe how many patients are interacting with the program or how many facilities you want to have Entering data in real time, you know and this can all all impact your coverage in some way right and I have a couple examples here For example of this is some examples from Ghana Looking at you know the mch program where coverage would be you know rather high You don't rather high coverage, but the complexity of the program itself You know it's pretty low right in comparison to some of the other programs things like HIV which are higher on the complexity scale You know there's a lot going on with HIV care and delivery This and you know they're enrolled for for a very long time, you know from the time They're they're initially diagnosed and receiving treatment for a very long time And but you know conversely there's typically not as many people who have HIV in the system, right? So there is a trade-off there between the coverage and complexity So we try to think about this in a way so that when we scale up You know we kind of work through this to understand, you know how things might work Maybe we need to phase things out a little bit more Maybe we need more resources for a very dedicated period of time For example the HIV program because it's quite complex You might need significant resources to get that up and running at the beginning right But then once you do so it might be you know easier to maintain But especially at the beginning and maybe throughout the run You need those resources in place to kind of manage that So we've developed this kind of model to think about to think about this a little bit more Right so when we think about the the kind of positives of the implementation tracker at scale Right so you know data use is a big one right we're entering individual information now Right and we're able to do a lot with that individual information And we've been looking at program indicators as aggregates But you would also have quite a bit of information on each individual patient We've been examining briefly for example at due dates You know sending reminders as an example for people to come to their appointment And this has a real life effect you know on people's ability to access the system Right gives them reminders to actually access real life services It's not just an electronic or a concept within a database anymore It's something that we can use to kind of affect our service delivery There are obvious benefits for patients and healthcare workers Because you're able to kind of have that more kind of detailed history on service And you know patients will have a more detailed history about you know what they What services they've received as well This is all kind of registered and tied together to their unique patient record Right there's also various operational clinical and performance improvements Right so we mentioned reminders a lot of you are asking about you know Things like overdue events so if they didn't attend their appointment Right you can then pull up a list of those who didn't come to receive a specific service Can call them you can send out a reminder to them You know you can you can use your resources in order to kind of follow up with those individuals And it helps a bit more than if you just have aggregate data And then obviously if you're you know your own policy and research Both in your setting and wider in the region and globally You know this contributes to this obviously because it's kind of a Quite a treasure trove of epidemiological data right and you can use this to make informed Decisions in terms of of your service delivery and the settings that you're working with Right but there are negatives right to this right not everything is just working efficiently all the time right You've seen now that the configuration can be quite complex and we've To be honest only kind of scratch the surface of what you can do right So a lot of you had other questions about other things that either you're trying to think through Or that you think might be relevant in your own settings right and there's a number of ways to implement those Either through a custom kind of solution or through adding on to some of the concepts We've discussed that are just a bit more advanced Then we had time to fit in right but this can become quite complex over time And this means you might only have a small amount of people who really understand how to manage this right and you need to kind of Take care of those resources, especially if you have a lot of health care programs or Education programs or whatever it is that you're managing right You also have a high number of users typically right because as you scale the system up More and more people will be entering data directly into dhi's too And you need a team to be able to manage all those users provide support to those users, etc Right you have a large amount of data as well And the data is now you know much more sensitive right because you're storing That this personal identifiable information So you need policies procedures, you know and practices real life practices that safeguard this information Right, it can't just be a theoretical discussion anymore Right, but now you're you're really kind of taking these people's information And you really have to be kind of mindful of what you're doing with that information right So what we've done is try to develop this kind of House, okay, basically that is laying the foundation for what we would like to see At least some of these components implemented In place before you kind of work towards scaling up a tracker implementation, right So start from the very very bottom from the top and and you know Some of you are implementing tracker programs and you might want to revisit this a little bit to see You know if we don't have these things in place What might we do to kind of think through some of these challenges in our own settings, right So so obviously, you know at the bottom we have this kind of just overall buy-in You know for people you don't want to do something that's maybe against you know senior ministers You know ways of thinking or you know senior or kind of development partners that are helping you to support this It doesn't work if you're kind of going to kind of fight all the time just to make kind of simple things happen, right And then of course funding right that's a big thing and it's not just a one-time cost, right So, you know dhi is to itself. It's a it's a free system, right? It was open source system can download the software and set it up There's a lot that goes into supporting that and and you guys probably realize that already by now, right But it's not just a one-time cost, right? You don't just set it up and leave it You set it up. There are changes that occur. There's users to support There's all kinds of training that needs to occur both once at the beginning and then over time as well as You know as refresher and new concepts are introduced changes are made, etc Right, so you need to make sure that at least a strategy is in place Even if all the all that funding is not secured yet But a strategy is in place to kind of think through that a little bit more, right Legislacies legislation and policies is another big one And I think in many places that we work these are not necessarily in place And in the interim you can use kind of global best practices You know look at other legislation that relates to this especially with privacy and standards, right Sometimes especially we you know, unfortunately have people who who will ignore this to a large extent and just kind of say well It's it's such in such country. This is not an important thing to think about but really you need to think through this Quite a bit of detail, right? And if there's nothing in place You kind of need to get that that started, right? You need to initiate this conversation with those people who would be involved In such a decision-making process to implement this To safeguard these people's information Um capacity and competence is another one, right? We've been training you and we've spent 10 days together and Like I said, you know, we've you know, there's a lot of different concepts We could still talk about right you could still spend a lot more time, right? So it's continuous capacity and competence and not just for people like yourselves who will be administering systems But you know, then you have the wider array of users that will be entering data analyzing the data, right? And that's something you want to really build up over time as well Infrastructure is a big one, especially as it relates to tracker and we'll talk about this more Especially as we get into the performance issues and then we have these these blocks You know, once those are in place we can think about some other aspects of what we're going to do So it support is a big one right making sure we have continuous support training and supervision You can see how it links to these kind of foundational Pieces the scale and the coverage of the system. So how far do you want to go down in your health system? You want to cover all the facilities, for example in your health system? And if you do then, you know, what realistically, you know, how do you plan that out? Also feeding into the HMIS right so tracker systems look at this individual data But you know, you would probably have an HMIS system That's been collecting aggregate data for some time and in some ways Especially when you're first starting out could be a more robust more accurate data source the quality of that information Due to all the time that's been spent propping that up, you know, might be to a higher standard And you also have to think through in terms of how you're going to collect this data Right, are you going to be using web-based devices, Android a mix of both? And if so, there are specific considerations, you know, based on which approach you decide upon if it's a hybrid approach For example, there are both design considerations and other scaling and training considerations as well Real time versus secondary data entry, that's another one we'll talk about Okay, just meaning, you know, do they enter this data at the point of kind of one of services received? Or is it kind of after the fact and there are different implications there Hosting and security, right? This is a big one, right? At least as it relates to tracker and closely relates to this infrastructure block And the whole design and configuration process So we've really only been talking about this this here this one little block during our training, right? So it's it's it's a small block in the grander scheme of things. It's an important block nonetheless But there are other aspects to consider All right, so real time versus second data secondary data entry the first one I wanted to highlight Okay, so this means, you know, are we entering data at the same time that the person is receiving the service essentially, right? So let's say a mother comes in and receives her antenatal care, right? Either during that service or right after that service is provided. We would go and update that mother's record, right? That's one way to enter data Another is secondary data entry rate. Maybe we fill it out in a paper-based register And then someone later goes and finds the mother's record and updates that record Now this the secondary entry is less resource intensive much less resource intensive, right? And it's not going to preclude you from providing any service either, right? If you're doing real-time entry and you are saying to everyone you must enter the data at the same time the service is delivered This system needs to be available, you know 100 of the time or as close to that as possible, right? If you're doing secondary entry and the server goes off for 30 minutes, it's not a big deal, right? Because people are just entering data retrospectively Also, you need people at that point of entry then to enter all that data, right? Whereas if you're doing secondary entry, you could maybe consolidate this You can have one person who enters all the data regardless of when the service was provided So not one approach is right or wrong necessarily. It's just thinking through, you know How you would deliver that service and and how you would kind of get to that point So feeding into the hMIS and we'll talk about this more I think in a moment But you know, we've seen that tracker data aggregates automatically And it's true that it relieves the data collectors of this extra reporting burden But there are some extra considerations in very large-scale systems and by very large scale I mean like systems with millions of records, okay? And I'm going to talk about this a little bit more as to why sometimes this becomes a problem And we'll see some of this from the history long a team as well Okay, I'm andrevers web So some Of you have actually mentioned Sorry for that everyone dog just went a little crazy. Okay. Um, so Android versus web so some of you have mentioned that you're using Android already in your implementation, right? And Android allows you to use data offline, right? But then you need additional, you know, we'll talk about capacity too You need additional considerations than when you're using those devices because you know, we skipped over a lot You can see actually all the baseline stuff that you would do to make Android work You know, that's what we've covered but in on top of that there's additional considerations such as, you know How the person interacts with the program? That's why all those color and icon options were available, for example For a lot of the configuration because you might want to, you know, consider other aspects But also you have that to manage those devices, right? Let's say you pick up thousands of Android devices. How do you manage those? How do you make sure they're functional and that you can support people To make sure that they can use them, right? You will have lots of users, right? If you're entering data at that level as well So there's there's all these additional kind of considerations to think about depending on which way you go. Okay I'm hosting in security. This is another room, right? So we talked about some of the security measures we could implement through various granular controls, right? User rules, user groups, sharing Program access levels, right? These are some of the things the software can do to help you, right? But at the same time, you know, the database itself You know, there are some considerations around how this is hosted and how people can access that information, right? You know at bare minimum, you know, some of these Basic server requirements should be met and we'll talk a little bit about them Not too much specific detail, but we'll provide you some links to have a look at this Right, but you know making sure that this is not overlooked I think it is quite critical, right? And this links to some of the stuff on legislation and policies as well in terms of You know how people decide That others can access this personal identifiable information in the country In the design and configuration process, right? So When we did this, we've really been working in isolation, right? And that's difficult, right? I mean, one it's an artificial course So we're using use cases that are semi-realistic, you know, they're based on real old use cases But we've given you a lot of the kind of information about, you know, how the workflow would look like How things might be configured in reality How the form might look like and how to interpret some of the fields But you're usually not given that information, right? You usually have to kind of work with others In order to identify that and this process as I mentioned It can take a little bit longer, you know Actually this back and forth Can take a bit longer than the actual, you know going inside a dhi's to and kind of adding everything to the to the system, right? Because you're really working then with subject matter experts who really know what they're talking about in terms of how this stuff is delivered, right? Because the whole idea is to support, you know, users in their work. It's not just to kind of create an artificial Oh, this is kind of cool and I want to get this done But you really want to kind of help the workflow of people who are on the ground, right? So it's important to kind of understand this more and do some studying to really work through the workflow in your setting So that artificial process we described at the beginning when we filled in our templates and looked at the programs You know, that's something that you should really kind of emphasize at the beginning when you're implementing anything new And then even when you're looking at revising, you know, what you're doing, right? So scale, right? So Once again, there's there's some considerations here in terms of the number of devices the number of users And then the support you're providing, right? So the lower you go down in your system the higher your scale is essentially, right? So I'm going down to all the schools in my in my country or all the facilities in my All the health facilities in my country, right? Then the scale of this can be quite enormous, right? depending on the size of your country, of course, but Even a modestly sized country with a population of, you know, over five or six million you'll you'll have a lot To deal with in such a scenario, right? And you really need to make sure, you know, this can't be run by one guy, you know sitting in the office You know, you need a team you need to train others to make sure they can help with this as well And then training, right? So training is a big part of this, right? So we've done training now for at the admin level, but all the end user and analysis training, you know That ends up becoming more important in a way, right? You'll always only have a small group of people, right? Not everyone in the country is going to be Concerned with configuration, right? It's just really a small subset of people But when you're scaling up, when you're dealing with large systems, you know You're going to have to find a way to regularly train people and have good content for them to refer to to utilize in their jobs So they can, you know, follow these processes that you've kind of set up for them, right? And especially, you know, you might have differences, different training for android, different training for web You might have training on, you know, the analysis of the data That's maybe for another subset of users, right? And just kind of working this all out all these different pieces of the capacity And even yourselves, right? You might have to kind of learn a bit more about security protocols, for example Learn a bit more about various aspects of the configuration. It's an ongoing process, right? It's not just a one-time thing where you do the training once and then, you know, it's done You kind of kind of continuously Are in this process and that should be implemented into any kind of work plans or anything like that When you're thinking about, you know, funding as we mentioned at the beginning You really have to think about, you know, this recurring cost, right? Not a one-time cost Which is the same for many of the aspects we've described, okay? Support, okay? This is a big part of implementing tracker, right? Once again, you need a team that can support People on the ground, all right? There might be, you know, you guys have run into some issues already I'm having some, I try to register a person and it's giving me this weird error And I don't know how to interpret it, right? This can happen in the field quite often And you need people to kind of help those individuals who are kind of stuck Right and not able to work through that on their on their own, right? And they need a quick mechanism to do so This is obviously more Intense if you think about our real-time data entry situation, right? Because then someone will be sitting there maybe with a patient or client and it's not working And and and if they have no one to talk to, right? It's going to be really really frustrating for them For both them and the patient because it will disrupt the service that they're receiving So so you need to kind of think through this kind of link together and not in isolation Right, okay, so that this kind of house I've actually uh, what I've done So there's a whole bunch of information on this. So if you go to the implementation consideration section I've linked to the tracker implementation documentation Which talks about all these aspects in a bit more detail, okay? Um There's quite a bit of information on on all of this in terms of you know, assessing if you're ready For using tracker, right? And and there's actually an assessment tool. I believe that's linked Look in here And then you know talking about how we would kind of plan For implementation itself, right? So I would encourage you if you're kind of either in the midst of one or kind of thinking about doing this To have a closer look at some of this information to really You know take some of these considerations into account in terms of how you might not implement this in practice And it covers these aspects that we've been discussing right and hosting training Android versus web, um real-time versus secondary data entry, et cetera, et cetera, okay All right, so we've talked about kind of these these foundational building blocks And now I want to talk about some of the challenges actually implementing them And I'll hand it over the Sri Lanka team momentarily to really describe some specific examples So just a bit of a background here and and where some of this information is coming from All right, so last year we saw kind of a really rapid expansion Of tracker implementations as a result of coven 19. So looking at coven 19 surveillance and coven 19 immunization and coven 19 immunization in particular Has some very challenging characteristics that we not yet dealt with before But that could potentially be extended to other use cases in the future And we just are trying to kind of get ahead of this and help people kind of make this work as best we can though As the Sri Lanka team will report There's still a long way to go in some cases right and they've had to come up with some very interesting solutions I think to get around this So now we're having DHS two systems come up where we're basically capturing the entire population, right? That's the idea. They might be registering you in advance So when you go to immunize someone, they're already there and you can find somebody Or they might be just going through one by one and you know registering millions of people At these sites literally depending on your population and if you're vaccinating maybe everyone over five for example Everyone would go in that system Over five or that would be at least your goal So you can end up with millions of records, right? And this also results in things like new sites Many many new users and a high number of users accessing the system at the same time, right? So your performance requirements are all of a sudden becoming very large. Okay Because you're having kind of this entire population that's wanting to access the system And register those individuals in your system This means that there's also high requirements for real-time data, right? A lot of people are basically saying, you know as you receive or as you provide the vaccination Update their record, right? Because there's millions of records if you kind of leave it for secondary data entry Sometimes in some cases, right? It gets lost, right? Because the backlog is just too much Right if you right if you immunize the thousand people a day or something in your facility You know and you leave that for a week, you know, is someone going to sit and kind of Enter seven thousand records on their own. It's it's really kind of difficult, right to think about There's also a significant impact of slow or down systems, right? And I kind of alluded to this earlier, right? But if your system is real slow and you're trying to kind of enter this data in real time Or if your system is down, right? This can have an impact on service. You can't enter the data, right? So then it's going to go in the backlog maybe and then you're going to have trouble catching up Because of the large amount of records that you're trying to enter We're also seeing a lot of high usage throughout the day And this is all all hours of the day, right? Because people are entering this data, you know And you know immunization services are still occurring on the weekends as well People are then looking at the data in the evening preparing Releases for their updates for various individuals in the system to tell, you know Give updates on what the coverage is today how many doses were administered etc etc So we're seeing a high amount of usage in both the entry as well as the analysis side Where people are pulling up this information a lot more frequently than we had seen before Right and then this importance of this linked and longitudinal data as well is more important, right? Especially in this case you could have linked programs like adverse events following immunization, for example Right and all this needs to be linked over a quite a significant long period of time We've been in this pandemic for you know, whatever it's been two years plus And and people are still continuing to roll out this vaccination program and it's still growing in scale So as I'm referring to this, you know, a lot of these findings are a result of COVID-19 vaccination programs But you could think about this if you're doing any kind of other type of mass population based activity, right? It'd be the same kind of challenges that that we've run into as a result of COVID-19 Okay, so our current findings on kind of Helping a bit with the performance are based on kind of six potential mitigation areas So server requirements and monitoring, right? And we talked about this is kind of one of the baseline components of a tracker system in terms of infrastructure So it was even complicated when we were dealing with programs that were much smaller in scale, right? An hiv program might only have a couple thousand people registered a year or even less Right And even then we had you know to think about the infrastructure Pretty thoroughly and now we're talking about maybe millions of records, right? And and then it becomes even more challenging There's also things we can do on the analytics side to mitigate the kind of performance needs of this There are also kind of specific tracker solutions and Android solutions For people who are using Android. There are various implementation strategies We can also think about And then this other areas where we can use tracker to aggregate data to mitigate some of the impact on the server load itself All right, so I'm just going to talk about each one real quick Just to give a general overview So this is more for kind of a server administrator person thinking about this But really what we're recommending is, you know, make sure you have Kind of a more appropriate software environment on your server And then kind of a more recent to dh is to version Okay, so if you're using an older dh is to version that's on an old patch If you're using an old kind of database environment And you know upgrade this as soon as possible basically if you're experiencing performance issues Server monitoring is also a good idea. So there are very server monitoring tools for database administrators Or system administrators, however you refer to them that they can basically monitor the performance and kind of see Pain points are times when there are problems, right? What are people trying to access what is going on on the server shuts down for no reason? You know things like this kind of help identify some of those pieces of information The server is also appropriately sized and when I'm talking about this I really mean if you're dealing with millions, you know millions of records if you're Still working on tracker implementations that are smaller in scale You wouldn't need such a stacked kind of environment Such a performance performance environment But really what I'm talking about is at scale dealing with millions of records here, right? So we have some minimum recommendations on the server and you can see that some of the recommendations, you know Look pretty out there, but when you're dealing with some of these These large systems that it's really beneficial to have some of this and this is kind of our minimum spec, I guess in some of these cases, right? Also, if you can separate the database and the Postgresql and have dedicated servers for them So there's some stuff I've linked to kind of a longer Discussion on this for people who are kind of interested or want to share this with their server administrators Perhaps to understand a bit more on some of these recommendations. All right On the analytics side. So one thing we've been working through, you know the past Yesterday, we've been talking about program indicators and how they work And yeah, they're they're a wonderful tool But what we've realized is if you're doing this on systems with millions of records This does cause some performance issues Okay, and this is because you're going row by row, you know tract entity by tract entity Maybe millions of times performing a calculation of some kind And this causes, you know, some some challenges. All right, and we'll talk about what we might do to mitigate this, right? so an alternative here is You know Sending those program indicators to the to the aggregate data model, right? So I make my count of whatever it is or my sum, you know, and I send it to an aggregate data element. Okay And there's a document that I've linked to That describes this process in a little bit more detail and we'll also discuss the kind of pros and cons about this approach in a moment And especially what we don't want to do is have a page when a person first logs in With several program indicators that are having to count or assess millions of tract entities, right? So let's say I have five million people registered in my system I have a program indicator that goes through and has to evaluate five billion records when the person logs in and it's the first thing They see it's it's going to really slow down their system, okay so As an alternative we in these large systems, you know, you can kind of do some things for the dashboard to mitigate this problem Right, so you can for example set up a text or only an information landing page That excludes some of these tracker analytics to minimize impact, right? Or you can limit the dashboards based on kind of specific program indicators And only to those users and user groups who need them for A specific purpose, right? And this is kind of against our normal approach, right? Where we just, you know, we want the data to be shared as much as possible And the data still does, you know, people still deserve to see the data So we have to come up with ways To share that with them, but in the interim, right, if you're kind of really struggling and you see at scale It's not working, you know, there are some measures you can kind of put in place Just as you kind of work through troubleshooting Well, how do we actually get this to work for everybody because in the end that is the goal, right? Even though, you know, here I have not for general data entry users This is kind of an initial consideration, right? Eventually, you know It's it's nice if the data entries can see some data, right? And not just kind of and have no feedback based on what they're entering Okay, some other things we can do is enable analytics cache, right? So that way, once it's downloaded the first time, it won't have to do it again If every time they're logging in and downloading the data, it is going to be difficult, okay? Another challenge is this continuous analytics option, right? So we're analytics is just running all the time So you're getting latest updates It's better to kind of choose a period of time to run your analytics Maybe, you know, in the middle of the night at like 2am When people are not accessing the system That way, if it takes a bit of time, maybe analytics takes a couple hours to run If you're in a large system, it won't really interfere with operations and bog down the system Because if you're running it continuously and people are entering data at the same time And you're dealing with millions of records, then, you know, this is a recipe for a slow system Another thing we can do for tracker, right? So when we're configuring our program, if you remember, we had this box Display front page list, okay? If you're having performance issues, you could also disable this It'll help a lot That's a very heavy load on the server displaying that front page list So you could disable this and just force people to search For the tracked entity they're looking for rather than seeing it on the front page Especially if that front page is not used that much, this can help quite a bit Okay There's some stuff you can do on the database then as well You can apply some custom database indexes for frequently searched attributes You know, things like first name, last name, sex, maybe Or ones that might be commonly used But also what helps is if you have unique IDs These are very fast and very low burden compared to Custom attributes that you've made So if you have an ID that's unique in the system, like a unique ID, a passport number of Some kind of structural service number that you use for national identification And searching by that is very helpful if you have something like that available Also, if you're making system-generated attributes And try not to use this random type of syntax, right? Sequential, there's this syntax called sequential And this is a little bit less load compared to random Especially if you're generating this many, many thousands of times in a day All right, on Android, so those of you who are working with Android Or thinking about working through Android Okay, there's a couple things you should try to get the administrators Kind of familiar with before you embark down that path, right? So there's an Android settings app that has various ways to sync data and improve performance This is kind of a global app, right? So it allows you from the DHIs to instance to control how all the Android devices interacting with the system Will sync their metadata, their data, and this can significantly improve performance, right? If you tweak these settings and understand the implications of these settings For people using these devices, it can really help them to access and enter their data Okay You should also create specific configuration for users That will access or be using Android, right? So if they have a data entry configuration or some type of user that's configured to access it via web You should really think about having a specific user or role or setup Sharing whatever it might be for that Android user, right? To really kind of describe what they can do Okay, because the way they interact with the system is going to be quite different, right? Also a way to kind of handle all the Android devices, right? So there's this concept of mobile device management Which allows you to manage a mass amount of devices centrally, right? And that's one way to do it But if there are other strategies There are other strategies that you can think of, you know Even kind of employing district managers maybe to ensure, you know Some some scenario where they're they're responsible for the Android devices Whatever it might be right in implementation strategy But there needs to be a way where you can control You know which version of the Android app that people are using Right? And then, you know, management of the device itself You know if they have problems with whatever on the device, you know They can't turn it on anymore or they delete the app or something for whatever reason Or maybe you don't allow them to delete the app at all, right? That's another thing you can do is just kind of control What people can do on the device you give to them So setting those things up in place before you kind of embark on that deployment to help Is very helpful, right? There are some implementation strategies We can also think about, right? So another another thing we can do is try to in parallel So you have your tracker program but in parallel Have an aggregate configuration available for reporting, right? This can be used routinely or as a backup In case something happens with the tracker system, right? At least you'll have the numbers for the day, right? Which might be very vital for reporting purposes If there's something wrong Obviously you want to stop the tracker system from falling over But in the case this happens, right? This is just set up as a mitigation measure And if you are using this tracker to aggregate kind of concept Then it's useful to have in place anyway So you can send the data to the tracker kind of Or sorry to the aggregate configuration, right? And another helpful thing you can do in some cases It's not for all scenarios But you can refer to these metadata packages we have available, right? So there are standard configurations that provide you with All kinds of reference metadata dashboards analytics That you can have a look at to think about how you might design You know some features Doesn't mean you have to copy them necessarily But it'll give you a lot of reference in terms of how you might build something out To make it work a little bit well Because you'll see in for example all our tracker packages Or a number of them they have for example A ton of program indicators that could be mapped To aggregate data elements as an example So you could push the data over, right? And that's the kind of last thing I wanted to discuss, right? So just to re-emphasize So why shouldn't we use program indicators When we're counting millions of TES, right? They're very heavy calculations And if I'm logging in or multiple people are logging in And it's performing these calculations row by row, right? Millions of times It can really slow the system down, right? So this whole idea of tracker Moving tracker data to the aggregate data model It allows us to map our program indicators Right? To our aggregate data elements So we still have a system somewhere That's using program indicators To calculate this information, right? But then whether or not that's done on the main server As people are entering data or there's a copy And that's being sent subsequently back as aggregate data Or something else, some other mechanism, right? Completely dependent on how you set things up But this is part of this infrastructure consideration In terms of how you get this working, okay? And then you would periodically generate Very aggregate data values from the program indicators, right? So rather than generate the data in real time, right? Every minute, if they see an update on the dashboard Maybe they only see an update every six hours Every 12 hours, okay? A more realistic timeframe To allow the system to process this And we realize this is not ideal, right? Might not always meet your requirements But it's better to have the system operational Than have it kind of buckle under the pressure Of having millions of records to deal with, right? And then dashboard itself If you are having this tracker to aggregate model in place Then you can build the dashboards Based on the aggregate data elements And that way more people can have access to them, right? You don't have to kind of control this so much If it's just aggregate data, you know Then you can have lots of users see this information And it won't really cause so many issues with your system Compared to if every user has access to a dashboard With very heavy program indicators So they're still needed But then you just kind of map them to aggregate data elements And move the data over And it's not a completely smooth process, admittedly There is a document that I've linked previously That talks about it more Okay, so please refer There's a full presentation on this Okay, in Moodle And there was a webinar on this too So I've linked to the presentation Okay, that talks about those concepts that I described In more detail And I've linked to the recording of the webinar as well Okay, so this is a full presentation on these concepts Okay, it talks about all these various aspects That I described in much more detail And provides different resources for this So on the server monitoring, I just had one slide But there's discussion here on all the different tools That are in place, for example Descriptions of what you see in a typical output If you're setting up some type of performance monitoring Et cetera, et cetera, right So if you have a large-scale system That you're either working with or thinking about implementing It might be very useful to review this presentation As well as the webinar itself, the link is here Okay, so there's an activity in the chat Which I haven't been monitoring I see other instructors have been trying to answer your questions Okay, I'll have a look now But I know this is a bit of a shift in gears But it's just to kind of think through What we're working with And what I want to do now is have Priyanka And Pramal kind of work through Some of the challenges they've had Because it's very interesting to understand The effect this has in real life Right, so Pramal, I guess Pramal or Priyanka I'll hand it over to you guys And we can continue with the session Okay, thank you very much, Suresh So everybody had a very detailed session On implementation consideration So let me share these implementation considerations In a practical setup, practical scenario Let me share my screen Okay, so I'm just going to explain Demonstrate what Surajit have mentioned As implementation considerations Using a real world scenario Which is the immunization tracker we used in Sri Lanka My representation is based on COVID-19 immunization tracker Which used to capture individual level COVID-19 vaccination data in Sri Lanka So obviously that is using DTIS with tracker So over these nine days I think you collected a lot of information Knowledge on tracker configuration and implementation by now But having a look at this practical scenario I think it will help you to consolidate Your knowledge and experiences So COVID-19 immunization tracker in Sri Lanka So obviously the program COVID-19 vaccination You can see here on the top of the COVID-19 vaccination We have so far at this time We have five program stages First dose, second dose, third dose Obviously we very selectively started with fourth dose So we had to implement this fourth dose program stage also And this is the final one is AFI So that is for adverse events following immunization So basically when we are talking about the scaling of the system So you should have it very clearly mentioned That when it comes to COVID-19 related systems Systems are going to get very heavy In terms of track identity attributes So in our systems, the COVID-19 tracker Track identity instances registered so far Is nearly 20 million With 31 million of events recorded Organization unit count is more than 16,000 And we have 3,500 nearly users With usually more than 500 current users at a new type So currently the database lies on the disk Is about 240 GB You can imagine how heavy this system is So from now onwards I am just mentioning the steps we have taken To manage this much of heavy system To ensure the smooth functioning of the system So as Surajit mentioned Our server configurations Now we did a lot of changes to our servers To match with the requirement So software versions we are using at the moment Is JDK 11, Post-Race 12, 13th version And DHI is 2.35 So we are continuously monitoring our servers With both Glowroot 10 Uning The both Glowroot 10 Uning We haven't stored in our servers And we are using those software to monitor our servers Continuously by our server management team And actually we have implemented 2-Ti architecture in our servers We have a separate app server And we have a separate DB server Database server with 24 CPU cores And 128 GB RAM and 500 GB SSD hard disk The app server is a different one with 32 CPU cores 64 GB RAM and 100 GB SSD So we always ensure fast and stable internet connection And internal network connectivity So we don't rely on one single internet Supplier and we have multiple suppliers With high speed supply And we are having as I mentioned a dedicated server For Post SQL database And a database server was optimized Now you can find the database Optimizing procedures in our DHS to documentation So this is just a screenshot of Glowroot monitoring tool So by looking at the Glowroot monitoring tool You can have a lot of information of the server And identify issues And we also have in Installed continuous resource utilization Monitoring with net data Net data with alerts enabled environment So net data allow us to monitor continuously About the resource utilization in the server Coming to the tracker design So we have disabled display front page list As Shuravit mentioned And we use on very minimal number of parameters For search As you can see in the screen We are using just three parameters For search in our front page And max Applying custom database indexes For frequent search Track entity activity also possible In our context We haven't used that But we have minimized the number of parameters Which are being used for search The maximum number of track entity instance Serves to return in a search That can be configured In when you are configuring Or when you are setting up your tracker program So we have limited it to a lesser number Like 5, 10, I can do A few other tracker design points We have implemented in our system So generating unique ID In our system We are generating a unique ID through the system So we are using a sequential pattern Instead of random pattern It could be performance wise Having some issues Custom modifications of tracker capture We did some sort of custom modifications To the default tracker capture app In view of improving the functionality Tracker capture updates event data values individually Though that is the normal thing That is how it works usually in default setup This can cause database rollable looking Because of database layer rollable looking This can cause some performance issues In the database server So to overcome this We have in our instance We have done a very custom change Not to send event data one by one But we have changed the complete button But you can see in an event To save and complete button And we are sending all data at once to the server So that the hit to the servers are limited So that the rollable looks are limited So when we are working with a high number of concurrent users This is very important And we gained a very great improvement in functionality With this custom development And only a few widgets on the tracker dashboards Are visible in our system Because you can see there are a lot of widgets You can see in the tracker dashboard But in our vaccination tracker instance We have allowed only very essential widgets So basically the profile widget And the data entry widget We have selected tabular data entry Because that is much more easier for the data entry users And coming to the dashboard So we have configured a dashboard Which is publicly shared with just with text So if the landing dashboard consists of analytical element Which needs some database connections and analysis So once a large number of users are looking into the system So that the performance will be limited and compromised To overcome that we have We have designed a plain dashboard Without any analytical elements Just with some text So this is the landing dashboard Which does not need to run sort of analysis in the server So this is the So once a lot of large number of users are logging into the system The problem to the server is much less in this context Again about the dashboard design So dashboards now we have designed dashboards for different levels Now we are not sharing the same dashboard for each level But we have designed separate dashboards for separate levels So for an example institution level Separate dashboard regional level Separate dashboard provincial national level Separate dashboards And we populate those dashboards only with very essential items And share those dashboard with the only the respective Users who need this data in a very frequent manner For decision making We usually don't know Most of our users are data entry users For data entry accounts For data entry only users We are not sharing dashboards So they are usually don't see on dashboards or see data Because they are main objective is to enter data to the system And our dashboards We have confined our dashboards To only two frequently used analytics items only So without having a large number of unnecessary Analytic items on dashboards We have confined to essential number of very needed Analytical element in the dashboards And also we have We are not including this sort of heavy visualizations From on the dashboards For an example maps with lower level load units If you are drawing a map with very low level load units So it takes a lot of time to load And thus of the burden to the service height Event reports more than with more than 100 rows Again will give a burden to the server And visualization requesting lengthy periods Like if you are visualizing the entire vaccination information From the beginning of the vaccination Which is now more than one year So that will take a lot of time and resources And visualization with enrolment type program indicators Usually when you are calculating dropout rates This happens So it takes a significantly large time to calculate And also the burden to the server is very high So we have not included that sort of heavy objects In our dashboards as a practice And we have minimized the use of program indicators in the dashboards So we have formulated or we have created Favorite items and stored When needed only the user can load them and see the favorite items But we have to great extent we have not included This dashboard items with program indicators Because some program indicators may compromise The performance of the system So as additional help So weekly basis we are preparing a detailed analysis Of the COVID-19 vaccination status of the country So and transfer it to PowerPoint presentation And we are distributing all over the country to the decision makers So that they need not to again analyze Again and again analyze those things Because they receive this weekly report every week And they can if they need some further analysis They can do that without doing that basic analysis So analytic wise now we have Identified we all know that running analysis analytics Is the most resource intensive activity in the system In Sri Lanka COVID-19 vaccination system The currently takes around six hours to run analysis So we have scheduled analysis at midnight And up to six AM it is running But it doesn't have that much of impact on For the system users because system users usually Begin the use of the system after seven AM We have turned off continuous analytics Because continuous analytics the system We observed the system was very slow And as Shuraj is very clearly mentioned The converted tracker data to aggregate data model Is a very good approach to overcome some of these performance issues In our system it is in focus Now I have mentioned that the process of converting Tracker data to aggregate data model In the following illustration So you have tracker data in your program In your instance on your tracker instance And with program indicators You are calculating the necessary values And through a script you populate some Aggregate data elements or indicators Through those program indicators And you are populating your dashboard Using those aggregate data elements or indicators So this will reduce the burden Of the tracker data model to the system Because analytics using aggregate data Is much less resource intensive than the tracker data Let me come to the caching So we have enabled caching in the DHIS to configuration You all know that you can configure caching In system settings under analytics So we have enabled cache for one day until six AM And we have set the cache ability to private To avoid engine expense Through user management also we have Gained a significant amount of control over the system So we have created several user groups On depending on their level of access and activity needed The activities needed to perform And user roles to assign system access and functionality Which are necessary to perform the activities And you all know to share in settings Metadata and dashboard was shared among the respective user groups So for each level we are having data entry and data view users We are sharing data entry accounts only with the data interpersonal And data view accounts we are sharing with the decision makers Or those who are required to visualize data for several purposes And user training and user support also is a very important part This is not a technical thing in the system But if you don't address these user training and user support So your implementation will be in trouble Especially in COVID-19 vaccine systems Like COVID-19 vaccine systems You don't have enough time The time available is very limited to perform this user training When it comes to national implementation As a national scale up So you need to address to a larger community At a very limited time period So our strategy was we developed one page guide So what is as you can see one page guide The very simple user guides With a lot of illustrations rather than text Which can be understood by even a person Who is not well literated in the computer So we have formulated one guide for each important action So for the data entry person Then there's one guide for a data entry person They can just have a look at that guide and continue data entry And also we have developed several implementation guides And standard operative procedures And also we have developed very short videos To demonstrate the activities in the system So through these modalities We could overcome this challenge of user training In a very short period all over the country And actually we didn't have sort of face-to-face training But with these strategies We could overcome those user training challenges And use of support We also with the Ministry of Health We have established a proper mechanism To user to reach for the help Through the telephone we have introduced through the email Even through the fax or through some social media channels They can reach us We have three tire architecture to support the users The frontline or the help desk is a tire one They may give solutions to most of the problems of users And if they are not able to provide a solution Then the problem will be passed to the tire two Where most of the emoji or users are there And they can attend to most of the advanced issues And resolve it And finally the most technically oriented issues Now we are getting access to Sri Lanka and Australia So this tire two architecture is working fine And we support users of proper User support mechanism It's a must or very important thing To succeed your implementation So I think that's what I have to share with you In terms of Sri Lankan experience Thank you very much for listening to me So Suraj, it was to you Thank you Thank you, Prangar for sharing It's useful for you guys to see You guys, you know, the scale of that system Is really a bit great 20 million tracked entity instances and growing I might add So, you know, they've dealt with some of these And you can see a lot of the guidance Is actually reflected upon their experience Right, because they've gone through this And kind of dealt with some of these things Before we had any idea that this would be a problem Right, so they're really kind of leading the charge To make a lot of this happen Because of the scale of what they're trying to achieve And you can imagine, you know, if you're trying To do something like this in Nigeria or Indonesia You know, your population sizes are even bigger Right, so, you know Just thinking about these considerations Can be important in such scenarios If you're dealing with smaller countries, you know Like smaller island countries, for example Then there's a different set of considerations Right, due to geography But in this case of these large systems A lot of these challenges are shared Right, so I hope you were able to get something out of that And thinking about all the different things We've been configuring over the past couple days You know, how you might apply some of this To mitigate, you know, some of these challenges That, you know, Priyanka has been describing