 If we can. Okay. If there's no one there. Yeah. Yeah. I think it's going to be a popular one. That would that be crazy. People can join online. We have 50 people online already. So we're, we're, we have quite a crowd here. So this one might go slightly over four o'clock. Two or three o'clock. Fantastic idea. So we're just going to, we're going to do it in here. One moment for people online. Okay, sorry for the, sorry for the, the mix up there, guys. We, we have a very large crowd for this presentation. So I'm going to go ahead and start presenting and the other group is getting set up in the other room to watch it live from there. And we will just get started. So it seems like this is a popular crowd, even though most of the people are no longer in the room here with me. But we're going to talk a bit about how to build a public dashboard and still sleep well at night, which someone told me today was a clickbait title. And I think that's probably true. But it's also important because there are, it's, it's tempting to build public portals. There are a lot of public portals out there, public access to data. But there are a lot of considerations that need to be taken into account when doing that. So we'll talk about a few of those. There are many more, but there are a few of the, the really critical ones and security is obviously of, of big importance and performance is another one. And making sure that you're doing this in a way that doesn't compromise or expose access to your operational DHS to instance or your operational system, whatever that happens to be. And so, just to specify what we're talking about here, we're talking about public dashboards the first of these, but I'll talk about a couple of these others as well because they are often kind of lumped into the same bucket. So public dashboards, the way that we're talking about them today are read only. They're accessible without any authentication on the public internet. And there might be some cases where you have a public dashboard that is only visible to a limited set of people on an intranet or maybe behind a password or something like that. The general idea of a public dashboard is that it's, it's for the public, it's for everyone right and COVID dashboards were very popular in 2020, 2021, where every country was publishing all of their data for the number of COVID cases every day. And everyone in the in the public was was refreshing every five minutes right to see what the latest numbers were. So that's what we're talking about here today. And this is talking, targeting the general public so potentially millions and millions of people that's important to keep in mind. And, and they're not, it's not targeting experts so it's not, it's not an operational system it's not a statistical system it's not something where you want, or you need to expose really sophisticated tools or want people to be trained how to use. And this is as opposed to a couple other use cases or things that are often bundled into the same concept. And when we when we talk about public dashboards or public portals. Sometimes people assume that we're talking about some other things. And those are just going to cover them quickly here because we're not going to talk about them today. Those are data exiltration which there's a couple ways to think about this one of this is push analysis, which is supported in DHS to a little bit of an antiquated support for that in DHS to but there are other ways to get data out of the system. And in particular to get data into the hands of to basically where people are anyway so a common request here that we won't touch on today because there's a lot of nuance and challenges with it is you have a minister who wants to be able to see their DHS to data. It's not data that you want to open to the public but the minister doesn't want to remember their password, for example, that comes up quite a lot. So that's another example of trying to get in a get data into the hands of people that should have access to that data without basically removing some barriers from from them seeing that we're not going to talk about that today because that's not the purpose of public dashboards. And then the third one here is individual access to data so if you are a tracked entity in DHS to if you are a mother and you want to be able to see your basically your own history of postnatal visits or something or you want to see the vaccination record of your child or you want to see the school record of your child or you want to see your own COVID history and COVID vaccination certificates those types of things, or even enter some data yourself like entering data into a nutrition program so you record your own meals instead of needing to remember it and then tell it to a nurse who records it into the system later. That's another thing we're not going to talk about here today but is often confused here. So that is a whole nother can of worms a whole nother thing that we could talk about another time. So that's the main thing we're going to talk about. Everybody take a picture of this and scan that please and if you know of any public dashboards. Use this QR code to enter a simple form and at the end of this will just kind of list them out. These are public dashboards right so they should be open to the public. So what we're trying to do is just get a list from the hundreds some odd people who are in this session. If you know of public dashboards particularly those that are have DHS to as one of their data sources and go ahead and fill out that form and and to start us off actually we're going to give a quick an example of a public dashboard that was developed by the his Vietnam team for a while, and I'm going to turn, yeah turn it over to my colleagues from his Vietnam to do that presentation quickly. Okay. Hello. Hi. Good afternoon everyone. So my name is Sam. I work for his Vietnam but based in law, supporting Minister of Health to do the implementation of DHS to allow. So, for me, I will talk a little bit about the, how we make the loud dashboard. So I will just go to the overview the goals the aims and the use case then my colleague. Yeah, we will talk a bit about the technical part. Okay, so. Okay, so I'm like Austin mentioned most of the goals of the public dashboard is for us is also more or less the same. So we want this dashboard or not we want but the minister or the high level people, they need to access to data. So they don't, we already gave them the username password but sometimes they cannot remember they have problem accessing and most of the time they don't use laptops like us every day so they rely mostly on their phones and their tablets. So we come up with this idea so to help them to get access to data. So first of all, this dashboard that I'm presenting here is mainly about for decision makers for the high level people to be able to access from their phone. So the requirement is quite simple so that they don't need to log in so they can just go to their website, get the data immediately. And the second one is also, since they don't have to remember also where to get the data so we want to have one site. And then go through all the data that they need that can change the, the, the web just to see other programs. So the public dashboard will get data from multiple right now we get data from multiple instances but in the near future next phase we might need to get data from different sources. And as I mentioned, so it should be a mobile first so they can just view from their phone or tablets no need to use the laptop and then almost real time, right now we realize on the analytics so at least they get one day behind data. And most importantly, the graph the chart in the dashboard to tell stories to them immediately, so they don't have to ask, ask or ask each program like what does it mean, at least they can see and they can tell what happened with, with the data or with the program. So that's the goals. So in for the approach. So once we know what we need to, to have or what we need to build for, for the ministry. So these are the approach we do so, because in law we have different frameworks we have different indicators targets, we based on this one and most of the indicators on the dashboard are derived from here. So in, in last one thing that important is the national assembly they have 10 indicators for embers to monitor and also to report to back to national health assembly national assembly. So this year 1011 indicators we try to capture that in the public dashboard so that at least they get some update every month or every quarter. Apart from that also you will see SDG and priority programs, although we cannot get all the data here but at least we get some, some idea or keep them some ideas of how each indicator that are doing. Once we have the list of indicators or the areas that we want to show, we also work with each program, we want to work with our partners like the ratio, UNICEF, PSI, and also program in the ministry like MCH, EPI to define what can be shown and what it will be useful for the high level or decision maker to see. And the last one is also about making this as a living dashboard so we, we just want this dashboard to, to the high level people last last month, but we will continue to improve. Now we get the dashboard to many of the program and the users then we will get feedback and we will add more and try to improve. And now last one is just about the next step. So with this dashboard now they can easily get access to data. We will also use this tool as the one of the way to introduce or to encourage the data use culture in the country now that they don't have to use the user name and password to login they can get data at the hands and then it will promote the data use more. And we also plan maybe this to use the dashboard for the MND work so that they can also get the high level view of the program here. And I also mentioned now the data are from DH2 instances in the future maybe we want data from more sources for different purposes and at least they can have some information like a primate or maybe the for flood monitoring etc. And lastly that Austin mentioned but for us right now we are we mentioned is the public dashboard but not nearly the public. So it's mainly for the high level people to access without username and password, but at some point in the future maybe we would like to have this dashboard to be accessible by the public and at least the public can get some ideas of the country or at least like I said maybe from fun people that we want to know something about the areas of the communities. That's the overview. So we like just to show a couple of screenshots of the dashboard. So as you can see over there on the corner we will have different list of the dashboard that they can select and go through. But if you want to see the public and you can also access this website and hopefully you can also give us some feedback to improve. But as you can see now the graph the chart might be different from what you see in this to be as I said the main goal is also to tell the stories at least for the top row you can see the high level people when they see at least they know compared to last year the same period how we are doing for example the SBA compared to this time last year you see that is decreased etc. Yeah, and the same for family health. See just that's the one activities that we did in Laos and not go through in the detail but just to show you different type of chart the graph that we can do. And also the same on about health facilities that at least they get the idea of now how many facilities are there, how many health workers in the country. And by I said the almost real time when new data are entered then this graph should change and at least they know the progress or they can keep track of the progress. Yep, and one last thing is also about the mobile. So this is the main goal that they can see the same graph on the on their phone and then be able to quickly scroll and see the high level of the data. Yep, and maybe I can talk a little bit on the technical part. Hello. Okay, everyone's I'm here from here's Vietnam. I'm a technical implementer. Now we come to the sleepy pack. I will just maybe talk is very overall overall so you too can easy to understand. Okay, this is how does our public dashboard dashboard work here currently in Laos. In the public box we have currently we have two parts. First is a front end part is a page that Sam already show you in a previous slide, many chart many grab table. That is a front end part. And the main technology we use for front end part is react ZS and we include in size inside that in many useful library for showing the chart map by table. Nice. Yeah. And then about the data, the front end will get the data from the back end, you can see you can call it a middleware and we using the express GS. And this is a back end web based application form of the notches and by the help of the express GS we can easy to create the environment for the back end and also create many our custom API for the front end can use it for requesting the data. And then from the back end, we will request the data into the to to get the data. So the front end will, you can see here will not directly connect to the day to to request the data. Then back end we use the library I see us from that library, we have many authentication type to connect to the day to live basic authentication. We can to patch authentication by token to connect to day to, and then day to will return the data back to the back end and inside the back end we will transfer transform the data, combine the data into the object and then send back to the front end to use. Why we need to do that because we support we can request, we can connect request and getting data to multiple day to so maybe after you request to get the data, you want to combine that data into one chart. So that's will be in the middleware to do it for you, and then you push into the front end to see. And in the future, we like said, we will try to get data from different data source like from the Excel from my square and as well lights, right, that's if I in the future case. And for the security here right now we just using follow we just using the basic authentication username and smart world, but it's put into the back end and put in the same day to instant. So, as a container, the box script container. But maybe in the future we will plan to use live because we do use the token, or maybe we will use the event hook when every time the day to running after running analytics we can put the data again for update for the last ball we have the newest data every day. Yeah. Yeah, that's all from my path of the technical. Great. Thank you. And Sam as well. Thank you very much. Is it was at the end of the presentation. Yeah, great. Yeah. Thank you for sharing. And I think this is a great, a great example of kind of the progression that we see oftentimes with moving to public access to data right you want to get it in the hands of decision makers. But you also want to make it available to, to the broader public. And, and so we'll talk next a little bit about the different considerations that are important when going through that transition, especially when you go to the public as a whole. And, but before we do that, if we have a few minutes, maybe for questions for the history Vietnam team about the loud public dashboard. And then we'll dive into the, the kind of guidance. Just a couple, couple minutes if anybody has quick questions in here or online. We have one in the room here. Just keep it close to your mouth and it should be fun. I would just like to know about the decision making process, who decided what elements they need on there. You talked about the National Assembly having very specific requirements. But we all know that there could be different requirements in terms of like general health in terms of maybe a specific disease and things like that. And what was the process of deciding what goes up there. And what doesn't go up there. Thank you. Thank you. So in terms of what that what indicators we will put up there. So, first of all, we have different indicators that at least at the national and even the provincial level that they need to monitor frequently. So those are the things that we put up there that every program they agree that they need to, to see that and also the high level people that they agree that they want to see those. So, apart from, from those things that the health program will decide and will tell us, like, what else did they want to see the thing that the boss want to see and also think that the lower level need to see. So, mostly we work with the program also with the visual that support each program to decide what additional information or initiatives that they would like to be added in the dashboard. There is one question, technical question online. And is there any caching at the Node.js layer here, or does the requests go directly to the DHS to servers. Is there any caching there. Yeah, we have the caching in the middle where after the one time you request the data is we restarting there also, so you don't need to request again when you reload the website. Currently, it will expire after 24 hours. Yeah, maybe in future we will set it and combine it with even hope to persist based on the analytics table time. And the analytics that this next question is about analytics table time. So, for you have real time data but it's every it's every day right so it's 24 hours. So you're running analytics once per day and then using that data. Is that correct. Yeah. We have a few more questions seems like this is an interesting topic I do want to get to the guidance. And so there's just this one one last question which is, and yeah maybe how do how do you do data quality checks before going public. I think actually I'll touch on that in a little bit later in the in the session here today so how to how to kind of validate and ensure that you have the data that you want before you that you want to make public before it goes public. So I think I'll get into that in the more general guidance. I think there was a question from the other room but I don't know if that's. Yeah, covered. Okay. Yeah, okay, we'll we'll go we'll stay with this for now and then we'll go through the guidance because that's a there's probably probably going to be some questions on that too. Thank you everyone for your patience as we go through this. Okay, so, and there are a few categories of considerations that we've compiled from different, different groups that have built public dashboards on top of the dashboards. And this is just sort of scratching the surface of what are some common things that everyone should be thinking about as you're making data public and DHS, giving the public access to data that maybe is collected or stored in DHS to So one important consideration, probably the biggest one that we see sometimes being overlooked is security right so this is should should be top of mind as you're making data public is how do you make that secure how do you make sure that you're giving the public access without giving them access to the DHS to system without giving them access to even be able to find that DHS to system in many cases. And in particular, hiding credentials so sometimes the easiest way to build a public dashboard is to put your username and password in some JavaScript and put it on the web somewhere, and then you have a pretty graph that you can see, but then whoever looks at that and knows how to right click and save you source can see your username and password and access your DHS to system. And just unilaterally that is bad, please do not do that. We do see that sometimes and it's even historically has been a way to embed visualizations in systems, but there are much better ways to do it. And that should be avoided when you're especially when you're giving access to the public. This has been something that with the kind of explosion of public dashboards for health in through the COVID pandemic. And lots of people know that there are operational systems behind them. There are lots of people that have access to these these dashboards and try to maybe can explore a little bit and and find ways to access the system. So, these are some very high level general security guidelines but there are definitely more to keep in mind. So, as an overall takeaway, think about security and think about security a lot when you're giving the public access to data to go through these individually. So, as a rule, if you're giving the public access to data, you should not give them access to the operational system that is being used by health workers or by the ministry to actually do their jobs, right. So, they can be completely separate systems they should be connected, but you should never give access to the operational system to the to the general public, and even accidentally so even if somebody is very technically savvy and can right click and save you source or do all sorts of different things they shouldn't you should protect as much as possible that operational system. And one way to do that is to hide the credentials as we said don't put the username and password in the browser. And there are lots of other ways that you might accidentally expose credentials so just be very aware of where the credentials for your operational system are stored. In many cases it's it's useful even to hide where the operational system is right so I'm saying operational system. I'm just going to say DHS to you might have multiple DHS to instances you might have different systems that are not DHS to, but if you, if you're a public portal that the government website where you go and say my my dashboard dot my country dot gov or whatever it is. And if somebody goes there and then they can see the requests that are going to DHS to my country dot gov which is where your operational system is, you're opening up a surface area for someone to explore that and to try to hack passwords or send phishing links to other people to users of the system. Ideally you should hide that completely so in the example from Laos there there is a middleware there right so the browser doesn't talk to DHS to the browser talks to a another server that then does all of the talking to DHS to on the server side. And that helps to protect the the operational system because unless somebody knows knows the architecture behind the scenes, they can't just figure out where the DHS to server is and start to attack it in some way. If possible, it might be useful even to hide the fact that it's a DHS to system at all. So, this isn't strictly necessary, but it can help to kind of have defense in depth to not even expose that the back end of the system is DHS to there might be other systems it might be multiple systems that are coming together. But if you are using the DHS to visualization engine and you are using DHS to API is, even if that's going to another server and then getting passed on to the the back end DHS to, then it opens up the possibility of someone being able to figure out and use that to their advantage in some way. So if it's possible, it might also be useful to explore obfuscating those API signatures or the API calls that the front end is making. So this is getting a little bit technical I'm not going to get too technical in here but we can dive into what kind of how you would actually do this another time. Another important consideration is performance. So this is again, sort of similar to to what we said with security, in that if you have a million or 100 million people that are all of a sudden accessing the public dashboard that you created, you shouldn't. That shouldn't be the same as having 100 million new users in your DHS to system, right, which will probably take down your DHS to system, I would say definitely if you have 100 million users. And if you have 100 million people, this is a little bit of an exaggeration let's say even a million people if you have a million people that are accessing your public dashboard. That shouldn't change at all the ideally the performance or the stability of the DHS to system that you're actually using in your country for your operational systems. So that's another important goal or consideration here. And there were some questions about caching and the team in Laos answered as well saying that yes, caching is done in the middleware. It's very important because if you are saying every time somebody hits refresh in their browser, and you have a million or 10 million people that are hitting refresh in their browser to figure out what today's COVID case numbers are. You shouldn't have every single one of those refresh clicks go and hit DHS to that can be problematic. So instead, giving data. Close to real time is almost always better than than getting than refreshing every single time somebody clicks refresh so even if you cash for in the in the case of the that loud public dashboard it was 24 hours. Maybe you need maybe you cash for an hour, but it's public data so it shouldn't be changing based on who's looking at it so you can cash very aggressively here to avoid any issues. It also is possible to pre compute or pre generate a lot of the visualizations or the data that is being displayed in the browser to the public. So if that's a possibility for you there's some more technical considerations into how to actually do this. That's something to consider as well there are other other benefits we'll talk about for doing that in a minute. The next example or consideration is timeliness. So as I mentioned, probably up to the minute timeliness like real time data isn't actually useful or necessary for the public if you have an actual public dashboard. You don't need, people don't need to know what happened in the last hour in most cases right. One day is probably fine one month might even be fine in a lot of cases because people aren't even necessarily looking at daily daily data. A little bit different in the case of COVID but in most cases you're looking at how how is the country doing this month or this year compared to last year or last month. And so you can use that to your advantage to be able to avoid to address some of the performance impacts of trying to give real time data is that you don't often need it in most cases so you can do aggressive caching again. You can pre generate or pre compute things before people actually request them. The next consideration is privacy and access control. And this is not the same as security. So it's, it's related and sometimes we say security and privacy in the same sentence, but they're not the same thing. So in the technical security considerations we're talking about how people might hack your system. So privacy is a little bit different in that you don't want to expose the public, not necessarily people that are hacking, but you don't want to give individual personally identifiable information on your public portal, for example. And then maybe you, you don't want to give access to some sensitive aggregate data that you did that is maybe used internally in the ministry but isn't ready to be published yet. Right, so you want to think about what data is actually being exposed to the public and this is, there were some questions online about this as well. So the important consideration for public data access is what data is actually being shared and who gets to decide what data is shared and if you're if you're just all of a sudden clicking a clicking a button and exposing all of the data in DHS to to the public. The DHS to system that might be really bad, or it might be fine. It really depends. So you need to know in the context of your country, the politics in that country, and where what who gets to decide what's public and how what is the process to kind of approve data for publication. Data is in general a good thing but sometimes you need to make sure you're following the right process to make sure that you're not publishing even data that might be correct or up to date, but is not necessarily cleaned up enough yet and could be misinterpreted by the public so that's also something very important to consider is that you're not just putting the data out there but also you're you need to make sure that people understand what that data means and don't misinterpret it in some way. So generally I would say public data should be as explicitly selected and approved as possible. There you can automate that and you can reduce the kind of burden of that process, but generally you want to make sure that it's well known what data is going to be made public and how you're going to do that. And then there's a few considerations here on this next one next two slides that are a little bit less and less critical maybe, but are quite important when we're talking about access to the public. And, and that is usability. So we're talking about a very different user base when you're building a public portal than when you're building a DHS to system, for example, right. So you don't want to have to have a huge training campaign to be able to train people to use a sophisticated tool. You want to make it as usable and as accessible as possible to the people who are actually going to visualize that data or see that data and use it. And those are typically, they can be people that are less technically literate, for example, they're, they can be people who have all sorts of different types of devices from very low spec phones to every browser imaginable and coming from all different countries and different languages and all these different things. So usability and there's a lot of considerations here on this slide and the next one that I won't go into too. Too much detail, but they're important also to keep in mind when you're building a public portal and not just on top of DHS to but just building something that's accessible to the public. One way that this ties in with some of the earlier slides is that the performance in the browser is also very important to consider so you don't want it to take a very long time for someone to load that data. And you don't want to make a bunch of API calls. And if you're able to do that by pre computing or pre generating some images or doing server side generation or aggressive caching, you can also have some of the security and performance benefits that we talked about some other considerations for usability, we won't go into too much detail here. Responsive design is also something we saw in the in the example from the law public dashboard that it's accessible on mobile devices, that's probably the majority of users these days will be accessing it from a mobile device rather than from a desktop so think about that when you're designing your your public portal. There's lots of other considerations here that are worth exploring as well. This one was brought up by one of the his groups so we collaborated with a bunch of developers from different his groups who have built public portals. And, and this was a good addition from that group and as well which is that user support and troubleshooting is very important because if all of a sudden you have a million people that are trying to call someone to figure out how to use this this public website. So to make sure that you have processes for that. Do they have an email address who's going to be checking that email address do you have a call center, all these different things so depending on the use case. It can be quite important and I think that was something that was brought up. I think that was a bit during the, like the explosion again of kind of public portals for coven data. Okay, now I wanted to share and quickly the results of this and whole for this quiz. I guess whatever it was called. See what it was. Oh no, I need to figure out how to get the results. They're going to be on my computer. So while I go through that. Yeah, feel free to go ahead and add some more few. If you haven't scanned this yet. Well, um, while I'm doing that does anyone have any questions on those on those considerations I went went through them pretty quickly but maybe we can on next step I'll talk a little bit about a few kind of low hanging fruit of easy ways you can achieve some of those with some some easy technical choices. I don't have any questions or additions to that list. Well, so I have Austin questions. The first one is, is this process applicable to almost all versions or are there any considerations. And second one is easier documentation available for the implementation part development part of this process to make the dashboard public. Thank you. Good, good questions so the first first question was around version, like DHS to versions you mean. Yes, yeah, so these are general considerations that it doesn't even necessarily have to be a DHS to system behind this, this public portal this is generally advice and guidance around how to build a public access to an operational health system, for example. So it doesn't matter what version of DHS to your running. There might be some nuances or differences in the way you actually implement some of those considerations based on the version of DHS to that you're using. And, and the second question was a good one is, is there documentation and guidance, and it's hard to have like a step by step guide for how to do this because it's a bit different in every every country every context. And, but that's what we're working on so I'll get to this in in a minute, but basically the next steps are, we have some of these kind of guide guidance for how to do this in a fairly easy way. We're going to be publishing that and also looking at doing building a reference implementation that someone could could base their implementation on if they wanted to that follows or takes in takes these considerations into account, and could adapt that to their local and that will also involve some guidance on how to do this technically as well as more expansion on the considerations that I went over today. Good question. Then can you hear us. We can yes go ahead. So we have some questions in the over spill auditorium. Thank you. Yes, I actually it's a question about the use of DHS to not maybe for the broad public but for partners working with the Minister of Health. Yeah, okay. Because you know, in many countries it's always conversation to go through the Minister of Health and the Department of Health information to ask for them to make the request. So I'm, I'm wondering if we can use the HIS to with an access that it's only consultation for not both the big but some partners working with the Minister of Health and if you recommend to do that or if you recommend not to do that. Yeah, absolutely. And I think that's to some extent what was done in Laos. I think. So it's, it's kind of a, a slightly narrower scope of what we're talking about here. So one way to do that would be just be to make a public portal and make it available to the public to everyone. But maybe you don't want to do that maybe there's some data that's a little bit more sensitive or you don't want to be misinterpreted or those types of things. You don't want to do this but put it behind a an intranet or a password on the on the public portal so still anyone, any of the partners can access that can can drill down can maybe see some details. They're not accessing the operational system. Doesn't matter how many people access that it's still going to going to have functionality in the operational system continue. So you could use a lot of these techniques to do that as well. And, but it's important to think about like putting it on the public internet means that it's public. So if you're just wanting it to be exposed to a certain subset so a certain set of partners or something like that. When it's when it's on the public internet it's public and you have to consider all of these things and you might be able to get away with avoiding a few of them potentially if you put it behind a username and password or give give people access in some other way. Hopefully that answers your question. One more question Austin. Sure. Yeah. Thank you. This is doing. I just wanted to ask about the reuse of the two analytics. Hmm. Yeah. When developing web portals. Most times would want to save time to come out with this nice public portals and already we know there are some analytics that have been done analysis have been done generated in within the highest two. And from within some version, I think 35 backward we used to have things to do with plugins and the like that could be used to help or aid and fast track the process of public portals. Is this something that's yet being maintained and if so, how simplified is it to allow public portals to reuse analysis objects that are within the days to. It's a good question. And so this is something that I think in in our technical guidance we'd like to make possible. And because there is a lot of power in the visualization engine of DHS to the analytics API is as well as the tools that have been built that you can design something in data visualizer and then it shows up on the dashboard and maybe you just want to make that particular visualization available to the public. The way the previous way to be able to embed that in an external website was not very secure so we have had removed that for that reason. But it is something that would we'd like to to make possible and it does kind of defeat the purpose of avoiding exposing that DHS to is there because you're using DHS to in some ways, but maybe that's not a that is important of a consideration of your use case and it would be useful to make it possible anyway, it, it is possible, even today, but we need to build some guidance around how to actually embed that visualization in a public website, and how to securely use a gateway to protect the operational system with caching and with hiding the the URL of the operational system and hiding the credentials of that operational system. So there are ways to do it, but it's and there are organizations and Hisp Uganda comes to mind and a few others that have have explored doing this in different ways. And, but we want to make sure that we have clear guidance on how to do that in while maintaining security and performance of your operational DHS to instance, which maybe thank you for that question that was a good question. And that maybe gets me to this first recommendation so I'm going to go through these six recommendations that are kind of high level. So if you do not adopt all of them. These are just things that you might want to consider to have some quick wins in terms of doing some of these. Having some of this protection for the data that you're making public and for your operational system. And so one of those that's kind of an easy win clear win is, don't, don't give anybody public access to your DHS to operational system. It's dedicated that only has public data in it. So even if somebody got a username and password for that system it wouldn't help them at all because all they can see is the public data anyway, right. So, if you have a DHS to system that are just two instance that is just public data, you can use the new aggregate data exchange service and application in the operational system to manually review and publish the data that you want to be connected to that other system. And then you have a, basically you can give, you can be a little bit, you're a little bit more protected by giving the public somehow access to that public DHS to instance. Even if you're not going to give them actually username and password you're still, you still have a kind of a layer of protection there. The second which I would recommend even if you have all of the public data in in a dedicated system is to use a proxy or reverse proxy is actually the correct term, which protects the DHS to credentials and limits the impact on the production DHS to system by implementing caching by performing rate limiting different things like this. Engine X is a one way to do this there are other ways, but the, they're very good systems for basically giving the public access to back end services in a performant and secure way. And so that is a general recommendation that you should you should probably always do that when you're having giving the public access to something. And if possible this this ties with in with number one again, you should do pushing data instead of pulling data right so you don't necessarily need to do this if you have caching setup correctly, but it's more predictable. If your operational system pushes the public data on a regular basis or maybe somebody manually approves it to a place where then it can be accessed by whoever maybe it's even published to Google Drive or something. You can push your data when you know when you know that you're ready to put this make this data public on a regular basis maybe once a day you can push it somewhere and then give other people access to it in some way using one of these other systems. So you can do that with the aggregate data exchange app you can also do it in other ways where you export to excel and upload that to an FTP server, all sorts of ways that you could could do that. So pushing it instead of pulling pulling meaning using the DHS to API to when somebody hits refresh in their in their browser on the public dashboard, pull that data from DHS to, it's much more predictable to instead push it from the system to somewhere where the public has access and three more recommendations I know we're a little bit over time so feel free to leave a few if you have to I apologize we had a little bit of a delayed start. But if, if possible, you might want to hide DHS to specific signatures so you would maybe hash the API calls that are being sent and then you use that hash instead of the actual API signature, so that people can't kind of manipulate it by saying All right, I see that this API call is being sent for this particular visualization or this particular set of data. What if I change the ID to this something else that I know exists or what if I try to like expand that out and find some data that I'm not supposed to have access to. So if you don't let them do that by like basically saying that the API is not is not the full DHS to API it's only certain certain calls that have certain ways to access it, that could be helpful. So that's something to think about. The fifth one here is use server side generation if possible this helps a lot with performance, not only of DHS to but also of the public website. You can pre generate the images of the visualizations that you want or the JSON for the data of this public data, and you do that on a daily basis, you do that on the server. Then the browser doesn't need to make a bunch of requests to the API over and over and over again, and you don't need to request it from DHS to over and over and over again. You can generate that on the server side, which makes everything go a lot faster and is more predictable. And then the last one is just don't don't reinvent the wheel. So don't be, don't build everything from scratch because there's a lot of tools out there that are very good for a lot of these things. So try to figure out how to piece together existing tools, rather than building them all from scratch and writing a bunch of bunch of code in node JS or Java because that's also a very easy way to accidentally introduce security or performance issues. If you're not very careful about how you do that. So again, what's next is we're going to be publishing some of this guidance the considerations that I shared, and consider continuing to collaborate with our partners his groups, all of you. So everybody that and how if you have ideas for how we can improve this guidance or what you'd like to see from this type of guidance. You can send me an email Austin at DHS to.org I'll share that on here as well. And we'd love to involve you in that. We'd love to share that guidance and also develop it into a more of a technical guide for how to do this. If you're starting from scratch, you can probably adapt that to your local. That you need to do it in your local context. And yeah, I think that's about it. I think, thank everyone for sharing the examples. And so we have another like 15 or 16 examples of public portals here. It would be great to work with all of you on, yeah, kind of curating a little bit of a list of those that that we might be able to. Yeah, to share or to, or to use as we develop some of this guidance. Thank you all for sharing that and thank you for your time. Apologies for going a little bit over, but hopefully it was a useful discussion feel free to come up and talk to me or send me an email if you have any questions after. Thank you. And thank you again to Salmonia for the Vietnam presentation.