 Okay, I think with that, we'll go ahead and get started. Welcome to the opening plenary session for day three of the annual conference. Very glad to have you all with us. I'm Mike Frost. I'm the product manager for tracker here at the University of Oslo. And just am here to introduce the session and then turn the time over to my colleague Bob. Just to give you a sense of what we wanted to do with this session. We're often asked about where we think DHS to fits within a broader health information system ecosystem, and also kind of how we view interoperability. There's often requests coming to us about how do we categorize DHS to what are the health information areas that we do and don't work in how do we view interoperating with other systems. And quite honestly, we're often difficult to define. There are many global efforts to try to put together a taxonomy of different digital health interventions and DHS to crosses a lot of those categories. Largely do I think to the fact that our primary mission and the features and functionality that we work on. We have a lot of life from trying to support ministry of health needs. And we think that that's a very important aspect of why DHS to has been sustainable and successful. And more importantly, that that's how we can achieve data for action is to meet the needs of those that are on the ground and using systems, which has often driven DHS to into various different areas. DHS to from our perspective as a kind of underlying bedrock principle to be a free and open source software and to be as interoperable as possible with other systems. We've always intended for DHS to to fit in any country environment whether there's a very established kind of enterprise architecture or whether we're talking about ad hoc linking of various systems. We want DHS to to be flexible enough to be able to accommodate these these various scenarios. So we have recently made efforts to define this more clearly from our perspective. So we're going to go to web page DHS to dot org slash interoperability. I think, or integration. Sorry, integration. Take a look at that page. You'll get a lot of the information from this session there fleshed out in some detail. That's also where we will continue to update and provide more recent resources and links around integration and interoperability. My intention is to have DHS to kind of meet the needs of where you are in your countries and in your projects. So the Bob Joliff is going to lead us through some discussion here. Bob has been working as a senior implementation engineer with the University of Oslo for some 13 plus years. That title like many of ours at the university doesn't really do Bob justice he works as a lead in interoperability security contributing to infrastructure decision. He's often the one coordinating emergency responses, you may recognize him as someone who's come and train the system admins in your country, or who has helped link the DHS to team from your country to others to build a stronger community. I also think Bob serves a bit as the conscience of DHS to here he's often reminding us of the need to be aligned with global standards and principles, which means that he also stays up to date on a lot of those conversations and spends a lot of time on committee meetings and phone calls that he wishes he wasn't doing. I would like to nail down Bob today for this plenary session to give us from his experience, how we view a bit of DHS to and how it fits into the health information ecosystem. So with that, I think I'll turn the time over to you Bob. Thanks Mike for that illustrious introduction. And welcome everybody. Good morning, afternoon, evening, wherever you are. Yeah, as Mike was saying what I'm hoping to do with this session is to introduce a little bit of background and a little bit of where we've come to in terms of how we view our posture around interoperability and try and maybe use this session a little bit of a sounding board as well and hopefully get some reaction good or bad towards the end of it. So yeah, without further ado, let me just quickly jump in there. I didn't come up with the title for this presentation. Unfortunately, I'm going to stray a little bit from the title, but we have to deal with what's on it and it starts off talking about ecosystems. So what we'll do just very briefly talk a little bit about ecosystems and why I'm a little bit uncomfortable with that metaphor in fact. A little bit of touching on architecture. Furthermore, technical folk will do a little bit of one or two slides just on one of the features of DHS to which support interoperability, and perhaps getting into more interesting stuff around one of the shared understandings of metadata required for interoperability to work. So a couple of examples, and then when I really want to get to it just the last two slides where we bring out where we'll present our kind of principles around approaching interoperability. So ecosystems. I hope I don't get in trouble for using that image. Just grab it somewhere off the internet because we look nice. Make a system as is as you all know it's a biological concept right it's defined on Wikipedia at least as a community living organisms in conjunction with non living components in their environment interacting as a system. People are interested in ecosystems because of growing awareness of environmentalism and like. I started talking about digital ecosystems I think the idea has come really from the notion of business ecosystems. Which, as you see derived from from from the biological stuff largely that that kind of discourse has been around issues of competition and collaboration. The kind of properties which are interesting from from ecosystems are things like self organization and scalability and sustainability, probably resilience which is a popular word nowadays, nowadays resilience of systems as a result of those three kind of characteristics. It's frequently used in terms of in the context of API and platforms and you'll hear it used in the respect to DHS to sometimes when people will talk about the DHS to as a platform, which facilitates the growth or the nurturing of an ecosystem around it. Okay, some things to maybe step back and hold on to. I think this ecosystem metaphor can be very useful, but I think it's worth bearing in mind. First of all that health information systems, particularly architecture are not natural, right. They're produced by deliberate human action. And when you start looking at production you need to take into account things like who is doing the product, who is doing the producing, who are they producing for why. The end of the systems is to enhance health workers ability to improve health health outcomes. Right, it's not in order for business to flourish so I think in that sense, there's a little bit of departure from the business ecosystems kind of notion, which a lot of it is about how to create environments for business to flourish. So yeah, the ecosystem metaphor I think is very useful or can be useful in achieving the ends of enhancing health workers ability to improve health outcomes. But at least for us and we don't view the creation of ecosystems as an end in itself and I've seen this proposed in a number of places I can react a little bit to it. I mean our interoperability posture is not about creating ecosystems. So it can be where they are useful. We do a more, maybe a view which it may be more in touch with our kind of traditional mode of operation is around the notion of collaborative projects. Anybody's familiar with the work of Andy Blondard but all that collaborative projects are seen as kind of the kernel social scientist or the unit of analysis of anything which has to do with human action. Right, if we're talking about social human action, then at the kernel of that is the notion of a collaborative project. So, what can we say about collaborative projects and DHS to where does it lead us well. People familiar with the literature that the the his literature or mythology or history or whatever you want to call it will probably be familiar with with references to the early Scandinavian participatory action research. And what that involved was the formation of collaborative projects between system designers and unionized workers. So you move into the early his days. Yearn is probably the best, the best speaker on this kind of subject. Early his place were certainly characterized by collaborative projects formed between his and the National Ministry of Health in various countries, South Africa being probably the earliest one. An important concept I think that underlies both of those notions of collaborative project as a notion of solidarity. That was and I think remains a very strong basis for why and how we we bring those those collaborative projects forward in the world. So another concept, I guess, which is which is part of now that the his folklore history is the notion of networks of action, networks of action, the famous paper written by Yearn and some deep and Eric Monteiro. And what they recognized in that that it wasn't sufficient to see a collaborative project as as a as a unit on its own, but need to be, need to be seen in a, in a more holistic global context. And the need for much broader collaborative projects which spanned countries continents and sectors. Yeah, that's the early notion of collaborative projects within his has always been central to to our kind of mode of action. More recently, the range and scope and nature of collaborations and the kind of collaborative projects we engaged in has has become much more complex. And at any one moment in time, we involved in a significant number of different types of projects some very large and some very small. I like to think of collaborative projects sometimes as if you think of the simplest prototype for a collaborative project in human activity is probably taking a walk together, right two people get together and decide to take a walk. That's something that that humans have have done and continue to do people have walked from Syria to Norway, right people have taken a walk around a field, whatever it might be people are coming together with some reason to collaborate on an activity. Currently, yeah, we've been in large scale collaborations with WHO, PEPFAR and UNICEF, as well as collaborations with with software other software vendors. Open MRS long history with we'll see more of that in a bit more recently we've been talking to the likes of DIVOC on COVID certificates, rapid pro long, long history of collaboration with the depth of engagement, particularly without the software vendors. There is quite a bit. But I think at its at the simplest and least committal this is like the walk in the park rather than the long march. We simply said well here's the API and here are the docs and you know what you want to do and see if you can do it. And there are fortunately I think many examples and probably some of them quite unknown to us where this has been very successful. And this is something we naturally want to encourage it's kind of a validation that our API works and that our documentation is suitable. When we see these kinds of uses. We get involved in more one on one interaction with different vendors to hammer out a common approach or common patterns. This is an example of which I suppose over the last year we've done. A lot of discussion with rapid pro for example, along these kind of lines it's not a short term thing we really want to go into a lot of depth in terms of what are the actual opportunities to be had the collaboration. Similarly, I guess with IVOC. There is a risk to do in terms of setting up regular meetings and doing the homework and and there is a risk sometimes when having in depth, intra vendor discussions like this that you're not always going to be well grounded in terms of what's happening on in the context but the subject right at the end. Often it's sufficiently important or the relationship that you have with the other parties sufficiently important to you that it justifies the effort involved. Thirdly, I guess type of engagement. It implies a kind of longer term architectural perspective and I think this characterizes for example the way in which we have interacted with open HIE since its inception. I suppose nearly a decade ago now I've got a slide on open HIE later so return to that. The point here I guess is to say that from the early days where the collaborative project initially was a relatively simple thing and characterized by very clear understanding of solidarity of purpose. Those collaborations have become more richer and complex. The challenges. This could have been 20 slides I suppose some of the challenges that we've encountered generally in in working with integration. First of all, I think keeping center our solidarity driven collaborative project with with country partners in particular as central in all the engagements that we take, we participate in. It is possible to lose track of that, particularly in the excitement of looking at new technological developments and possibilities and what have you. Staying in touch with the base is really important. I think we find over and over again particularly people working quite closely at the core in Oslo that in the local his groups who often have a much stronger understanding of architectural concerns on the ground. As you have back in Oslo and keeping that very strong that strong connection is is is yeah ongoing challenge. Obviously rationalizing time and efforts I mean what I think we've realized and we've talked about recruitment and more recruitment and the need for more interoperability engineers and delight is that it can be very, very time consuming. Obviously if you're involved with lots of one on one interactions or involved in lots of standards committees and discussions and like, there is certainly need to rationalize. We have had interesting challenges grappling with standards. Generally, from our perspective we view standards as very important as key enablers in simplifying a lot of interoperability and integration scenarios with participate quite directly with things like ADX for example for aggregate data. As part of the technical committees maintaining and developing those standards. GI on the GIS side I think we're quite strong in terms of very, very clear support for the likes of GeoJs and WMS, TMS I think we still support GML also to a certain extent. We grappled a bit with fire partly because it's arrived with such a bang in a way. I'll talk a little bit about fire shortly hasn't been easy. The other challenge I think we've had historically is dealing with the notion of scope creep everybody knows that DHS to can do a lot of things. And very often people are pressuring DHS to do even more things. And historically I think we've had a tendency to drift into areas where, where we probably would be better holding back a bit. I think that the work that's currently going on with this newly formed logistics group is very interesting that regarding the sense that they are, they are very clearly trying to demarcate in the LMIS world, where it is the DHS to has a role to play, and at what point, it's much more useful and beneficial to have, have clear interfaces with, with other LMIS systems. In terms of interoperability features I guess from more technical perspective what's provided in DHS to which makes interoperability possible. The most important thing is to, is to talk of the rest API. It seems almost, almost trivial to be even mentioning that at this point, but it's not been around forever I think the rest API really took shape only around 2011 2012 I suppose. The API is something which exposes pretty much all of the functionality of DHS to the fact that it's used by the DHS to sort of stock web apps means that the, the functionality exposed is fairly complete. It's an ongoing moving target I think, I think that at the moment, we can say it's, it's, it's pretty strong it's, it's almost comprehensive I don't think we can, we're 100% complete. It's something that we constantly striving towards. Clearly, it's good enough that it enables quite a lot of interoperability at present, but yep, that will, that will continue to, to be more become more rigorous. People who work with DHS it will know that it has a very flexible metadata model, which you know is a, is a blessing and a curse, but having this flexible metadata model some of the more important aspects from an interoperability perspective is that it allows for, for custom attributes so it's possible to do things like create multiple identifiers with different types of entities to be able to make a matching between different systems possible. We have very, very primitive support for notifications. I mean up until quite recently what we could do is send out an SMS or send out an email. Part of a result of our discussions with RapidPro in fact and also with Tyvoq has led to the conclusion that we really need to have much better, much better support for, for webhooks. And that's a little bit of functionality is has been implemented already but that's going to be a strong, a strong focus for development of the future. Maybe an API I would say is a little bit like ice cream right it's just going to sit there but you need something that's going to lick the ice cream. So an API is inherently a passive thing with webhooks we enable a little bit of more active responses to events in the system. And the syntax of things like REST APIs and data models and the like. Obviously for interoperability to work, you need to get into the into the semantics of what the data actually means. And one of the things that's clear is that if you were to share data between between any kind of systems, it sort of presupposes that there's some kind of common understanding of, of what I call anyway the structural metadata. That consists of things like facility identifiers. There's always a starting point to, to, to try to exchange data between any two systems or five systems is they need to have some common understanding of how to identify facilities. And as you grow go a bit deeper into these metadata ontology, things like data sets data elements tracked entities, all of these things. You need to have some way of, of getting common understanding between systems on them. Similarly control vocabularies for option sets and categories code lists as people often call them. All of our communities not just DHS to have been grappling with the challenges of how do you actually optimally reach and maintain shared understanding of this structural metadata across different systems. The facility registry was proposed 10 years ago as a low hanging fruit to resolve some of the issues of facility identifiers. I don't want to go into all the detail about here but I think many of us know that is not quite as low hanging fruit as perhaps people, people thought some years back. There's a role I think for terminology services and standard standardized code systems, ICD-11, SNOMED CT, MEDRA has come across quite recently. More and more I think that if we start using standardized code systems it can potentially ease the burden of translation. But again, there's it's no silver bullet we often find that the types of metadata we have there isn't really a corresponding standardized code that's useful. We have IP issues to resolve some of these code systems are not freely available. It causes a bit of conflict with our licensing model of particularly around the metadata packages which I'm going to come to on the next slide. I think people have been around DHS2 community the last couple of years one of the things many of you would be aware of these these packages, which evolved initially from collaboration between between University of Oslo and and WHO. The original rationale is my understanding has been that that they allow a very rapid and simple configuration of the DHS2 of what was current WHO guidelines on on areas like TB, malaria and HIV. The benefit of that being that then all users who who use those packages could benefit from having these pre-designed indicators and dashboards and graphs and charts and the like. I don't think it was considered at the time. I think to do with interoperability, but what has happened I guess as a kind of unforeseen side effect is that now as you start to see a number of systems which effectively have the same packaging of metadata. It's actually quite easy to design data exchange flows right whether you do it with with native DHS2 or ADX or even with fire. If you have some kind of standardizing of the metadata that are in different DHS2 instances, it becomes it becomes an attractive interoperability target and I think we've seen a lot of interest between the last year, particularly around that. One of the things that we've been able to do, at least with the HIV and the COVID aggregate package is to take them to the IHE, to the Quality Research and Public Health Technical Committee and produce committee-balloted standard profiles out of them using ADX. There's ongoing work I think within the metadata packages team to investigate different ways and find different resources for improving standardization of terminologies. I say where appropriate because sometimes it's been hard to find a good fit there. I'll just briefly talk a little bit about fire because it is very important and it's something that I know a lot of our fellow travelers have invested a lot of time and effort into. It's probably become of increasing interest over the past five years. It's a really exciting thing for anybody who's worked with fire and particularly developers get very excited about the richness of the model, the kind of level of energy around it. It's quite challenging, particularly for kind of legacy systems I'd say or systems which like DHS2 are primarily involved with help but not entirely. I think we also have been used in education as a lot of you would know, but also in some other areas, food security. Putting a fire straight jacket on that does, it doesn't quite fit with all of our use cases. So we've grappled a little bit on how best to integrate with HL7 fire-based activities. There was a fire adapter developed back in 2019 through our interoperability or integration team. And we're going to hear a little bit about that in the interoperability session coming up after this. My colleague Morton has developed some quite clever Python integration scripts, which act as a little bit of a facade for for example presenting our native DHS2 option sets as fire value sets, all units as MCSD fire resources. After we've seen relatively low adoption of any of that work in the field. And so, I think as it was exchanged between DHS2 and fire, I think it's always going to be a bit lossy in the sense that the data models are never going to be 100% the same. And for the moment, I get most of the actual interoperability work that's gone on. I think fire has been a little bit on the periphery of that. And that may change over time, but at the moment that's the case. We're following the work on fire, I guess, particularly that's been going on with the WHO smart made lines and within HIE. Many, many challenges and rivers to cross there still. I want to call out a little bit on HIE. It's kind of difficult to talk about interoperability and health in developing countries without without recognizing I guess the place that open HIE has had in all of that. One of the things that is really important to point out is that there's no DHS2 and they open HIE people often I mean I've been engaged with many conversations around this. Why are you using DHS2 you should be using open HIE or some variation on that. The DHS2 is part of open HIE and we have been part of it's not quite very, very start but very near the very, very early years. I should say maybe this is different perspective to some of the other or other open HIE collaborates. We see open HIE as a convening forum for important architectural discussions. I tend to view architecture as the work of architects. Architecture is the practice of what architects do and open HIE has been a very rich forum for bringing architects together to hammer out approaches, resolve conflicts and generally I think in live and deepen the discussion around architecture. We don't really see it as a source of software. It's not like something that you download and run. I think some people may have different perspective on that. Historically within the open HIE community we've been part of the facility registry sub community before it was part of open HIE. We have the HMIS sub community which is in fact run primarily by myself and Morton at the moment. More recently and I've been trying to drive this for some years and I think we're getting better. We've seen some involvement of other parts of our teams with different communities within open HIE, particularly around supply chain terminology. There's been some participation with the COVID-19 task force. Elaine is going to kill me because I haven't mentioned the data use community. There's quite a deep and long term collaboration within the whole open HIE space which we hope to continue to develop. I see it as one of the most fertile forums for architectural discussion currently that we have. I want to point out a few interoperability examples. It's really a shameless plug for the session that's coming up next. I want to then move quite quickly to interoperability principles. We've got open MRS long, long history of attempts to get good integration going with open MRS. A very interesting presentation coming up on that. Bikina Faso has got a really interesting use case. Again, it's about individual data from the periphery towards aggregate data at the center. It's a big theme that's interesting use cases around AMR. Anybody who's worked with AMR over the years will know that who net is the thing. We have John Stelling who's the sort of bees knees of AMR and who net. Working together with Julius describing integration that they've done there. Fire adapter people may be quite interested in what's the fate of the fire adapter where it is now. Ranga will be giving us some progress update on that. So this is just what we were able to talk about in the interoperability session which is coming up next, but obviously there's many, many more. Probably one of the more interesting things we're looking at currently is talking to open function who are they produce a kind of middleware integration software. And it's quite possible that we will end up using open function as the kind of middleware software. But also as support in terms of collaborating open function group around things like code with digital certificate generation using diver. Okay, so I want to finish off really presenting some principles which I think we've had a galva starting to galvanize around as DHS to community and bounce them off the rest of the world to get some feedback off your as at the past 10 years since the birth of rest API we've seen lots of developments we've been involved directly in large numbers of projects. Some of which should be pointed out maybe most of which I don't know but some of which haven't actually born a lot of fruit but consumed a lot of cycles. And DHS to is very, very far from perfect. We've got lots of walks, lots of rough spots. But I think continues to evolve in a good direction. We've galvanized as a team around the set of of six guiding symbols are just run by you here now. But firstly, I can't actually rid of all of these cameras so that I can see what I'm doing. Okay, good. They're all of DHS to and his in keeping with our 25 years historical mission is to act always in solidarity with our country partners brought up solidarity again. Bob just to mention because I think we're stuck on your open HIE slide and if you've advanced already to the side with the principles. My screen sharing is false. Sorry about that. Resume share. You have me. Okay, we can see you now on the full presentation. I'm going to go down to where I was there. It's my little thing to make me go full screen. Okay, you see that principles. First principle and I think this is this is the most important principle and one that we want to keep very much at the forefront is that the role of DHS to is to act always in solidarity with our country partners. And this includes particularly when we engage with, with many other collaborations with many other vendors and partners and funders and whatever it might be central to our collaborative project is the solidarity with our country partners. Resulting from that I guess is that all interoperability efforts that particularly ones that we play a direct role in should have a clearly identified use value to the end users and system owners which kind of implies that they need to be involved from the start informed, as they by their direct input, what is useful. We've seen unfortunately too many interoperability projects where where data has sometimes flowed. Maybe not maybe for a very short time, but not sufficient thought has been put into well what's the use of that and the use for whom. The interoperability use value has got to be balanced against the cost right. Often people will say we want to have this system talking to that system without without a sense first of well what's the value in doing what the use value in doing that, how expensive is it to do it. It's important that that we, we keep this balance at the forefront where we're evaluating what kind of engagement that we need we would like to have or we would encourage our his partners to have for flame. And this is this is lesson from bitter experience I suppose is when you're talking about integration don't start with talking with the vendors of different systems would you it's much more fruitful to start with having data conversations between the data controllers or data users program managers about what's useful. If we have a clear emphasis on what is the data use that's being reinforced by interoperability, then we can have much more informed discussions with vendors about what kind of data flows are important. Fifthly, I have mentioned I think that the important thing to me about architecture is well who are the architects, right architecture, who is who is doing the architectural practice. Very often, the country level architects who are bypassed or ignored in the in the discussion and working with country level architects. Influencing them where possible if they don't exist, facilitate their emergence where we have local his his country teams regional teams, ensure those history involved in in the architectural discussion and not simply focused on the DHS to And sixthly, we view it as incredibly important and beneficial to to participate in girl in global collaborative processes I think I think that they inform a lot of activity on the ground. And it's sharing expertise and like so global collaborative process around interoperability and architecture and standards participation in standards bodies I think it's really very important. But all of this, you know, with with a very very strong foot on the ground. So that's it really that all I was going to present you all what I'm really quite interested at this point is to try to get some reaction, particularly on our six principles, whether they are offensive, useful in need of further elaboration. Otherwise, it's for me. Thank you. Thank you Mike. So we have, I can see john Lewis has his hand raised there john do you want to unmute and tell Bob that he's he's wrong. It's nothing about wrong. It's, you know, these six principles are very good. We all agree, not only in DHS to committee but like in a in many like whether it is open the maras, like everyone like who will be with open standards or things. This is how we just to go, whether it is on open HIV, all the other things. But now like my problem is like, when we are trying to promote this one in the Minister of Health level in in our countries in the lower research settings and just saying these are things. And they think this is DHS to standards. You are from DHS to you only want to use these standards. So I guess like it's somewhere it gets into. Yes, like we adopt these standards but they at the, the Minister of Health level they think, especially these different vendors, the Minister of Health don't really know much about the IP system at all. So they just say, oh, these are DHS to standards, but this is not DHS. These are just like a normal global standards which has to be which DHS to adopted it. And like the you're using it and changing it to meet the global standards. My thing is like if we can try to as a community come around together, like we have the blature work package, if you can have this. This is a standard as a separate one doesn't have to be so much onto the HR. So they say these are global standards, like if a Ministry of Health wants to get into building a COVID self registration the module or COVID vaccine module or other things. These are some something which they have to think through, rather than just like going back to 20 years back when they're just like doing things in the in a very silent matter. And all of the developers country, they know about this one. They include it but in low settings, the countries where we don't really know the message has not reached that. And every time like the his members or things we want to try to educate on things. It's always it is interpreted as these are DHS to standards, not as a global standards. Maybe we can try to include something on that one. John, are you suggesting, first of all, when you talk about standards you're talking about these principles. Are you suggesting that we should be elevating these principles to a more broader audience than just than just DHS to. No, I think. I think we can try to lead by example. Just listen these are the things I'd say but when we are talking about it, the people think because we are promoting VHS to. That's what all the people, they think wrong, but it's nothing to do with VHS like PSI, which they used in Vietnam or out place. They have their own app and other things they talk to you to VHS to send the data perfectly fine all this major actors they know about the standards, but not the Ministry of Health. And Ministry of Health, they get into developing small from their own local company and other things. They just sending the data they will download in Excel and then like upload it up all the patient data and everything. So there are these kind of problems what we are facing. I guess like when we talk about this one, like maybe open HIE or anything like we are trying to promote but instead of like this like diverting to towards DHS to. Because these all the things. It's, it's nothing to do with the VHS to itself. Yes, we are doing it in our principle, but it can also be a bit more, let's say, the guidelines. Maybe like other people can from the field can clarify. I'm not sure I see another clarification on that point but there is one that come up that may be fun to respond to so a question from the chat was for principle for where you said don't start with the vendors of systems does DHS to classify as a vendor and if not why not. Yeah. That's good question. I had a grapple to bit without words. I know if there's a better word for it, but as as we produce a product right we produce DHS to its open source people can download it and use it. But there is a sense in which we are a vendor. I'm not sure if there's a better word for that so the producers of systems or whatever it might be but yeah I in the way I'm using the word vendor I would say yes DHS or the University of Oslo classifies as a vendor. But I'm not using it in the sense of you put money in your vending machine and get the kind of coke out. Maybe maybe vendor implies a more commercial exchange I don't know. I think it's also worth saying from our side I mean we do straddle that line of it that we also from the University of Oslo we always consider ourselves a university first with a research program attached to it. That doesn't mean that you or your country need to trust that but this is the angle that at least we come from and that kind of guides the things that we choose to work on and the principles that we promote. So we're offering a free open source platform. People can take or lose but we also think very much in terms of you know from an outside perspective you'll see that research coming out of the University of Oslo also is critical of DHS to at times and what we're doing and we try to learn from that and grow from that so. Anyway it's it's again very much up to up to the country how they view that and I think we would all feel comfortable saying don't don't trust DHS to either. I mean take a look and see what makes the most sense in terms of data flow and what is the data need that you have. Does the system work. We don't think of DHS to is being perfect and we don't think of it as covering every possible use case. So, I don't know if you want to add any more to that Bob. I'm very open to suggestion to a better way than vendors. Like when we deal with open MRS rapid pro. I mean these are all produced by different groups with different kinds of organization. There's a sense in which they're all vendors of systems, but I'm happy to use a different word. If somebody's got one for me. Okay, so I've got I've got a hand raised here from Adam, Adam, do you want to jump in. Yes. Thank you both for this presentation I think this is highly welcome, given the context in which we are now and particularly in countries in Western Central Africa where people believe interoperability is now a sort of. magic words to skip to escape criticism about duplicating system or implementing fragmented ecosystem in. I think I and I hope that we have many partners in this session. So, on the principles, I think. What I would suggest is that. First, we, we need to have a sort of. Readiness checklist of tools or whatever for countries and stakeholders to move if a country, what it takes to have interoperable systems and if the country is ready for it. Often when these initiatives come. Sometimes they don't even know who to talk to is it the technical people is it the decision makers. Why are we going to talk to and from what you said in this presentation it is clear that you need to talk to many people, the program managers, the users, the it people. The sometimes the politicians and so forth so I think we need to have this kind of to be in place to help people in knowing what to do and. And who to talk to. The other points I have regarding the the six principle is. I mean he says it's a good attempt, but as we all know the, the entry barrier for this kind of arenas is very high. And it's not given to countries or people in countries to participate in this kind of arenas and gets their voice heads so I know there is no. Magic solution for it, but we need to think of how can we make it's possible as much as possible. Thank you. Good thanks thanks Adam and you touched a little bit on operationalizing, which I haven't really got around to in this presentation. Some of the things like like checklists for for architectural discussions and like, but we are working in that direction at the moment. Yeah, thanks. Like how are we doing on time. Yeah, I think we just have time for one more question I see Scott has his hand raised as one of our researchers here at the university probably to disagree with how we described a vendor. Oh, great. So that kind of myself now. No, I don't disagree at all. Well, I have some other thoughts but that's, I'll put those in the chat. I actually just wanted to say I really appreciated the historical and kind of theoretical framing Bob that you did of the presentation I thought that was a really illuminating and not something that really we hear a lot of and it strung together a lot of different elements that kind of shaped a larger approach. I was going to ask, you know, kind of a practical question. You have been in a lot of interoperability projects. At what point, you know, many of those are unsuccessful, unfortunately, and at what point. Do you think there are diminishing returns or maybe to use like the classic analogy by nor stoop stoop's kicking it you know trying to get a dead horse or trying to ride a dead horse. Right when when is it that the interoperability project is is failed. Does it ever fail and at what points of countries like reconsider the interoperability approach. Do I hear a cynical answer. Yeah, of course, yeah. That's when the money runs out Scott. The interoperability projects will run as long as they are funded to run, regardless of what use value is being delivered on the ground. And the point where you pull the plug is when the funding gets pulled. Okay, that's that's that's the cynical answer. I think that sometimes sometimes projects seem to have failed and they somehow. They reemerge in a better form. I mean, I think the open MRS report was I actually wrote the first open MRS DHS to module for reporting to DHS 10 years ago. It was kind of horrible. I think it's still works to some some state in India in Himachal Pradesh, but generally it was really very horrible, and not not something I was very proud of but I really pleased to see that there's new workers come along where people have a new DHS to module and fixed all the horrible things in the early one. So sometimes you know there is a phoenix that emerges from ashes of bad interoperability projects. So, in terms of, you know, when to pull out or when to pull the plug. I think it depends a lot on the nature of the calibration that you have with the partners that are involved as well. But very often it's about when the money stops. Because that raises the question, whoever has control of money when should they stop the money. I would love to keep talking forever but I think that we better give some time for the next sessions that people are going to be dispersing, but wanted to say thank you very much Bob for for your transparent and interesting presentation with us we we are always open to feedback on these things and are constantly learning from the what we learn or hear from the community so please take a look I did put a link to the website where we've started to elaborate on these points. And if you're interested in the topic of interoperability as Bob says there's a session coming up just now this afternoon so take a look for that. So thank you everyone thank you again Bob thanks for all those that participated, and we'll see you in the next sessions.