 Ok, let's begin. Good afternoon everyone and welcome to this session. Very big auditorium, bit presumptious to expect that kind of a crowd for this topic, but we'll see if someone else turns up. So thank you for joining us for this session. It will primarily be me speaking, and hopefully we'll possibly have some discussions in the end. I also have some slides, we'll see how fast I'm able to go through them. Yes, so let's see if I can change the slide. So in this session, this will not be a very technical session. It will be more along the lines of some of the sessions that have been today on capacity building, the HSTU ecosystem, the activities that are going on during implementation of the HSTU. And I'm here representing the HSTU design lab. I'm a researcher and lecturer here at the University of Oslo, PhD in information systems and doing research on digitalisation processes and digital innovation processes. And that's also what we do in this lab. We're not doing research on specific applications or technologies. What we're very interested in is how do we organise the processes where we implement technology. And a lot of what we do is in collaboration with HIST groups, HIST network, work for instance together with Andrews team in Rwanda, sitting over there, people in Tanzania, in Mozambique and so on. And we also study the same type of processes in the Norwegian IT context because we see that there are many challenges and practices globally related to implementation of technology. And particularly within the HIST network, we're interested in looking at how can we move from projects that are very IT oriented when we implement technology to projects that focus on organisational improvement. Meaning shifting a bit of the focus of design, from we're designing IT features, user interfaces and so on, to we are designing organisations with the use of technology in order to achieve certain outcomes and improvements. So in this session or first, the aim of this session I think is that I want to plan some ideas, thoughts, perspectives and then perhaps we will have some time to have some small questions and discussion in the end. And then hopefully some of you will be interested in discussing further these type of topics with me and others in the lab. So we will first try to look at drawing up some stereotypical projects, the classic mirroring paper IT project versus projects that are more oriented towards organisational improvement. So a couple of examples on that. And then we will try to dive in a bit on what are the differences between those processes, what characterises the processes that are more geared towards organisational improvement. And finally, what are some typical challenges that we meet when we try to organise those kind of processes. So starting with an example, this is a very typical IT project and also in the DHS2 context where we have an existing paper based system, for instance for managing medical records at the hospital. And then one way of organising this project is that we design a computer system that more or less mirrors what we already have there. So setting up a digital version of the paper based reporting. Of course this has some benefits related to data collection aggregation and so on, but from in this context it's kind of more or less a replacement from paper to computer. Then what we are very interested in is that we see more and more projects with DHS2 that does something different or more. So this is for instance an example loosely based on some work that South Digitus has been doing, his group in Mozambique where the project started off as a classic kind of DHS2 replacing the medical record project. Towards looking at, okay, how can we improve more of the flow here at the hospital. A classic problem with these typical projects is that the users here they actually say that paper was much better. It was more stable. It's not affected by electricity dropouts and so on. And it's easier to use generally, more flexible. So what they did was to look at, okay, how can we do something here in this context and what they saw. Okay, we can actually create a new role at the clinic. We can see, could we take some of the work of entering stuff over to a new work role. So then they created this health secretary work role where the patient when they show up, they register into the system before they go to the doctor. And then the doctor is left with a calendar where they have all the patients listed and then they just click on the next patient and then they can enter what is actually, you know, what matters the most. When they had done that, they also saw, okay, maybe we can also have an application allowing the patients to register before they come. So then they're also shifting some of this enter work there, but that also helps the patient because they don't have to wait in these massive queues at the hospital every time they show up. And now they're also looking at, can we send SMS, you know? Reminders, is it possible to then connect this to the lab tests, you know? And then it kind of accumulates on that. That's one example. Second example, during COVID, there was a lot of projects relating to, okay, how can we use DHS-2 for registering lab samples, for instance, lab tests. And then one approach that was tried in several countries was to, okay, we have a paper-based system, we're introducing DHS-2 for entering data. However, this could create a lot of workload and this is again loosely based on the project in Ethiopia where they first tried to implement something along these lines. And they saw that this was too slow and they started to experiment with a lot of different ways of organizing the processes around the technology. So what they ended up with, is to use some volunteers in registering and testing the patients, getting data in. And then you've got some kind of barcodes that can be scanned at the lab, pressing only positive or negative, which is, and then it's triggers SMS that is sent to the patient, doing a lot of the workload on these workers. And the point here is that we are dealing with two types of implementation processes, basically the very typical IT process. And this is of course not the his thing, this is in Norway now, we have some big cases, for instance around medical patient journal systems being implemented, where there is a very IT-oriented focus and not that much focus on how is this, how do we design the organization in light of the possibilities that we have in the technology. And this, when I now use the word digitalization following from this, this is what we refer to. Not just the design of IT, but the design of organizations based on possibilities in IT, that makes sense. So in the lab we were interested in, okay what does it require to go from a focus on this, more towards a focus on the digitalization in an organizational sense. So the first question that we want to explore now is what does this type of process involve. And I will touch upon three things. And there are of course many more, but these capture some of kind of the central basic elements of this kind of process. So one thing that is different between the classic IT project and this more digitalization, digital innovation type of project is that the scope of design, when we are implementing technology is not just IT features, user interfaces, we're actually designing the organization. So it has what we call a social technical, meaning at this both technology and organizational arrangements that are subject to design at the same time. So we look at what do we have in the technology, what are the possibilities for instance in DHS2, tracker, aggregate, all these kind of things. And then we can start also looking at, okay, how can we then also shape not just that, but also the organization according to those possibilities. So it means that the intervention that we are doing in the organization is comprising both of technology and of for instance work processes, roles, services, standards, and of course also a big hurdle at least in the Norwegian context, laws and regulations, which often also shape what can actually do with patient data for instance. That also means that we as IT people, if I can call us that as a collective, we cannot just understand the technology, you know what is there in terms of IT stuff and how can we improve that. We also need to understand the broader organizational routines and practices and the design teams that go into doing something like this has to have organizational capacity, organizational mandates and so on beyond the typical kind of IT mandate. It also means very, very practical thing, but which is interesting. When we talk about humans, when we design technology, we often refer to them as users, who users have to design and user, user testing and so on. But when they are part of the system that we're designing, they're not just users, they're actually system participants. They're actually part of what we are shaping, if that makes sense. So they're both users of some parts of the intervention, but they're actually also the living thing that we are actually trying to shape in order to achieve some kind of organizational improvement. So that's the social technical scope. Then this also has some consequences on how we organize the problem solving processes that we are doing in this project. So a very general kind of perception of any type of project is that it's about problem solving. So we have a situation that is less than ideal and we want to move towards a more ideal type of situation that's problem solving. And that can also be a very technical exercise, moving from the lack of technology or a problem with the technology to an improved technology. We can also approach as a social organizational type of problem. And the way we do that is very different. So a very typical model for problem solving when we deal with technical problems is the so-called rational problem solving method. This is the typical thing that you learn when you study IT here, for instance. You have a programming problem. You divide that into several small pieces. You solve each piece, and then the totality is your solution, for instance. Same with engineers, and they kind of disappear almost. So if you see here, we kind of formulate the problem. We diagnose this until we know everything about the problem. We know all the dimensions. We divide it. We can build a solution, put it together, and then implement it. One issue that we meet when we try to do the same with organizational problem solving, when we have a scope of design that is not just technical, but also organizational, is that the problems that we're trying to address is not the terminate, specific, easy to define. They're what we typically call wicked problems, and they are, there are a hundred different perspectives on what the problem is. If you ask me what the problem is, that will be different from my boss, the complete opposite, for instance. The problem is changing constantly, because there are problem solving activities, for instance, going on in the organization at the same time. So what was the problem yesterday might not be the problem tomorrow. And there really are a million different components to the same problem, which are also relative. So one analogy that I sometimes use is that when we are building roads, we can see those two types of problems, the technical types of problems, which relate to how do you build a road. And then you have these wicked problems, which is more about where are we going to put this road, and do we need the road, and do we need something else. So, and I think this is fascinating, because when you drive on such a road, so we have some new roads, for instance, in Norway, when we drive south of Oslo, and it's very impressive, you know what you're impressed of, is how extremely sophisticated this technology around the road is. There are signs, there's electricity, everything tunnels, very, very complicated stuff. But this typically is not the most difficult thing, at least in Norway. This would take three years, for instance, to build this, where this process of establishing this line, where is the road supposed to go. That could take 20 years, typically in Norway. We have this endless project going on in Norway, where we just discuss and discuss and discuss, where is this road going to go, and then suddenly someone says, we don't even need a road, we need trains, because trains is the future. And then people say, no, no, trains are all their own rails, now it's electric cars, and now we need a road. And I've been following this process of this specific road, for instance, and here you have all kinds of interests, you have different municipalities, they want to strengthen their area, in terms of new stores, industry, and so on. So then they want to road closer to them, or to some place in their municipality. Then you have some people concerned about nature, for instance, saying that, okay, we have some wildlife here. So we can put the road there. Then you have some people that just kind of built a cabin here. So everyone, you know, and the more you look at the problem here, you know, the more dimensions kind of come up, and there are different perspectives, it's possible to say that this is the one definite answer, or this is the definite problem. It's a big, you know, thing. And I think it's interesting that in digitalisation, this goes on in parallel. So here at least we can kind of put the line there, and then we build it, and now you're kind of stuck with it. But in digitalisation this is kind of an integrated process. You know, the building, the deciding what it should be, anticipating how it will affect things, that all goes on in kind of one big thing. So the rational problem-solving methods has some problems here, because it's very hard to kind of diagnose your forever, you can kind of diagnose forever, until you kind of end up doing nothing. So then there are some alternative problem-solving methods, and which are also argued to be more conducive to innovation, because another problem with this thing here, is that when you have decided on the problem, it's often a way of going back, even though you see that now we have a new solution that could change what would be the right problem to address, for instance. It's a very typical situation with technology. You know, we plan for five years, now we have the problem, but then all the technology has changed, and we have a lot of new possibilities and so on. So one way of looking at problems, instead of these kind of definitions that we need to define in detail, and then so on, is that problems are what we call frames. So there are ways of seeing a situation that points to different types of solutions or interventions. So if we frame the problem as a wildlife thing here, then we will see a set of interventions or possible things to do. If we frame it in another direction, then we might see others. And this is very useful in the digitalization and digital innovation process. Because, ja, vi kan show you an example here. One thing I also want to mention is that this separation here is also problematic. I touched upon it that, you know, what are the possible solutions that we can implement. We also affect what is the relevant way of seeing the problem. So, with frames, the idea is that you kind of explore both the interventions and the problems together. And the problem is only valuable, not for being the one true best problem, but the problem framing that shows the best interventions that you could do. If that makes sense. I will give an example now. So this is a typical DHS2 problem situation, where we have a data entry screen, where the usability is poor. I've been part of several projects, but this is the problem. And then one thing you can do is to diagnose in words. That's the classic problem solving way. So, okay, why is the usability of this data entry screen poor? Well, it is a mismatch in the terminology of the health workers and the computer system, for instance. So the health workers aren't really understanding the terminology there. It could be that the forms are hard to navigate. I think that's the classic DHS2 situation. The form is very big, for instance, very hard to enter data in the right column. And so on. But what you also could do is to try to diagnose outwards. So why is it a problem? Why do we care that the usability is poor in this data entry screen? And that will point us to another set of problems, which are broader, for instance, that there are many data entries. That's why we should care about that usability is poor, for instance, or that reporting rate is low. What is interesting here is that each of these things, and we can go on outwards forever. We could say, okay, why is it a problem that the data quality is bad, for instance? Well, that's because we need good data to make decisions. And that's a new problem. Each of those problems can highlight different types of interventions. So let's say this is loosely based on the project that was involved in East Africa some years ago. We began with this usability kind of problem situation. And then we zoomed a bit out. We looked at, okay, the reporting rate is low. Why is that? And then we ended up with a really session that, okay, what we were actually failing to do here is to support the work of the hospital pharmacy managers. This was a logistics, health logistics kind of project. And we were able to do this because we were trying to kind of zoom in and out, try different ways of framing the problem. And from this, we then started the process that typically goes on, where you look, okay, if we frame it like this, what are the interventions that we see? What are the possible ways of using DHS2, for instance, to address this problem? So you see, this is something that I already touched a bit upon. You know, depending on how we frame the problem and the level in which we frame it from points to different types of interventions. And returning a bit to this kind of separation formulating the problem and formulating the intervention. What typically then is useful is to look at, okay, if we frame the problem in this way, what kind of viable potential interventions do we have? So if classic with the DHS2 experts, for instance, is that they will see a lot of potential ways of doing DHS2 things with this problem, right? It could be that, okay, I know tracker. And that was happened in this project that is kind of behind this. We saw that, okay, we could support the pharmacy management better. We knew that tracker had some capabilities that could be useful there. And then we saw, okay, maybe we can capture all this data, where the usability now is poor, not from a reporting form, but from a process support system, for instance, where we supported the work of the hospital managers using tracker and then taking that data out to aggregate from the tracker reporting. And that's one big problem in many projects now. We will return a little bit to it also. Is that when we then separate the problem formulation with the formulation or identification of the solution, then we're also not enabling the people that knows this technology to be part of saying, what is the right problem to solve here? What is the low hanging fruits? If we frame it in this way, what kind of consequences would that have for the technology implementation, for instance? So ideally in these kind of design processes, what you want is to be able to both look at different ways of framing the problem, different types of interventions together, and ideally do this in a team where you have both technology experts and organizational experts, because then you can see different interventions based on the frames that you have in mind. And this is the way we have decided to divide up societies of course into a lot of these kind of silos of expertise. So we have people that know technology. They will be inclined to see technical solutions to any problem. We have people that are organizational experts, health experts, and all of them will come with different types of frames that will affect how you kind of see the problem of the interventions. Final feature of these kind of processes, we see that okay, they have a social technical scope, not just a technical one. We see that there is something, hopefully you follow at least some of it, things that affects how we deal with problems when we have these wicked, complex problem situations. And finally, it also affects a bit on how the process is going in terms of feedback loops. So another very typical feature of the traditional type of problem solving processes is that when we have defined the problem and the intervention, then we have a set solution, and then we implement it. And then with agile software development, which I think some of you would know, then there typically are some cycles back and forth, depending on some kind of functional requirements. However, when we are trying to intervene in these organizational systems, these are, as we touched a little bit upon earlier, complex and evolving. Meaning that when we have come up in ID for what kind of intervention you want to do, it's very difficult to anticipate what kind of effects will this have on the organization. One interesting example that I discussed with some colleagues from Kenya yesterday, for instance, was when they've been dealing with the problem of reporting rates for some time. So the reporting rates are low. Then they tried an intervention where they put up this monitoring of the reporting rates. So everyone will see the number. 80% reporting rate, for instance. And then people will be confronted if they are not really having those high rates. Interesting with that intervention was that, what happened is that the reporting rates went up, but not the complete rate in the forms. So what people are doing then is to just click complete on the form, which will make the reporting rate go up. But there is no data in the form. And that's hard to anticipate these kind of things up front. Because this is not a computer that you can kind of order to behave exactly what you want. It's an organization, constantly flexible, working around, very smart. Smart in a very different way than a computer. Also in addition, the system is constantly changing. This has been a big problem in some of the Norwegian big IT projects, is that this defining the problem and coming up with requirements for intervention takes eight years, for instance. So after eight years, then we're ready to implement. But then the system is really not there anymore, because the organization has changed so much. The health sector might have changed the way they organize things. The technology is legacy. So you're really not designing for the organization that you thought you were designing for anymore. So in these more organization-oriented, innovative approaches, we need to see the things that we define in the beginning of the project, more as hypothesis or guesses, ideally well-educated guesses. Because we have been doing a lot of activities to really try to understand the problematic situation and so on. But we cannot kind of spend eight years on that and then anticipate that we will know exactly what is going on. Rather, we need actually to go into organizational evaluation of interventions quite early. And this is of course a big problem in these massive projects, because you then need to really break down the intervention into very small pieces, which is very different from the way that these big IT projects typically are now, where you want to kind of replace the whole system all at once. To these smaller types of cultivation, where you try to change something, you learn from the effects qualitatively or quantitative organization. And that feeds back into extending the concept or replacing it. In the example I just provided, for instance, then it would make sense to see, okay, this might not be the right way to incentivize or measure. Here we need another way to measure this, to secure that it's not just the completion rate that goes up, but also the action. The actual entry of the data. Good. So that was three features of such processes. Social technical scope, certain way of dealing with problems and this kind of organizational agility, not just focusing on technical requirement, but also on actual organizational effects that feeds back into what are we designing here. And then I wanted to spend a little bit of time on some issues that makes this very difficult to do in real projects. And I stressed some, as I also did on the features, because there are many, but I had to be very selective here and I've taken some challenges when I find quite interesting. And these are both related to how the projects are perceived, scoped and measured. It's related to the capacity we have for dealing with these kind of things. And it's related to the practice of managing projects, which is not normally geared towards these kind of processes. And I hope this also gives some biases for some questions and discussions in the end, because I think one thing is discussing what kind of processes do we ideally need to design, but the other is how do we build environments for these processes to take place, which very often comes down to projects. For instance, how do you really project that is conducive to these kind of things. So the first challenge is that very often, and it has to be to do with perception. When we are doing this project with DHS2 and also other places, it very often falls under the IT or HMIS department. So it's seen as an IT thing. So what we're doing here is implementing IT. Okay, it makes sense then that's the IT people's job. And that has also traditionally been the case that we have an organizational management section of the organization. They're coming up with all the needs that we have. Then they make lists and then they send it to the IT department. And the IT department finds out, okay, how can we address these requirements. And then they design user interfaces and functionality and so on according to that. But what is interesting now is that the technology is driving what we can do in the organization. So it actually has to go, and some people wouldn't like that idea, but it actually has to go the other way around. You need to know what can we do, what do we have the possibilities of the IT, and then what should we do with the organization. The organization also need to respond to what we can do. So if we know that we now can do things related to tracker, for instance with DHS2, then we also need to rethink, okay, how are we organizing things in the organization. That is not the typical tradition in many organizations. It put under IT, and IT is kind of ordered to deliver based on a certain set of requirements. That also means that it can be just scoped as IT, which I think is kind of the number one problem we see in the projects that we work together with his groups in, is that they find all these interesting new things that could be done in the organization, but there is no mandate to do that. And they tell someone in the management of the organization, but they say, why should I listen to IT person on this? So there is a separation there, and it's very difficult to communicate across. So one question that I'm kind of looking at, is how can we change these type of perceptions and the way of structuring this. Big question. Another part of the first challenge is how we measure these projects, the IT projects, which is very interesting. Very often, this is very often the case in Norway also, what we measure an IT project on, is delivery per requirements. So we define requirements, technical requirements, then IT people come in, they deliver it, and if it addresses the requirements that are noted, then we are getting paid as IT people. And that doesn't really leave us any incentive or accountability for what is actually organizational effects of that IT. So if you kind of make this in the next stream kind of case, what we are tasked with is to increase the presence of IT in the organization, push more IT into the organization, and hopefully that will lead to some improvement. But we are not accountable for that link, which is not related to this agility or organizational effects. It's not really leaving any unsettling for doing so. So we have a grotesque example now in Norway. We have implemented this huge medical patient journal system on the west coast, where there are so many challenges. Half of the doctors at the main hospital there is threatening to quit because they say that the system is kind of destroying all the workflows and everything. So we have to make millions a day to kind of compensate for all the issues with the IT system, having to postpone various other initiatives, like building a new mental disease war. But the IT people, they have fulfilled their part of it. They have delivered the IT system. So they are enjoying all the payments and everything, regardless of the IT system, basically kind of ruining a lot of the practices, like the IT system. So in systems thinking, which is a way of thinking about these kind of things, you know, they say that if you tell me how you will measure me, I will tell you how I will behave. And if you think about this in relation to IT, this is a bit of a dangerous thing, right? If you just measure on IT, we will deliver IT. All right, push IT in. But if that's not really accountable to the organizational consequences, we have what they will in systems thinking call a perverse system and so this is interesting. However, there is also a problem, how do you actually then measure the organizational consequences and how do you not scare away all the IT people if you say that we will hold you accountable for what will happen in the organization. That is a very tricky problem. So it's not the easy solve here. So big question, how can we incentivize and measure this digitalization project so that we try to kind of move the focus away from increasing presence of IT to how is IT actually supporting us, how can we strengthen organization based on IT. Second type of challenge is related to capacity. And I think on both sides, meaning that on the side of those that are ordering technology, for instance at the Ministry of Health or whatever, and on the kind of delivering side of IT consultancy or whatever, a lot of the capacity is related to IT and it's also related to kind of traditional IT project management, traditional problem solving methods and there is kind of a limit of capacity on this in between whatever it is, digitalization manager or whatever IT organization manager. And I think that's one of the biggest problems we've seen in some of these groups also that are interested in being more accountable to organizational challenges. I think it's very difficult to recruit that type of person or that team from our, for instance, local universities. It's the same here, most of the people that take a degree in informatics or computer science, they are very technical. And then you have the organizational people but they are not really technical so you have that gap between, which is interesting. So another big question is how do we build this capacity, where and how. Is related to project management practices. So of course project management, that's a discipline in itself, how do you organize a project, fund it and so on. And I was at the conference in Oslo the other day in digitalization conference and they said that their number one challenge in digitalization work is that we struggle with a 60 year old understanding of projects. So they're taking this road and the figure I showed earlier, road project management type of practices into this digitalization innovation type of projects. Where, for instance, the way that we try to control projects is often related to the output. Meaning, if I hire someone here, Eric for instance sitting there to build a new system for me, I will try to kind of control that project by defining very clearly what I expect to come out of the project. But of course if the project is about innovation, then that defeats the whole purpose. If you already have defined upfront what will come out, then what kind of innovation are you expecting and the more detail you define this, the less room for innovation, right? So we have a problem in how we are controlling the projects related to innovation. And this is a general problem in project management literature as well because you know, innovation project is a big thing. But we really don't have the tools that can secure kind of the control that we need and that clients for instance need when they hire consultants while retaining some kind of flexibility for innovation to involve. And then there are some interesting approaches emerging, for instance, more type of interactive agile project control mechanisms where you have much more frequent kind of interaction between the controller and the one being controlled. But this is something that remains a very big challenge in many previous two projects. This also relates to procurement where very often what is procured is a project. It's not the, for instance, this problem framing stuff that I talked about, which is very kind of flexible and intangible type of work. That's not what we're procuring, we're procuring very often a list of requirements, right? So how do we do that? How do we develop the project management and procurement practices that are conducive to digitalisation? I think you can look at this presentation online if you want to see these links. In the EU they have a big initiative around innovative procurement, for instance, because this is of course a big challenge for the public sector everywhere. How do we retain control in projects while procuring innovative type of activities? And what they do there is that they have these more open processes in the beginning where there are several people involved in this problem framing intervention activities. And then they have a second procurement where they actually decide which of these are coming out. But this is also a bit about what kind of risks do the consultancies, for instance, take when they're coming into the project. If they spend a lot of time on this kind of activities but they lose the procurement of the final thing, then who is in charge, for instance. Big case in Norway, now exactly this, where they have one consultancy spending I think 100 million Norwegian kroner, which is 10 million USD, I think, on all this preparation work, but then they lost the competition. And then who should pay for that? Yes. So that was some challenges, some features. I would just mention that in this lab, before I open up for some questions, in this design lab we are trying to work on some guiding resources for these kind of processes that we have discussed today. So we are, for instance, working on the model for problem solving, basically digitalization problem solving. We're actually going to work closely now with his brother, Andrew's team on trying out this on some problems in Rwanda in the fall. But this is available in a, I would call it the early alpha version online. So I tried to put this here. So maybe if you take the phone there, you will actually get the link and you can click on it. And since this is in an early version and since this problematic situation that we are dealing with has so many dimensions related to project management and how do you for the things, we would be very happy to have discussions with people that are interested in how we can make this a bit theoretical, fluffy things into concrete, you know, things related to how do you organize the processes, the capacity, projects, management, and so on. So I really hope that some will try to scan this and then reach out. And that's also related to this. If you find this interesting as a topic, I really hope that you will reach out to me or Nina sitting there, also working in the lab, but she's much more around here than I am. So that's why I'm mentioning you. If I'm not here later today, for instance, we will be here a bit after the session, but also I have my email address here. If you find any of this thing interesting, it will be great to have more discussions. For instance, you know, we could have a session with your team, whatever that team is. Is it in the ministry? Is it in the donor organization? Is it an expert team? To try to discuss how can we learn something from this and how can we learn from you in processes that you have been doing? Yes, so I'll end with this slide. And that's some of the questions that I just asked during the presentation. And open up for questions or comments if you have any. Thank you so much for the people who have been here. Thanks a lot. Great initiative. I mean, seriously, very, very impressive. A question for me. I did actually try to do something similar, something similar. Oh, yes, sorry. I'm Agata, I'm a Solutions Design Manager at Sympens. We deploy responsible biometrics. And working on an intervention, a project where I'm trying to scope out a new intervention that will take into account immunization protocols, but also procedures. What community health workers do in terms of visiting households, different responsibilities that they have. And there's many different ways of how you can approach the problem, look for the solution and so on. But the challenge that I'm coming across is that it can balloon so quickly. I'm getting all the stakeholders involved. And I'm trying to find for solutions that would kind of be in the center to satisfy all the objectives and so on. But obviously we're working in a constrained environment in a sense that there's only that amount of funding, that amount of time available and so on. How can I sort of keep that heard in one place without having it like balloon out of proportion to it's good to get ideas and so on. But then if I can't do anything about that or I feel like it's a bit of a wasted process that I need to be a bit more focused. So just how to find that good balance, I guess. Thank you very much. Huge question, I think. And I think one short answer. It would be really interesting to discuss more on that because I think that this process and many of these topics leaves more questions in one way than let's. And that's why we are in its research in one way. It's bleeding edge research, as we call it. However, I think there are some interesting points in the process where we need to define what are the kind of absolute boundaries that is relevant in this process. Because what you're pointing at there is a tension, which is very interesting between on the one hand not being which is the very typical fallacy to just kind of marry one solution, one problem immediately and then just start developing. I think that's a very classic way of doing things. On the other hand you have that road construction situation that I showed where in Norway I think we have a shameful legacy of these projects taking 30 years of discussions and no one is breaking through and saying this is what we're going to do, we need to try something. And I think the answer lies partly perhaps in two things. One is the ability to define some absolute boundaries at the beginning. So what is the absolute end of the mandate here, for instance, because the mandate very often is too narrow, but if it's anything, then it's impossible to do anything. So what are some of the absolute boundaries that that is kind of defined before going in and rather possibly challenged later if that seems like a big hurdle. And the other element, which is this idea of interventions not being gigantic things, so that the planning process is not a 20-year thing, but that intervention should be breaking down into learning from this agile software development where you, for instance, have these minimum viable products. You know, I could say the same in the organization that has minimum viable interventions where we reduce our hunger into a very, very tiny thing to begin with and then learning from, okay, is this even working at all. And then building upon that because the reason I'm mentioning this here is that that can also help in narrowing down this enormous balloon process in the beginning that what we need to do in the beginning is not to decide on the course of action for 20 years. What we need is to see this as more of a learning system where we try something small, and then we learn from that and perhaps then we can go another route. But again, which challenges a lot of this project management practices where you're not really want to hear those kind of words being mentioned at all, right? So yeah. Yeah, thank you. Actually, you're making us to think. Sorry, I'm Andrew Mohide from Rwanda, Minister of Health. Thank you. Thank you for the good presentation. This is a very good initiative as far as I'm constant. You're making us to think beyond the implementation and I think for me, the social technical approach is the knowledge that we need for implementing the information systems, which makes me to think that maybe we need to come back to the implementation basics of information systems. But again, of course, there are some of the challenges when you're implementing this kind of projects. But maybe what we need to do is coming up with the steps. What are the standard steps that someone needs to use when you're implementing information systems. But again, when do we say that the system is successful? I mean, as far as the users are concerned. But one question that I have for you is some of the projects that doesn't allow you to link up with users. And some of these projects are many. Like Andrew, I think that I have discussed with users. Then we just come in the middle. Then you design while you are interacting with me. Then when you go the other side, you find users are not really happy with the design. They are not happy with even colors. By the way, users even colors, they say we don't want this color, that's it. So I think those kind of perspective would be, those kind of gaps would be addressed when we have steps. It means if I'm designing, I can just say for me to design, I need to link up with users, the end users. You are not enough to give me the requirement. But also I need to test up with the users. So that's why I'm supporting and recommending this initiative to cover the gap between us and the users. Thank you. Thank you. I think there are many things to comment on there. But I think yes, this user thing is important for several reasons. I think one is informing the design process perspective. The other is that when you are part of a change process and you are involved in that change process, it's also more likely that you actually commit to that change process, which is another dimension of it. And then there are a number of issues I think on, especially working on national scale systems in how do you actually organise all types of interactions and the kind of proxies that you typically would have in that process and what they are actually, you know, how far, close or far away are they from the systems that you're really working on and so on. So, but thank you for the comments. Huge room. Thank you. My name is Louise from London. Thanks so much. Very, very interesting. And so my question, thanks for sharing that story about the Norway implementation in the hospitals, because I think it reminds us that a lot of this is behaviour change as well as organisational change. And at the end of the day, we're all people with our own, as you say, opinions. I think I'm really interested in the paper digital kind of combination almost, because I think sometimes, I don't know, one of the hardest things I've ever found is that I need people to stop doing some of the paper systems and move on to the digital. And so you end up with this double system with all the old paper systems that no one will stop, plus multiple digital systems. And the whole system just is creaking because of the double burden. And so I just, I'm just really interested like how do you see that we can, if you like, manage that process, manage that process in a kind of human-centred person-centred way. Because I sometimes think that the baseline paper that we're trying to digitise is like 50 years out of date. And then we're trying to digitise it. And it's like literally sailing the boat and building it all at the same time. And then it feels like we're sinking. I just love your thoughts, thanks. I think the right number of things there. One is, which is more concrete, is I think the problem of mandates again and the problem of the design team or the team that is tasked to do something about this, you know, implement this system, where one very classic issue is of course that the paper has to be there. That's the rule. And then we are building computer systems in parallel, but we're not allowed to touch the paper for some reason, often because that's another department's job to design these paper forms and to do them. Then you have projects where you have a much broader type of mandate where present data rationalisation, the review of indicators and data elements is actually part of the process of moving to computer, which will of course be more of a social technical process because then you're also interested in the organisational standards related to reporting. But I think that this is a problem that I've met many times with, because also I think one of the typical ways of implementing these systems is that you have facilities that are not yet ready for the computer systems. So you need some way of having paper together with computers over time. But I think then the problem could be a problem of perspective. What perspective are we deciding from? Is it from the top reporting type of perspective? Or are we taking the perspective of how does the ecosystem that we have at the facility look like? And how do we support that? So for instance, this move that I showed in this one example, this one, that's not just a move in terms of problems, but it's also a move in terms of perspective in one way. So it's a move from the how do we work with the existing system that we're having, try to get data upwards through perspective on how does this look like from the perspective of hospital, pharmacy manager for instance. And this is also a critical issue because that's not easy to, how do you start, what perspective do you start with, how do you shift that perspective, and how do you account for different perspectives. Because of course the whole idea in the project here in the beginning was that we want better data quality upwards to the decision makers. And in this sense it was about hospital stocks basically, pharmacy stocks. Then the point was to make decisions regarding distribution of vaccines and medicines and so on. So while we shifted the perspective here towards more what does this look like from the ones that works at the pharmacy, we also had to keep the perspective on we need all that data reported upwards. And then we tried to integrate that into the same. So we made a solution that both takes care of the reporting and tries to also think about how do we support the work, which is perhaps one of the most important things that we try to do. But the work is very different, it's the work of the health managers that are going to make decisions on the top level. And it's also the work of the clinicians for instance at various levels. And the switching of perspectives I think is difficult. Because the project typically is funded from one perspective. But then when you go down you meet all these other perspectives, which are perhaps not even what you're funded or you don't have the mandate or that's a different program that is dealing with that. So yeah, a bit of a messy answer on that one, but I think that's, it's a messy problem, it's a wicked problem.