 Alright, so welcome to this session on the DHS2 design lab where we will talk about challenges and opportunities related to user-oriented design and innovation in and around the DHS2 ecosystem. My name is Magnus Lee. I'm a PhD researcher at the University of Oslo. I've been working with ISP and the DHS2 for six years or something now, mainly related to research, so understanding what is going on and connecting that to users and concern in the field of information systems research. Yes. Please, if you have any questions along the way, you can ask them in the chat. There is also this thread on the coming to practice where it's also possible to air any experiences or whatever. So in this session, I will first talk a bit about focus that we have in the lab around user-oriented design and innovation. Then I will specifically talk a bit about the three conditions that kind of affects the ability to design and innovate when the DHS2 is implemented in implementation projects. I will say a bit about some of the projects that we're working with. Among some things, we're developing some resources to support more innovation and then a few points about the future. So the kind of basis for what we're doing in the lab is that the DHS2 is becoming, as you know, and which is evident in this annual conference, that the DHS2 is becoming increasingly flexible to be implemented in very different types of use cases across different user organizations. And all of these typically would have some similar needs and some very different needs. And this is kind of both a challenge and an opportunity, because first of all, it's difficult to make one kind of generic software solution that works perfect across an extremely diverse and heterogeneous user group. So it's difficult, increasingly difficult, of course, to develop kind of one size fits all solution on the generic level of design. Also, as the solution is becoming more flexible, which is one way of, of course, addressing this kind of diversity, the implementation process also is kind of increasingly complex or complicated, requiring a lot of competence, both kind of technical competence in terms of how you configure the DHS2, all the different possibilities and opportunities that lies within the configurability and also the increasing focus on extendability, the ability to develop apps on top of the DHS2 to address specific needs. But also more kind of social organizational capacity, such as how you manage quite big projects since implementation projects are flexible and you can do a lot of things then it turns into kind of a real software engineering project with which has to be governed and negotiated and driven. And of course also increasing need of user domain knowledge in all these different domains. So those are challenges, but on the other hand the DHS2 is kind of turning into an infrastructure for what we call implementation level design and innovation. So with increased flexibility, all these apps that exist, both developed by the core team and third parties in the ecosystem, it's starting to become kind of a really infrastructure where you can do a lot of interesting innovations on top of it. And thus there is a great potential for doing more user oriented design innovation, meaning to build kind of useful tools for the needs of different end users based on kind of the specific needs within different organizations. So that is the starting point for our work. So, so one way of looking at it, which is part of our work also is in comes to kind of conceptualizing what we're seeing when we are exploring these topics is that all these resources that are developed to be generic to what we call generic level design, a lot by the core team but also about from a lot of different actors for instance in terms of third party apps documentation, all this is kind of could be seen as an infrastructure that supports the implementation processes in specific organizations. For instance, you can leverage all these resources and then configure and extend them according to the specific needs and again that provides an opportunity for more of course focus on designing useful, relevant tools for end users that are kind of novel and innovative. So the way we do this in the lab is that we do both diagnostic design and intervention oriented research. So we try to understand the current practices that works and what doesn't work in terms of designing and innovating in implementation projects. So we follow a lot of different implementation projects we focus groups and, and so on with different his groups and other implementation specialists. And then we try to explore resources through design so we design resources that are intended to support practices and we also participate in interventions trying to introduce new things for instance related to app development, supporting more efficient projects for instance or, or related to innovation methods for instance. Yeah, so some of the projects that we have have some kind of highlights we have participated in a lot of projects in India for instance, or more concretely two projects in India where we have explored challenges and opportunities in addressing usability. And we have had several sessions with various his groups on on the practices of implementing the edge house to and again the challenges and opportunities with kind of focusing on end user needs in these projects. And we collaborate with Norwegian Institute of Public Health in some projects in Randa where we try to do more kind of social technical innovations as we call it, so trying to not only think in terms of technology innovations but also in how do you kind of innovate around a social issue with with the kind of toolbox of not only apps or a specific tool but also kind of practices and other other types of interventions. And yeah, and we develop, collaborate with the core team on developing and testing the web of development resources and so on. And just to kind of break up here a bit. We are very interested in talking to more people within the edge house to community. So people that are have experiences with projects related to app development or projects that have been particularly challenging or successful in terms of user orientation. Please scan this QR code and submit your email address because they may be very interested in talking to you also if you have experience with agile software development related to the edge house to implementation. That would be very useful. Yeah, so returning to the presentation. I thought I would dive a bit into some key conditions for user oriented design innovation in the edge house to implementations. And this is based on some of these initiatives that I mentioned where we have participated in projects and been talking to different implementation experts of the edge house to trying to really understand what what does it take to to actually kind of build an infrastructure that supports design and innovation of useful tools for for for users. And there are particularly three conditions that are that are kind of standing out as very important in these initiatives and one is the affordable design flexibility as we call it ability to actually design things. You know, it features based on whatever needs and the practices that are present in the individual organizations. I will return to each of these more in more detail. Then we have the project configuration meaning that way projects are organized in terms of scopes and mandates and what how is the problem that the project is trying to address defined. And what types of methods are used for structuring the process. For instance, is the project kind of geared towards this more waterfall type of model versus agile will greatly impact kind of space for innovation and focus on end users in the implementation process. And then we have the implementation practices, which refers to the kind of usual way of doing things of the implementation experts that are collaborating with the user organizations that could be the his groups or, or other kind of independent implementation specialist, the way they typically organize their implementation processes. So, diving a bit into each of these, these aspects, and also then after this we will talk a bit about implications and what, what we kind of work on in order to, to try to address these conditions of it. So, starting with the project configuration, which is kind of an overall very interesting aspects of implementation projects, where these method typical methods for design and innovation that focus on end users such as user centered design, participatory design, and also agile software development methods. They typically promote building and improving systems through these iterative design processes, so not defining all requirements at the beginning and then spending two years building the system and then kind of testing it, but rather building smaller prototypes and then iteratively evaluating the usefulness and usability of the things that are being developed. So typically, starting with understanding, you know, the problem and the needs of users and practice practices, then formulating problems to kind of orient the design around, then ideate and prototype and evaluate and that goes kind of like a iterative process. This is, of course, very much constrained or enabled by how the implementation project is configured. So, for instance, what types of scopes that are defined, you know, what is actually being focused on here is it a very, very, very specific technical detail. Is it, you know, a large kind of social organizational issue. What are the mandates of the implementations experts. For instance, some projects that the people that are, you know, working with the technology, they are kind of working in silo and then you have someone else bringing in the requirements with kind of limited ability for the people building the technology to kind of interact with users. Their mandate might be only the technology which could be a constraining factor when actually trying to do these iterative processes. And then, of course, the expected outcomes of the project. If the project already is defined very kind of detail in terms of what is the output, then it's very difficult to work in this iterative manner. And when in this kind of mandate scope thing, a special kind of particularly challenging issue is this balance between the flexibility in the project. So, how flexible is the project in terms of kind of changing the expected output of the project as, you know, the project evolves versus the predictability of the project. Because, obviously, all parties are interested in some kind of predictability. So a user organization, for instance, let's say a Ministry of Health. They obviously when they are procuring and negotiating with his group or another implementation specialist group, they of course need some kind of predictability and what is actually going to be the output from the project. And also at the same time, the implementation specialist also needs to have some kind of predictability to avoid scope creeps and projects getting out of hand, which is a very typical challenge. But then on the other hand, if the project is kind of defined at kind of a certain or very specific in terms of the output from the outset, then you run the risk, of course, that you're building something that will not fit in the end when you have kind of started to develop. You know, the user organization, then you might discover that there are other issues. Technology changes over time. There are a multitude of example of projects where you kind of define the whole project and it takes four or five years to really implement it. And then when you do that, the organization have already changed and the technology have changed and what you're building is actually then, you know, tomorrow, yesterday's answer to yesterday problem type of situation. So, so this is a very interesting tension and they're interestingly also there are some groups that are employing interesting strategies for dealing with this tension. So, for instance, in Mozambique, they, they are working quite systematically with kind of trying to identify opportunities as the processes are moving so there is a defined project and so on but then as they identify opportunities, that could be kind of prompted either from the organizational side that there is a need that have no solution yet and then there is a potential for building an app, for instance, for that, or also the other way around, there is certainly an app available within the DHS2 ecosystem that has a potential impact within an implementation project and then there has to be of course a process of kind of renegotiating the project configuration so that those innovations could be catered for in the project. Others are, are actively working to kind of build in some sort of spaces in the implementation project so that you know requirements could be renegotiated, but still keeping some form of predictability. And also, you know, there is our significant differences in terms of is the project defined around organizational goals or is it defined around specific IT solutions. So some projects are more defined towards you know, this is the organizational outcome that we want to have. And then that is actually what you're measuring kind of in the end. And then there is more flexibility throughout the process to define the technology as you learn about organizational practices and iterate throughout the process. Another interesting challenge, which is also well discussed in academic research is this tendency that many projects are defined as IT projects, while sometimes it might have been more fruitful to define it as an organizational kind of project. And this of course both affect the scope and the mandate of the project. So if the project is defined as an IT project then there might be limited ability to change for instance practices and social aspects, which which might be more or maybe more ideal than actually changing the IT solutions. And there are several interesting experiences from that in some of our projects where, for instance, we have to kind of build apps which could have been avoided if we were able to rather, you know, modify some of the suboptimal ways of working. But that is beyond the scope of the project. And then, you know, you have to deal with building and maintaining a lot of technology that you wouldn't possibly not need if you had the mandate to work a bit broader on the, on the, but then on the other hand that will of course require much more kind of trust between the user organization and a lot of more capacity in on the implementation expert side if you're interested with not only designing the IT solutions but but the practices and the routines within within the user organizations. Let's see. Yeah. So just interesting example, related to the project configurations. So, one project that we have worked with in the past, we have collaborated with one of his group on on commodity consumption and trying to design a system that that supports on based on DHS to that that supports the reporting of consumptions of commodities within the public health system. And in that project has been kind of an evolution with multiple ways of framing the problem that we have been focusing on, which provides some examples of you know how important this problem framing is for what you're actually looking at what you're looking at what you're seeing within within an implementation project. So, the first, first problem framing that we kind of started out within the project was how we can support health commodity reporting with DHS to. So, technology is already kind of part of what your defined problem. And what it basically then resulted in which also proved to be quite useful for many actors was, which the kind of typical DHS to implementation project where they have these paper based systems, which were then set up configured into the DHS to moving it from pace paper, paper based reporting to reporting in this for many of you very familiar data entry application. And then that worked quite well for many of the data output could suddenly be used, you know, in the DHS to with with analytics and different types of presentations, which improved kind of decision making around around the commodity but one group that did not kind of benefit from this transition where the people that were entering these reports. So they actually reported that you know, the paper based the reporting system were more stable, easier to use in terms of usability because this was quite a heavy kind of screen to enter data into and so on. So then we had a project on friend more around how we can make the data entry more usable and relevant for that. For the data entry clerks and hospital store workers that usually filled out this form on a by monthly basis. So then the framing were more kind of explicit around the usability and relevance for a specific type of user group. And of course then also the project were designed more around, you know, specific methods for for for understand the practices and challenges of the users, trying to prototype and iterate and build, build something that could be more usable and relevant for this existing form and then what we ended up with was this data entry kind of dashboard that show reports and deadlines and stuff. And then we have this stepwise data entry thing which also validated numbers as you kind of filled in each row, which here you had to kind of go back and then try to identify it in the form where you made a mistake. And that somehow increased, you know, the usability of the solution for these data entry user group. However, when we did that project, then we also saw some interesting signs of, you know, there is a potential here for maybe doing this also even simpler. So we also carried out an exercise friend more around how we can better support the work of the hospital medical store workers. So in, you know, in both of the farmers the tool were kind of defined in advance that okay we want to to use, here we have the DHS two thing, and we know, you know, we have you know that recent data entry and that is what is going to be improved and that's going to be made you more usable and relevant. So in the third problem framing we tried to be a bit broader and formulate the project more around, you know, how do we, how do we better support the work of those medical store workers which obviously was a bit more open ended. And we then ended up with with exploring different solutions around commodity dispensing. So that's what we saw that, you know, rather than entering this by monthly report. We could instead register the commodities on on the dispensing, which then previously did on paper in these talent sheets, and then that would actually generate these reports automatically on the later points, or every, every other month. So is that this, you know, the way you're framing the problem, the very type different types of, of kind of solutions you will possibly end up with them. Again that have, you know, could be enabled or constrained by that the way the projects are configured in terms of scope, the mandates and the formulated problem that you kind of enter into. And I think there is a lot of potential in many of the DHS implementations, which I think is, you know, a great great opportunity that there are so many implementations in different countries and with the ability to develop apps and and so on. It's, there is a lot of interesting innovations that can happen on top of this already established, you know, the classical kind of data entry regimes such as this commodity dispensing module that was developed as a, as an app. Yes. So, a bit laggy slides here. Moving to the next condition. So now we talked a bit about the project configuration as one important kind of enabling or constraining elements for designing and innovating around users in implementations. A second condition is these implementation practices and with practices you mean that they established ways of doing things. For instance, within his group or another implementation agency. And here are also big differences in how people, for instance, typically structure implementation projects is also of course varies from project to project related to project configuration, where some are more active in pursuing kind of an agile way of working in in implementation projects. And some are more oriented towards the traditional waterfall model and some are more what you maybe could call pragmatic which is more kind of doing what works best at that point in time without too much kind of definitions around it. There is also no great differences in the both the capacity awareness and motivations for working with methods as part of the process. So some groups, for instance, in Mozambique are very kind of eager to promote the use of user oriented methods. Malawi also tries to typically negotiate, you know, user oriented design or users have to design methods as part of kind of the project mandate structuring process. While others are trying maybe to avoid that as much as possible and rather, you know, working with closely with management team, project management team within the user organization. But of course also depends a lot between different projects. And interestingly also, you know, the perspective on end users and why you want to work with them is somewhat different also someone see, you know, it's kind of elementary and extremely important to see the users as someone to learn from and then try to support with technology while someone's system maybe more as someone to kind of fight with them try to convince to use something that that they maybe don't want to want to use. Interestingly also is this software engineering maturity in different different groups also so some are more maybe cautious on, you know, how do we work in implementation project are there any improvements we can make. Is it possible to introduce new types of techniques and methods for managing our implementation projects. Yeah, so there are there are some differences and a lot of interesting examples of good practices which is some of the stuff that we want to document and kind of share with the broader community in the, in the design lab. To highlight two main things that we are kind of concerned with is the, the obvious one which is kind of the capacity and motivation for carrying out more user oriented design and innovation activities. There is a real capacity of, you know, contemporary methods for for working agile and close to users and kind of building this fruitful arena for innovation, but then also interestingly the capacity and motivation for negotiating for also this one is to carry out methods for each other. You know, as seen with the project configuration there is also a lot of negotiation and, you know, a lot of responsibility for organizing good implementation project is of course on the implementation specialist it has to implementation specialists. You know, some groups are very active in negotiating for, you know, a fruitful arena for thinking about users and deciding for them while while others are not that active in that element so that's also very interesting and important element. See slides again a bit. Yes, so turning to the last of these conditions. We talked about the project configuration and implementation practices. And then a final conditions which have kind of a large impact on the implementation process is that what we call affordable software design flexibility. And with that, we refer to the ability to actually respond to specifically user needs. So, in many implementation projects for instance, some of the ones we've been involved with in India. Then we, you know, we find a lot of interesting things that we could improve, but then it's not possible to easily have configured that in the edge as to. And often also if the issues are not, you know, enormous, then it's not considered viable to develop an app for it. And the key principle in this user oriented design methods is of course that the needs of users should kind of inform the design innovation of all five solutions. So the, and in the edge as to you have, obviously is configurability that you will find in the maintenance app. This is this is data and reform builders and so on, which is beneficial because it's it's relatively easy fast and have limited maintenance costs tied to it. It's kind of automatically transferred when you're upgrading the software and so on. And then on the other hand, it's, it's limited to metadata data and reforms and some kind of switching on and off functionality or choosing between different apps, which is of course becoming richer and richer as there are more apps developed by the community. And the app development that is kind of monumental change in the edge as to system that you have the ability to develop apps for in principle any kind of need that you would discover. But then on the other hand that requires software development competence, it is costly than in terms of the immediate, you know, development time and competence. It's also something that has to be maintained on that individual implementation, if not kind of generified into a generic app. So one thing that we find interesting is that these generic features and the adaptation capabilities, the configurability and the extendability acts in many cases as some sort of a lens in the implementation projects. So if you're very experienced the edge as to implementer, then of course you know a lot of what is possible with the edge as to and you would know what would be difficult in the edge as to because it's not part of the configurability or the existing generic features. So that means that in the implementation projects that the generic features and the adaptation capabilities have some sort of kind of a gravity that that you know pulls the focus towards what is readily available and leaves things that are not that readily available in the dark. And that's why I think this application development resources. SDKs, the app development platform, the DSH2 UI library and so on are very important interventions into that kind of DSH2 ecosystem because it supports more affordable design flexibility. You know with app development platform it's these components for instance they are still maintained by the core team and then you kind of assemble them as an app, meaning that in some point in the future you could imagine that you know there's only kind of the glue between the components that are maintained within the specific implementations, yet you are able to kind of combine different components and build very, very different user interfaces and new types of functionality, but still with limited maintenance on the level of the individual organization. So kind of trying to look at what this means is that you know thinking about different kind of forms or scopes of design, so if you know design focusing on usability for instance which is from form of user oriented design and user interface phases will of course have some kind of demands on the project configuration for instance you know there has to be defined that usability testing, involvement of users in evaluation and so on has to be part of the process. It obviously will have some impacts or demands on implementation practices and capacity in that you need methods and capacity for negotiating those into the project and conducting or carrying out such as methods needs to be part of the practices of the implementation experts and it has some effects on the software design flexibility or demands on the software design flexibility and that you have to be able to shape user interfaces, so that they are you know usable for the user groups that you have. And these issues could be quite challenging sometimes, one project that we worked in for instance where they wanted to refer to data sets which is a common term in the HHS2 as something else, but that is not something you can configure very easily. So that means that you have to you know, rather than either then you have to build new apps for everything which is costly and not a viable solution, or you have to retrain the users to adopt a different type of language for what they're working on, which might be the best choice in that situation since modifying that part of the HHS2 is very difficult. If you are, you know, have user-oriented design projects focusing more on IT tools, you know, innovation of tools that are relevant to end users, that of course again puts increasing demands on project configuration and the practices and the software design flexibility. And again here the apps, you know, the ability to develop web apps or Android apps is a great, you know, step in that direction that it's possible to extend the solution with functionality and user interfaces beyond what is offered in the generic solution. But again that also is tied very much to, you know, the capacity and how projects are configured. If the project is configured with the solution up front, before you kind of go into the practices and needs of users then it's not very easy to of course include those aspects as part of the process. And then you have kind of the most extreme form of user-orientedness which focus more on the social technical system, so designing IT and practices in kind of tandem. But then again with even more kind of demands on the project configuration and the software design flexibility and so on. Another thing I will say about these conditions before we turn into some of the implications and the work that we're doing in the design lab is that, you know, these aspects are interesting also interdependent so it means that, you know, the way projects are configured are also affected by what is kind of the affordable design flexibility in the edge as to so during the initial negotiations of project configurations. Then of course, you already start typically talking about what is possible with the edge as to what is the design space what are the generic features that could be relevant in this project. The generic features and the adaptation capabilities is of course part of the negotiation in the beginning and now that the HS2 is becoming more flexible, easier to develop apps, more generic apps available from the community then of course also that will have an impact on the negotiation of the projects if done kind of correctly. And of course also these project configuration is something that is going on typically for many years and as there are new opportunities emerging in the end of the HS2 design infrastructure, new third party apps, new features provided by the and of course that also an impact and potentially if you know that infrastructure is monitored by implementation experts then that will lead to more new opportunities. Also interestingly of course the project configuration affects what you can do with a given flexibility so if the project is defined around here there is no room for any application development for instance then of course you are not able to utilize that app development flexibility if it's already agreed that we are only building this system using the generic features and configuring them. Similarly, the way projects are configured will both be affected by the implementation practices you know whether for instance his groups or others are negotiating for a user involvement and user orientations in the projects. And also the other way around what has been agreed upon in the project configuration also affects what is possible to do, given the capacity and with implementation groups. And also of course the capacity to develop apps affects you know what is considered affordable software design flexibility that way. And the implementers are constantly of course affected by what is available within the design infrastructure with within the bounds of the HS2. So that's why also the type of work that we're doing in the lab try to kind of address several of these aspects because they are heavily interdependent so if the available software design flexibility increases then of course also it's easier to promote practices focusing more on flexible design and also promoting negotiations of broader projects and vice versa. Yes, so turning from the these three conditions. I will also then in the end of my presentation talk a bit about what we are working on concretely around building resources to support design innovation practices given these conditions that I just talked about. So, one, one thing that we are working on and where we have several master students at the University of Oslo, both working on kind of content and format is this design innovation toolkit. So the toolkit is intended to support and promote methods and techniques and experiences from different related to different user oriented types of way of working. And it's currently I'm kind of very early beta but we have now students working on it for instance this summer to, to add the content based on both kind of contemporary methods and techniques and also experiences from the DHS to community. So here you can see a screenshot also from an idea is that you have activities and techniques and that you can search for for different techniques which are kind of specific ways of doing things for instance interviews or observations or stakeholder mapping, or you know, analyzing, you know, existing systems and practices, and then we have activities which are more kind of groups of things so that could for instance be prototyping. And then prototyping have a lot of different techniques that you can use to prototype for instance wire framing, storyboards, paper prototyping, and so on. So the idea here is that we will have different types of, you know, things that are relevant in in implementation projects, and then that can be used in projects to find, you know, contemporary ways of are suitable, which is the most important ways of, for instance, working with users or organizing innovation projects. Another activity which, which is interesting is ideation, for instance, so what kind of techniques can help someone in the design process in ideating and also analyzing for instance the challenges of specific user groups and based on that prototype solutions. And then you see that this is also screenshots from some of the early version of the, of the, of the method toolkit, where there is a description of for instance how you do as of now are in descriptions of how you do do interviews and what are important concerns in that process. And the idea also over time is that this, we're trying to build this as a kind of a flexible thing where you easily can add new types of activities. So we have, for instance, a team exploring agile software development principles for implementation level design. And then the idea is to actually, you know, build guidelines principles techniques processes that we can add into this toolkit. So we could have here and kind of agile software development. So what you want to report with and you can select agile software development. And then there will be experiences from projects, you know, common challenges ways to overcome them. And then there will be principles again techniques that can help in different phases of an agile software development project. And also similarly, for instance we have one activity we should call an open innovation project, which just kind of starts with the open ended problem area and then you kind of iterate to innovate through prototyping evaluation and so on and then there are different techniques tied to those different problems. And the hope is that we can work closer also with with people in the community, his groups and others to try to learn from this, how they kind of successfully for instance as I mentioned, are able to negotiate the right mandates, the right balance between flexibility and that that is something that can be shared in this toolkit, for instance under negotiating an open innovation project as kind of an activity with some activities and techniques tied to it. Yeah, so, so that this is kind of an early basis for for a lot of resources that we hope to be able to build together with the DHS community of practitioners and users and so, and we particularly and that's also why I'm promoting this, this, this form that I shared the link for earlier or sending an email. If you have again experiences with you know implementation projects focusing on users, both challenges and things that have worked well. So that we also might be able to share some of these experiences within this, this toolkit thing, you know, for instance under what are common challenges how the people overcome them, how are people achieved you know successful, you know, how are people in the community of innovating together with users, call designing, and so on and so. I see there is also a question in the chat, where can we access these two kits, glad to that some of our interest in the masks. So this is, this is at this point it's a beta but we hope that in August, we will be able to have something that we can actually share, both for testing but also hopefully for use. I'm happy also to, to, I will try to deposit on the community of practice when that time comes, and also Maria if you send me an email, and maybe we can stay in touch and I can also share with you when it's, it's available for testing. Yes, so that is the desired innovation toolkit. Moving to another resource that we have been working on a bit. This is also tied to, because the design lab is tied to also a course at the university also a big course with 100 students where they have a project where they develop the HS2 apps. So they have kind of a design and innovation project building apps for the HS2. And this, this is something that we also use in that course this design innovation toolkit resources, which we're also going to test in that course in their project in the fall. Obviously a bit of artificial kind of project configuration there as compared to real implementation projects. Anyway, very useful with of course methods and techniques for how do you analyze, you know, the knowledge about users, how do you ideate how do you prototype and so on. And another resource that we have been developing in relation to that course is the HS2 web app development course, which is available online but which also is a beta version that we are this summer we have several people working on improving the content of that course. And this is that over time we want to develop it into a kind of full self-paced the HS2 web app, the learning resource for web app development, which we can use both, you know, within the HS2 community but also with our collaboration with universities in other countries. Yeah, so what you see from the screenshot is that you have, you know, learning kind of the whole stack of technologies that you need to learn to be able to develop with the HS2 app. So while the core team provides a lot of, you know, useful guides on how to do specific things with the HS2 app development platform and the APIs. This takes you kind of from, you know, essential knowledge about front-end web development to JavaScript to react, which is the kind of preferred JavaScript framework for developing apps for the HS2. And then, you know, to the to the HS2 specific how do you set up the environment for developing the HS2 apps, how do you communicate with the API and so on. So, and then there are some exercises. So in our course at the University of Oslo, the students there, they use this to kind of get from not knowing potentially any front-end development to be able to develop the app through a project with fellow students. So again, also if you have experiences with web app development or want to learn web app development, we will be very happy to talk to you on testing this resource and seeing, you know, what could we do better or differently in terms of building capacity for the HS2 web app development. Yes. And finally, related to this issue that I mentioned in terms of how, you know, how much capacity and how much awareness are present in terms of how you structure implementation projects as agile or according to agile principles. We also work on, or have set of students working on principles guidelines and techniques for organizing agile the HS2 implementation projects. And hopefully these resources are some of the stuff that will feed into this design and innovation toolkit, because agile is kind of quite closely connected to this flexible projects where you can innovate and decide. So yeah, again, if you have experiences with those type of aspects, you know, agile challenges, opportunities, successful stories of, you know, working agile in the ISP group or in implementation projects in general. And it would be very valuable also to talk to you on, you know, your experiences, possibly, you know, documenting and feeding into this toolkit resource and to our further research. Yes. So rounding up, I just want to say a bit about the future that we're working on and what we are seeing kind of in the future. So our focus is kind of continuing on the understanding these practices and processes related to design innovation within the HS2 ecosystem and what type of resources that may, you know, enable and constrain these practices. And as I said, you know, we're very interested in talking to many of you on exploring the unsuccessful or successful, you know, ways of organizing things and the challenges and opportunities related to working with users and co-designing and so on. For instance, just to mention last year we had, I had a session also on this topic at the conference and then we actually initiated contact with one partner and implementation project and we now have master students participating in that project, trying to work on issues such as user involvement and how do we frame the project to kind of maximize the potential for useful innovations and so on. So please reach out if you have, as well as there were some mentionings in the chat which I thank you for, I will try to contact you on that. Yeah, and then as COVID hopefully will be calming down at some point then we also hope to get back to more, you know, participate in projects in different countries. So, for instance, then using kind of app development projects as engines to try to explore how we can design and innovate on the HS2, learning both about methods and techniques and about, you know, app development issues, opportunities with app development for the HS2. And we also have a lot of these interesting plans for collaboration with universities in Mozambique, Malawi and Tanzania, kind of exchanging master students and building capacity, not only with kind of well educated and finished students, but also people working in groups, but also kind of working already with universities and building capacity on these kind of things that can later feed into our work in the industry. So yeah, rounding off just again urging you to either send me an email if you have any interesting experiences to share or some kind of proposal for collaboration or scanning this barcode and filling out this very, very brief little form including your email so we can stay in touch if you find any of these elements that we've been talking about interesting. Thank you very much for your attention. I hope it was interesting.