 So, hi everyone, I'm going to be talking about the work that PSI have been doing in using model as a backend, but delivering the model courses using shell bots. But before I go into the technical side, I want to give you a brief introduction into the work that PSI does, so PSI or Population Services International works in over 50 countries in different programs relating to health. So we work in what is called low and middle income countries, Central Africa, Sub-Saharan Africa, Central America, Sub-Saharan Africa, and the Saudis of Asia. The work that we do is all along the diseases with a higher level of burden, so we work a lot in malaria, we work in HIV, we work in sexual-reproducted health, and a lot of the activities that the PSI does have to do with the delivery of services, typical medical services in terms of testing, prevention, treatment of those illnesses. But we're also working in digital health, and it has to do with the way that we approach to the consumers, because there is a lot about letting them know that there are services that they can access, because in many cases they don't know that they exist. We work with local businesses, we work with governments that mainly translate into ministerial health, so industry partners. But when we talk about the way that the digital health gets delivered at PSI, there is a lot of consumers, we do communities that have influence, but in this place of e-learning is when we get to that central space that we would call the workforce. What I'm referring to is the frontline health workers, is the people that either we refer to them as community health workers, they work directly with those communities in either educating them about the availability of resources, or in some cases doing some rapid testing for HIV, rapid testing for malaria, and when it was appropriate rapid testing for COVID, we work with doctors. It's obvious, but a lot of nurses and midwives are the ones delivering the health services, and we work with pharmacists and lab technicians. And that is the people that we are trying to reach through electronic education. But you will think, okay, you have something that is called the Moodle Lab. We've been talking about, and we know how well the Moodle Lab works in offline environments that is, in the majority of our cases, is super important, because we do have problems with access to broadband and the signal is synchronized in certain places. But the reality is that many of these health workers are not necessarily familiar and comfortable in using an application that for us students in secondary or tertiary education that they use every day could be simple. Instead, we're trying to build them in a technology that is really familiar to them. And those technologies, we have determined that are things like WhatsApp, are things like Facebook Messenger. Those are applications that are already installed in their devices. We don't have to go through what is actually a quite complicated process of getting them to download the app, install the app, put a URL, and login. That becomes a major challenge, and then it becomes a problem of trying to train them. In most cases, they are remote to us. There is over 20,000 to 30,000 community health workers across all the countries in PSI, the North SES, and doctors, and it's not possible really to reach them to get them into a room and get them into the step-by-step of how to using the model app. So what we've been experimenting, because this is not yet in a big scale, is the use of chatbots, primarily WhatsApp, on delivering the courses that are set up in model. Another reason that this actually works in many cases is that in many of these countries where you get your airtime, many people don't have contracts and they have payouts you go, and in those payouts you go, they are bundles that are subsidized by Meta. So you have access to things like WhatsApp, Facebook, and Facebook Messenger without impacting your one gig or whatever you have bought. So that's another important thing. Obviously the way that we do this is we render the model course in a chatbot-based conversation, and we're going to be seeing this in a second. So the way that it flows is that, I mean, once you get into the telephone number, you call into the number and just say hi, is if we see that that user is new, is, well, if it's not new, obviously we already show the courses that are available for enrollment or that he or she is already enrolled, or otherwise we get the user to select the language. At the moment we have French, Portuguese, and English, and show terms and conditions, obviously you need to agree. We do the user registration, show the course, use like a course, show the modules available on those courses. We support very few activities. At the moment we have mainly implemented the lessons to display the content of the course. We support quizzes and we support feedback. We're going to see a full demo in a second, but basically the user can go through the chatbot, make progress across those activities, eventually come into the completion of those activities, and hopefully the completion of the course. So let's actually look into more detail. We have an animation in a second, but let's look at the five kind of key components. So user registration, we keep it very simple. It's first name and last name, we make email optional, and then we use the WhatsApp number or eventually when we do Facebook, messenger to Facebook, conversation, the scope page ID is as the identifier. So that becomes the unique identifier of the user. Email is actually optional. We set the language based into the international dialing code of that telephone. We can say, oh, this person is in Mozambique, so he must be Portuguese. This person is in Tanzania, so he must be English. Next thing is there is a limited number of courses that we mark as chatbot compatible. So we show to the user, okay, these are the courses that you can enroll or you're ready to enroll. You can take those courses, we show progress if you're ready to enroll. So lessons, what we do is that we deliver the content of the text. Obviously, you need to design these courses in model thinking that they are going to be rendered in a chatbot. So your paragraph needs to be concise and short because they are going to be coming as text bubbles in the chatbot. You can use images and you can also use videos. But if you use videos, they are likely to be very short. You have to be very aggressive with your compression algorithms because particularly in the settings where we work, there are challenges with the bandwidth. But certainly images, JPEGs, but our recommendation is to be very aggressive with the compression. We show every time we show, you know, where are you in your progress within this activity within the course. Quizzes is fundamental to track the progress of the student and to be able to measure knowledge retention. So I'm talking about our community, our workers, our nurses, doctors and so on. We support currently only multiple choice and true false questions. They are automated, so we're not going to do open text. We always show the last version of the question. We support multiple attendments. And as always, we always show at the end the progress. And we also have feedback because at this point, it was very important for us to be able to understand how comfortable are they feeling with the use of this format for delivering knowledge, for delivering education. Let's look at the full demo, these last three minutes. So the control is there. But how do you get it started? Okay. So somehow we get you into the bot, either because we send you a message or through posters with QoRs. And basically the bot in this case is for a new user, you start with the process of creating an account. And what you're doing is creating an accounting model. So we check that you want to continue this conversation in the language that we have detected based on your dialing code. Show your terms and conditions. You are expected to accept that otherwise we end the conversation. And we proceed with the registration that, as I say, is very simple. We ask two questions. What is your first name? What is your last name? So in this case, it's Victor Mendoza, Victor is sitting there. And he's one of the developers. And then we ask for your email, but you can actually bypass this question. And fundamentally what we have done, we have created a user registration on the back end. So from there, we show the course offer. What we have is a custom property on the courses that you need to mark that as a chat compatible course. So we render here, you select your course, you start your course. And then we're going to be delivering the content on that course. So obviously we are developing simple content. The each paragraph is actually being rendered as a message bubble. If there is an image, it will come in as part of the chat. You continue, you have to go through all the pages. And eventually you will come to a point that we mark completion. So this is later on very important for the statistics about when did you start, when did you finish, how long did it take to conduct that. So the second part is quizzes. This is fundamental in the way we do a lot of pre-testing and post-testing to try to measure the knowledge retention or the change in behavior. We support through false multiple answer questions. So this is how the quiz is defined at the right, how it's defined in model. At the left, how it gets rendered. You get your results. Actually, you can also retake the test if it's defined as such. And then the last activity that we are supporting is feedback. We support only through a simple selection and open text. And actually this has been very useful for us to collect feedback about this way of delivering courses. So at the end, I mean, you structure your courses probably with pre-test, post-test, multiple content in the middle. But you fundamentally end with a full record of a student where you actually can track his progress and his output throughout. So let's actually have a quick look just to some information about the some of the things that we have done and the results. So for example, we did this on the Ministry of Health in Kenya. And it was about contraception choice is about the how health workers recommend and do the counseling session with a woman in terms of offering contraception. So we managed to enroll 3,000 users of which we got a 69 completion rate. So that means that people took the original test, they did the whole content. And then we have a final test. And more recently, that year is round is 2022, is with COVID also in the Ministry of Health in Kenya. And in this one is that this chart is interesting because it shows the results of the pre-test. In blue, the people that have the correct knowledge before taking the course, you have in yellow the ones that did not necessarily have full accuracy on the on the knowledge around COVID-19. And the health workers that have very little correct knowledge of the different elements that are important in COVID-19. As you can see after the course, we have a significant evolution on on terms of the we reduce the number of people that we increase it in a significant way the people with having the correct knowledge around COVID transmission of COVID testing of COVID and so on. So how does this work? So we basically at the moment are using Twilio as an intermediary. Twilio is a communication service that you can use for WhatsApp for some time was one of the few providers of WhatsApp. But now it's possible to connect directly to the backend of WhatsApp directly with Meta. It wasn't possible before and we used Twilio. Basically, we have an application that is written in C sharp using the Microsoft.net framework. That basically talks back to Moodle. We use the standard APIs of Moodle, there was some functionality that wasn't present and we needed to augment it and we did a plugin that basically have functions that creates new endpoints that we can use for the functionality that we need. That plugin besides having these endpoints also stores session information. So that allows users to go back to the to the chatbot and be exactly in the same point that they were the last time. So even if the session have expired you can resume your conversation at the same point and obviously we also have some system logs for troubleshooting and so on. We also store some files as part of that plugin to have all the multilingual prompts that the pod has and support the multilingual piece. And as I mentioned we add two elements, one to the user profile in terms of we have a country dialing prefix and we also add to the courses a property to be able to know if of all these courses that you have in your Moodle listeners is this one of those courses that I should render on the chatbot. So in terms of the installation I mean the plugin is downloadable from GitHub. We are publishing this as open source so we just upload it into Git and we have done our best effort on creating documentation if other ones to explore use it. The application per se as I mentioned is a C sharp dot net application and you need to run it we have it available as a docker container so you can run it as a docker container or you can install it if you use if you are familiar with dot net you probably can install it natively. There is yeah we do have a lot of documentation now. Limitations is at the moment we only have the connector to WhatsApp through Twilio. We're planning to change it to have the direct connection with Metas API so we can do WhatsApp and Facebook directly with in the business API of Meta. The plugin is working in our Moodle 4.1 we already know that it has a couple of things that we need to change for 4.2 but that's one of the things that we're planning to do. We want to explore to for example when you use Twilio you don't have access to what is called quick replies so that's the reason that we present most of the options as press one press two press three but when you actually use the API from WhatsApp you can actually do what is called quick replies that it would be easy for interaction and actually the translations we don't have a UI for that so that's one of the things that we plan to do. We support a plugin there is a plugin called custom certificate one of the people like at the end of the course is to get that certificate and but that's something that we didn't put on the public release because it's kind of an adaptation that we did from that other plugin. I have the impression that this is it and I don't know if we have any questions. We have 10 minutes for questions so if anyone has some yep I see one over there. My question is what is the model mobile teams point of view regarding to this development just to enforce what you have done if they approve or sustain this. We have no idea. So here I will be very happy to hear the reaction. So there was the the previous presenter about testing I don't know if he's still in the room. Yes yes I mean you're asking me what is the opinion of the model. So the question I don't know if the model development team is here is there anyone from the model development team here. I am I have no official comment one way or another but what I will say is when I saw this proposal come through for the submissions I was very excited to watch and be here and listen to it. It's great it's really cool a really novel use case. Tim I had a feeling you would say hello. Well what I mostly want to say is this seems absolutely awesome in my own life my interaction with chatbots is very very bad customer service and I can see this is actually a chatbot that does some good presumably mainly because it's being used to deliver high quality teaching and I think your reasons for having been doing it with a chatbot and not model mobile app made perfect sense it's what's needed in your context. Yeah I should probably ask a question and I can see you already have a very full roadmap but at some point in the future do you think you might support some additional question types for the quiz. Yes so we will explore that I mean at the moment yes I mean we want to explore other possibilities at the moment our focus is more around enhancing the usability so that's the reason for the quick replies and using Facebook Messenger using the API because the cost but yeah certainly the idea would be not only other quiz other questions types maybe even explore other activities when the time comes is is we at the moment yes that's and if I'm allowed one other question actually did you hit any technical difficulties making this work with Moodle sorry I'm speaking as a Moodle developer are there things we should be doing to make it easier for people to create solutions like this. Well I mean we're very new to the model I mean I use Moodle previously when facilitating workshops as a teacher and so on but never development and it's only been in the last 12 months that we've been getting into the into the Moodle war so the API give you a significant number of functions but no everything but then it was incredible to realize that you can write a plugin and create more functions and create more endpoints and expose whatever you want and create a table and store whatever you want so so in that sense although the API does not expose everything they give you the tools Moodle gives you the tools to basically build anything on top of Moodle so so that that gives you a lot of flexibility we believe that this has to be a completely separate app because it's running completely separate and connected to so I can nothing right now of telling you what else is near that no well I mean excellent doesn't mainly just brilliant work and it's it's nice that Moodle is contributing to something that has clearly having a positive impact in the world so thank you very much and thank you for sharing it with us today. Can you tell us a little bit how you enable teachers to create courses and what could be done to increase the number of courses that are fitting that format yes I mean I guess I mean is that the at PSI they they have been developing content for 50 years and in many cases it was printed copies with illustrations for community health workers so it's more than institutional development of courses rather than teachers so we don't have individual teachers deciding okay I'm gonna do my own course it's more an institutional course with created content that it had been from a medical point of view quality assure that they have the correct content and it gets tested previously before delivery so so we don't have any cases of allowing anyone to just go and say I'm gonna have this course available this is normally the department of the sexual reproductive health already have a curriculum and they will modify it into a shot compatible course and then if the malaria if we have a course in malaria testing there is already a protocol and in some cases is the protocol for that country that is mandated by the ministry of health so in that sense it's quite close what we're doing because you have to be curated and unvalidated from a medical point of view they they I believe I mean I am not directly on the delivery of these courses but I I think for some cases that evaluating to to to run the course and to use the that as a credit towards your professional qualifications as a continued practicing nurse or community health worker I have a question also we are trying to reach how to reach groups in global south and tried it with IVR solution as an S based training but not yet with a connection to Moodle which would be very beneficial for us if you have the direct connection to to what's that for example to the API is there any costs involved because we really have to scale and the solutions so far were really a cost effective for us yes so so in that sense is a Facebook messenger has no cost at the moment this is changed all the time yes one day is free the next day is not free and in the case of WhatsApp is there was kind of you we were paying 0.001 per message with Twillio but with WhatsApp you get 1000 conversations per month when you connect directly to the API of WhatsApp and after that there is an standard price but if you are a non-profit organization you may negotiate with them and they give you an exception for 12 months and then they will renew it or not so so yes they are cost involved but they are minimal yes so they are quite cost effective and this is constantly changing is do we have any more questions just like how big is the PSI group yes so so so PSI is is active okay yeah I repeat the question so the the question is how big is PSI so so PSI is based out of Washington DC they operate in over 50 countries and in each country the operations vary I mean in my mind is one of the largest operations that is over 3000 clinics with 7000 health workers but then you have other operations where you may have only 20 clinics and you have a workforce of about 150 health workers but then again the idea is that PSI is trying to move also into the public sector and there is collaboration with the public sectors so in many cases what we're trying to do is to move that expertise that knowledge that those techniques into ministry of health so there is a lot of work in collaboration or directly with the ministry of health on those countries just be aware of the Moodle trademark because what you have done it's it's it's a great thing so if probably in the future this might be just using Moodle chatbot this might be a conflict or infringement in the future just maybe I'm here the lawyer the devil's lawyer here okay okay the the legal it does redirect you into a confluence page where we have the the full documentation of the of the word on that's that's all we have at the moment okay but I will check with you now you tell me more about that otherwise feel free to get moved to the Moodle stand and ask for someone higher up okay because our lawyer loves stealing with copyright