 So, hi everyone. My name is Alice Akelova. I'm working as academic coordinator within his PIO, University of Oslo. So welcome to this session, which will be mainly focused on digital academies and trainings. Another thing that I wanted to let you know is that we will not take any questions and answers during the session. However, Mathieu is going to share now in the chat the link to the COP. So we will encourage each of you to post any questions you may have during the session on the COP and each of us will definitely reply to you. We will also make sure that we continue the discussion even after the session. So welcome to everyone and let's start. No, sorry. Before starting, I would like to present my co-presenters, obviously. So the session will include a first presentation from Mathieu Pinard and myself on the process we have followed to turn all our regional on-site academies into digital academies. And then the presentation will be followed by another presentation from Shayna and Paige for PEPFA. They will basically presenting how to build a flexible virtual solution for trainings. Then we will hear our colleagues, Sandy and Lynn from East South Africa, who will be presenting how to turn on-site trainings into digital formats. And then the last presentation from Arthur Aywood. He will share with us his experience on delivering digital learning in Zambia. So welcome everyone and let's start. I'm going to share my screen now. Is it OK? Great. So just to give you a little bit of background. Each year, we are combining the Digital Academies calendar. So for the beginning of 2020, we have scheduled a total of seven level one and level two DHS2 academies. So for those who don't know, the level one academies are in general run by our partners, our HIST partners. And they last approximately six days and they will cover, they are covering actually different aspects of DHS2 configuration. So on this slide, you can see the three academies that we had planned for the first semester of 2020. And then the second category of academies that we are also hosting are the level two academies. These are very specialized courses. They are run by a team of trainers. They are managed by HIST-UIO and they are open to global attendance when the level one academies are mainly regional. But then in March 2020, the COVID-19 pandemic started. Several countries or lots of countries actually took some measures in terms of travel restrictions, borders closure, quarantine, social distancing, home office. So the direct consequence for all the academies that we had planned for this first semester is that we basically could not host them any longer. So we had to find how to deliver, how to turn these regional unsight academies into digital academies. So basically the main idea behind all that is that we wanted to offer the same quality and the same contents. Only the method of delivery will differ. So we had several goals. Obviously, the first one was to keep building capacity through our training program. So as I said, the digital pivot of this level one and level two academies was the main objective. We also wanted to make sure that all the participants enrolled in the previous published academies, the seven academies that I introduced earlier. Will be also trained digitally. So to do so, we had to develop a series of level one and level two digital academies, starting from June 2020. So it means that basically we had two months to put everything together. So the first step was to make sure that we understand and we know the needs of the HIST groups and also of the DHS to community because the DHS to community is basically our main target. So before doing anything, we needed to identify what were the needs in terms of training pace. For instance, the most suitable period of the day for the community members to attend the digital academy, the ideal duration, the training model and the preferred video conferencing and learning platforms. So we asked the members of the community, but also the HIST group actually to take a survey. So the result was that we needed to make sure that these digital trainings will last between two and four hours a day. The hosting these digital trainings in the afternoon was working better for most of the community members. The average days will be approximately 10 days. I say approximately because this can change depending on the level of the academies. For some academies, for instance, we will host them a little bit shorter. So regarding the training model, we quickly realized that a mix of live sessions and asynchronous sessions would work the best to keep the engagement and the interaction with the participants. And then in terms of platforms, we chose Zoom and OpenADX, but still it was based on the feedback that we received from the community and from the HIST groups. So we had to took few actions which were key. Basically, the first one was obviously adapting the content that we had so that it can fit a digital format. When I talk about content, it's roughly 45 hours that we used to have for onsite academies for the material basically that we will split in order to fit the digital format and the two four hours of trainings per day. The other thing that we had to do was it was specially for level one academies. We had to make sure that we will support our partners from HIST in the use of the platforms for video conferencing and also the learning platform OpenADX. And obviously, we also had to adapt, update the training environment for each academy, such as, for instance, the training and the DHS to instances. So one of the main things that we also wanted to do was to make sure that the community members understand that even with digital academies, there were key advantages for them. For instance, these academies because of the current pandemic, we thought that it would be better to deliver them free of charge. The other thing also very important was that the curriculum remained the same than the onsite academies. The academies will also still be conducted by HIST groups and subject matter experts for level two academies, for instance, the other advantage for the participants obviously was the fact that they will be trained on the material and on DHS too, but in safety from home or from their offices, for instance. And then one of a very important thing is that we will still deliver the DHS to certificate as we do for the onsite academies. It's an official certificate, which is also traceable, which is really important. So we had main challenges that we faced and we tried to find the best solutions to solve them. Because of the digital format, there is no human contact between facilitators and participants or between participants themselves. Also, because of the pandemic still, participants are very busy because most of them are on duty in order to support their countries in the responses against the pandemic. Another challenge for these academies were that we needed to make sure that we encourage participants consistency in the sense that we wanted participants to be able to attend the academies every day and to complete the assignments and exercises within the deadlines. And obviously, we also wanted the prerequisites to still be one of the main criteria to select the participants. So there, because we have a limited time, we needed to make sure that participants will have the prerequisites to attend the academy, meaning encouraging those who needed to complete the fundamentals course before being able to attend the academy and in a very short period of time. We also have the main challenge for the regarding the facilitators was that most of them are and were very busy because of the COVID-19. So they had to basically provide all the support and also developing the how to say that sorry, and also they were involved in developing the material while they were on duty supporting the country against the COVID-19. So to do so what we decided to do, as I was saying before, is that we basically in for the for the for the participants participants engagement, we decided that we wanted to mix live sessions and asynchronous sessions. In order to have a better human contact between participants and between facilitators and participants, we decided that for each academy, we will have one on boarding day, where basically participants and facilitators will meet to talk about the technical requirements, but also to get to know each other. We also tried to encourage participants engagements by using certain some tools such as Slack workspace where they were able to directly interact with the facilitators, but also with over participants. We made sure to include in each session some quizzes, but also to collect the feedback from participants so that we can improve the academies. So here, this slide is very interesting because one of the main goal of that we had obviously, one of the main advantage of the digital academy is that it's enabled to reach out to a larger audience compared to the onsite academies. So we basically what we did is that we opened registrations for each digital academies and then we were trying to compare the registration for the digital academies and the registrations for the onsite academies. And it's actually very interesting because here I have mentioned, we have mentioned four academies, digital academies in 2020, and then the same academies for the same region in 2019 when these ones were onsite. So for instance, you would see that the analytics tools academy, the onsite one which was held in 2019 in Lomé had 60 registrations in total, while the digital version in 2020 collected 253 registrations. And basically, it's the same for the following academies. We see an increase in term of registrations when it comes to digital academies. So yes, so I can say that we for from this slide, we can see that we definitely managed to reach a larger audience. So now we'll give the floor to Matthew Pina who is the DHS to online academic coordinator and Matthew will present the different tools and platforms. Thanks. Thanks, Alice. Can you hear me well? Yes, I can hear you. Sorry. Thanks. Thanks for this overview of the landscape and how it how it started with the in these times of pandemics, where we had to basically pause in March, our digital, our DHS to academies, and how to rethink rethink the way to deliver these trainings. As our main objective, obviously, during this, during this, this pandemic was to to keep the ring DHS to academies. And not at not at any cost, obviously. And first, we obviously wanted to keep our our structure in place, meaning that we wanted to have the his groups to have the ownership of their level one academies and the subject matter experts to keep the ownership of the level two academies. We obviously needed to understand the participation, participation limitations in terms of in terms of time to be spent online connectivity and meet the needs in terms of when it would better shoot them, how long could an academy last and so on so forth. Alice went through that. And from a his post law perspective, make sure that we would ease the transition from an in person to a digital training. So how did we how did we try to to tackle all these all these points? So how did we do the pivot first and on the support side, with putting in place a range of range of tools to help us to support the his groups and the SMEs. So sorry, the subject matter experts. And we created a GitHub repository and wiki. So that we could and we gave access to the his groups. It's actually a public public report that you can find on GitHub. So that we could, we could list all the, all the different, we could have all the documentation. And we could list all the best practices that we were going through during the first academies in order to build our own capacity in the process. And we created a trailer trailer board so that we could have an interactive to the list and help the his groups to go from one step to another and prevent us from forgetting any any key step that can happen when you end the rush. Because as Alice was saying, we needed to deliver fast and we need to deliver under a lot of pressure. So it's easy to to to forget a step in in the process. Obviously, we had many zoom meetings. And and slack was our constant channel of communication, I think this one to make sure that we could sync up with the different teams, groups, his groups and the slow. So that's on the support side and on the delivery side. And we have an extended knowledge of OpenEDX. So which is the platform on which is you is run DHIS to online academies. Obviously, we suggested this one together with Moodle as we have trained the Moodle repository. It was up to the to the his groups to choose. And Alice was highlighting that during our our tests, sorry, our question questions being sent. OpenEDX was favored. We used still Moodle though. And in for some specific level two academies, we actually used GitHub as well. And and obviously, zoom was the favorite tool for live communication. And together with a logo that is in between zoom and slack and completely invisible on my on my slide, which is YouTube for the asking for use for the asking for use version. So for all the learners who could not actually yay, who could not actually thank you, Alice, who could not actually for 10 live, they could watch the video on the YouTube on the YouTube playlist that each academy had. We used slack in some of the academies in order to ease the communication between the participants and the the course team, in order to add a bit of human aspect in that as Alice was stressing it out. And obviously, we were using a DHS tool as the main the main play field for learners to practice. Because as you all know, DHS to Academy is around is pedagogy is bringing around how to do things hands on. And for one specific web development, we've been using codes and box. So we've we've been using what I wanted to say here that we've been using many, many different tools. We tried to use these tools in a very in a manner that was not that was beneficiary to the learners. And I think that the next slide will tell us that there has been some we actually somehow managed managed to do that. There's obviously an avenue for improvement. But but provided the conditions of delivery, I think that these figures tell us this is only for one Academy, which is the last one. So in Sri Lanka, or organized by his Sri Lanka. And you can see by the figures that, while five being obviously the most positive, and that the learners were very, very satisfied with their experience. And finally, because I think that I'm running much out of time. And finally, I'm here as well to announce the launch of the DHS to events fundamentals, which have been the content has been developed as a team led by Shorajit. And and it's a self based course. So it's same, same format as the DHS to fundamentals that builds upon actually DHS to fundamentals and focuses on individual data. So I invite all of you to spread the word and enroll yourselves in this new massive open online course. Thank you very much. Thank you so much, Matthew. And now we're going to start. We I'm going to introduce actually the second presentation from Petra. So Shaina page. Are you ready? So the basically, you would present on how to build a flexible virtual solution for trainings? Yes, absolutely. Thank you very much, Alice. Thank you, too. So my name is Shaina Padovano. I am with Guide House supporting PEPFAR. And I am joined by my colleague, Kate Schoenberg, also with Guide House, supporting PEPFAR. And we're going to be talking a little bit about some of the work that we've done with the current training, using the PEPFAR Virtual Academy, which is actually built on Open edX, which is the same platform that the DHS to online Academy. So a little background. ESCAC is the US Office of the Global AIDS Coordinator in Health Diplomacy. That's the office within the State Department that houses PEPFAR. And just a little tidbit about PEPFAR. As we know, it's a HIV AIDS program that has served more than 17 million global beneficiaries since its start in 2003. And a little bit about the creation of a new staff role that we had in 2019, ESCAC created a new position called PEPFAR Program Manager. So this is the person that serves as the day to day contact for the operating units or OUs, countries around the world, where PEPFAR services are delivered. And this PPM is a very important role. They also work closely with another role called the PEPFAR Country Chair. And the PEPFAR Country Chair serves as a most senior ESCAC representative for the specific operating unit or country. So we needed to create a training that would help not only the PEPFAR program managers, but also the country chairs, get up to speed and understand sort of what they need to know in order to do the jobs, their jobs the best they can. So it was it was a challenge. We did have a new crop of PPMs and chairs join us this year. And so there were some unique challenges that we faced for onboarding and training these new staff members. And while our goal was to create a training for them for the next several months, we also wanted to create a training that had longevity and provide information for staff across the program to be able to access easily on the PEPFAR Virtual Academy in the future as well. So some of the challenges that we faced is many of the PPMs come from a variety of subject matter backgrounds and they may have different skill sets. They have many responsibilities that they have to manage. A program advisor, data management, they have to understand elements of our data systems as well. So there's a lot to learn. And on that note, the steep learning curve and folks were quickly onboarded. And just minimal time for training due to the fast pace of the program and the business cycle. And then of course, the last item here are fully remote workforce right now, due to COVID. So that's just a little bit of background and framing. And with that, I'd like to pass it over to Paige Schoenberg, who's going to get into some of the details of the solution. Paige? Thank you, Shayna. You could just scroll down. Perfect. Okay, so Shayna described some of the many challenges that we faced in developing a training for these PPMs and chairs. So our solution to that was to develop a flexible virtual learning solution with a digital learning focus, utilizing the PEPFAR Virtual Academy or PVA, which is PEPFAR's online eLearning platform, in addition to live webinars and question and answer Q&A sessions. So the PVA is our version of the DS2 Academy. And so you're familiar with that, you can think about PVA in that way. And this approach was supported by senior leadership across PEPFAR. And we had several aims. The first was to provide new and existing ESCAC staff, particularly chairs and PPMs, with the foundational knowledge needed to succeed at their roles. Two, to easily scale up trainings for a growing number of learners and content developers. Three, to offer maximum flexibility via self-guided, sustainable opportunities for learning, that gives staff ample time to learn, reflect, ask questions, and engage in open discussion about critical job related topics. Four, we aim to house all training content in one centralized location, so that staff can take these trainings at their own pace and refer back to them, including during the relevant times in the PEPFAR business cycle. So as Shayna mentioned, at the PEPFAR business cycle is incredibly fast paced and complex. So it would be impossible for everyone to learn everything that they need to know all at once. So during this initial time period, they'll be given an introduction to these tools, our data systems to the program, and then later on at the relevant times, we'll come back and review the content. And then finally, five, we wanted to reduce the demand on training developers to present live webinars and provide an opportunity for these needs to refine the presentation, as well as to provide an opportunity to follow up with additional resources for learners, such as an FAQ sheet with frequently asked questions or additional content that they may choose to add or resources they may choose to link afterwards. Next slide. So these training efforts were led by a group called the ESCAC capacity building group or the CBG. And so this was one of the key points that helped to make these training successful was the creation of this group to lead the effort. So the CBG comprises members of PEPFAR data systems, program policy and technical teams. And these individuals were chosen for the cross functional nature of their work for their relatively high profile within the office and their ability to collaborate across multiple teams. Second, the CBG leads project management institutes best practices in adult learning, reviews the content created by the subject matter experts, we use this with the SMEs and the POCs and manages the PVA content. And finally, the CBG created and managed a development and delivery timeline, which was approved by senior leadership. And having this timeline approved by senior leadership gave us the authority to reference this timeline in order to support effective engagement across the program and manage stakeholders. So we could point to this timeline in order to keep SMEs on track the specific milestones and also let learners know when they could expect new content to be released. So we broke our timeline down into these three waves. Right now we're in the first wave, which is taking place in September to mid October. And this is where we'll deliver high priority fundamental courses as well as quick wins via the live and online fora. And then we'll conclude all the way through wave three, which will take place in April through May after our annual business cycle after the budget and target setting process has completed. So next slide, please. So part of the flexible learning approach relied on the Pepper Virtual Academy, which I've mentioned is Pepper's online e-learning platform for learning about Pepper's program, its history, data systems, data analytics and more. And PDA is designed to capacity Pepper staff through self-paced or through hybrid learning, which might mean that learners are guided in more of a cohort style where they're led by an instructor and an instructor and they need to complete certain tasks by a certain date so that they can then discuss with their peers. And PDA is free to use for all Pepper staff, which is really key. We want to make this as accessible as possible. So here is a screenshot of one of the PPM and chair trainings. The first module that we presented is the Pepper 101. This was presented by the deputy principal of Pepper and talks about the history of the program. You can see that we've then created several sections, the first being the recording of the webinar, then a confirmation check, which we've asked learners to check off when they've watched a video or completed a module. And this is how we can track that learning management to see who has completed which trainings. And some SMEs have chosen to get a little more advanced and to also include knowledge checks that test learn your retention of the topics. But as a very bare minimum, we've asked everyone to include a confirmation check. Then we also have a PDF of the slides that can be downloaded as well as a Q&A where we've put Q&A documents about some frequently asked questions. And then you can see we had an upcoming module, the HIV Basics module, and we've started to mark those as new when they're recently added to PVA so that learners can easily go and see what content has been added. Next slide. So using PVA was critical to be able to scale up these trainings. So some numbers to date. We have four courses covering over 30 modules, which includes 20 webinars. And these courses are organized by learner-staff role, but they are available to all USG PEPFAR staff. So even non-PPMs and chairs can enroll in these trainings if they choose to. We also have 40 plus knowledge and completion checks and over 70 enrolled learners. The content has been developed by over 30 SMEs, so it's a huge undertaking to coordinate. And we'll talk a little bit more about the process that we use to manage this process in later slides. And critically, as part of our hybrid approach, this or our flexible approach, there were a variety of formats from which SMEs could choose. And so we allowed SMEs to choose whichever format they were most comfortable with. And we provided resources to help them. So it could either be a pre-recorded webinar with or without a live Q&A session following. It could be a live webinar with a Q&A session or it could even just be uploading documents and standard operating procedures for learners to review and download. And then this may also be followed by a live Q&A. So as I mentioned, to manage the challenges of coordinating such a large-scale training initiative, the CPG used a variety of methods to manage the development and delivery of content in collaboration with over 37 subject matter experts, as well as program support officers who helped us with the logistics of managing these webinars and kind of wriggling all these SMEs together. So we used the adding methodology to assess design, develop, implement, and evaluate training content. And this is a proven adult learning methodology that we use for all of our trainings. We also communicated with SMEs and with learners via a shared inbox which helps us to track emails and make sure that all the members of our team are on the same page. We used the SurveyMonkey Survey tool to solicit feedback from SMEs about which webinar format they preferred, their dates and times that they were available, and anything else they might need us to know when planning these trainings. We also very critically centralized all of our content within SharePoint. So we could house all of our content there, edit and review it. And we created folders for each training topic so that each SME would know where to upload their own topics and content. We also have been holding twice weekly office hours to support the SMEs and PSOs. We supported content development on PVA by uploading videos and uploading the text as well as managing knowledge checks and all those technology related aspects. We leave learner management by enrolling learners and communicating about the PVA registration process. And we also very quickly empower points of contact to set up and run the webinar logistics since we are a small and it will be very difficult for us to run the webinars for all 30 topics and for all of these different speeds. So in order to help SMEs and learners we developed a variety of resources to manage the challenges of coordinating such a large-scale training initiative. And so we some of the examples of things that we developed are a process map to explain the content review and submission process and there's a screenshot of that below on the right. We also created a shared calendar of trainings which is updated in real time and we probably update this at least three times a day. So this as goes in you know as new webinars are finalized and everything anyone is able to answer or to reference this calendar. We also created and managed a comprehensive spreadsheet of all of our training topics and you can see a screenshot of that on the left and that is just a small snippet of what is included in the spreadsheet. So we have the name of the module which we visit in who is responsible what is the time and date for this webinar and all those good things. And then we also internally manage a content status tracker for all our training topics which help us to track if content has been submitted, if it's been reviewed by our team, have we provided feedback to this yet, is it up on PBI and so that has been really helpful as well. And then we also provided several more resources on creating content on adult learning best practices, how to host webinars, how to record content in PowerPoint, and several other topics which we've made available on our SharePoint page for all of our SMEs to reference. And then finally we've identified a few key takeaways. So while the PPM and tier training effort is still ongoing and we are coming to the end of wave one, we have identified key takeaways from this process. So a few key things when leaving a large and complex training initiative, it's important to one implement strong processes for coordinating with SMEs to develop the content, to review material, and to deliver feedback to SMEs and we've talked this in the way that we've done that. Two, it's important to create and direct learners to one centralized repository for all resources related to the training initiative. And this was really what PBA has been so helpful in doing in prior years. All of our content has been scattered across multiple folders in SharePoint or in some cases on our support sites which is based on Zendesk if you're familiar with that. So there really was no one centralized repository where learners knew they could find all of the content that they would need. And so this year putting everything on PBA has allowed us to do that. And then finally it's important to identify leaders who can serve as the point person to drive forward development of each larger topic and its subtopics. So as I mentioned, we have over 30 modules. Some topics have upwards of 10 sub modules. So it's really important to have a POC for each of those larger topics who can help you to coordinate and work with the SMEs. So that brings us to the end of our presentation. I know we said we'll hold Q&A until the end so thank you very much. Thank you. Thank you so much Pays and Shaina. Now we're going to have the third presentation from Lynn and Sandi from HIST South Africa. Lynn Sandi, are you ready? So this presentation will focus on how to turn on site trainings into digital trainings. Good afternoon everyone, it's Lynn here. Sandi and I are going to talk to you today about our journey of taking DHIS2 trainings from the classroom to online. We started this journey three years ago with little knowledge of e-learning and online engagement. For us, this was a daunting but exciting and strategic move and it enabled us to be ready for COVID and lockdown. Daniel Menden is an international speaker best selling author consultant and trainer and we're going to use some of his some of his steps on converting face to face to virtual online training and we've added a couple of steps of our own. In our presentation today, we're going to share our results, successes and challenges under each of these steps. We're also going to use Daniel's analogy that moving from classroom to virtual is like moving home. So the first step is to convert and plan strategically. This is just like moving home, you need a plan. Moving home as you know involves deciding which boxes to keep, which to discard, which boxes to unpack first, how and where the contents should be unpacked and of course there'll be some items that are missing a new home that you'll need to purchase. So we've kept things like our reference manual and other important documents that are still relevant to our courses. Some items have required some modification to fit into our new environment like assessments and exercises and marking guides. And of course there are lots of things that we simply didn't have that we needed to acquire, a new platform, new course design, new software like iSpring, Camtasia or Dacite, and new activities, things like WhatsApp groups, polls, interactive whiteboards, H5P, ongoing feedback and so on. And lastly there were things that we had to trash, like PowerPoint presentations with bullet points, no more death by PowerPoint. I'm a hoarder by nature, but this is no time for hoarding and hoping to use or repurpose. By doing that you are depriving learners of more appropriate, modern, exciting, motivating ways to learn. So just like moving from a big house into an apartment, training is moving to lean, mean, and fast. However, good instructional design and fancy tools mean nothing if learner engagement and commitment are not at the heart of it all. I'd like to briefly explain now how we have structured our current online courses. We have a DHIS2 Foundation course running over two months for learners to complete in stages. There are no live sessions and no face-to-face support. In the EPR project in Zambia we replicated this course, edited it, added to it, focusing on EPR reporting. After doing the course themselves we trained a small group of people to become trainers and we referred to them as tots. And Arthur will be speaking more about this project after our session. We have a course setting up daily data capture in facilities. We have also trained tots in the provinces to provide online and face-to-face support. Let's move to the second step which is more interaction and more engagement. And here I'd like to talk about some of the things that we've put in place in our courses to improve this. The content pages in our courses are simple. Each module or section has the same format repeated, so as to improve navigation in the course. Our courses have multiple sections but only one section is displayed at a time to avoid cognitive overload and the dreaded scroll of death. We've kept our activities short and to the point so that students will complete them. When we first started we made the mistake of videos being a little too long. Ideally there should be around three to four minutes max. Not only do they load faster and cause less frustration but students are more likely to watch them to the end and even re-watch them. We've tried to use a range of activities, discussions, polls, noodle books, videos, quizzes from various sources, exercises, self-assessments and scenarios. After our pilot and staff training we've run five online DHIS2 Foundation courses since June last year. We find that students were registering but never getting started with the course. Unlike a MOOC where a high dropout is expected this is a course that carries a competency certificate and we wanted to get as many students to start and complete as possible. This graph shows how the percentage who registered but never started that is the grade columns has decreased from the first course to now and that the percentage who start and remain on the course has increased. From our experience with our courses we have learned some useful interventions to retain learners on the course. There are WhatsApp support groups where students can ask questions with both facilitators and students giving feedback and less it is something that needs tackling in working hours a solution can often be provided immediately and it's also a great way to give personal encouragement. Students are also welcome to contact us by email which some prefer. This is a key point for us. Sandy and I did an online course at a South African University and we were not able to speak directly to any of the facilitators on the course which we found frustrating. When clarity on some areas was required. We do post announcements in Moodle but we also send a clear introductory email with screenshots on how to log in how to get started and ongoing email reminders and tips throughout the course. We're now starting a course with a Zoom meeting where we briefly introduce the Moodle platform. We invite some learners to log in and we explain how the course works. Where possible we are also provided to give face to face and online support as with the daily data capturing and EPI project in Zambia. In YouTube videos sorry in YouTube the videos don't take so long to open and the resolution can be adjusted to use less bandwidth but we discovered that in some locations YouTube is not accessible by students so we felt it's important to now include both options that is accessing via YouTube and also in Moodle itself. And lastly we conduct short surveys using Google Forms on the reasons for lack of progress or dropping out to help us to understand the reasons why students fall by the wayside. I'm going to hand over to Sandy now to go through the last three steps with you. All right. Hi everyone. I'll be reviewing these last three steps very briefly with you guys. The next step we're going to be looking at is preparing of facilitators and learners and facilitators need to be familiar with the course learning objectives they obviously must do the course and we prepare a testing ground for them. This is a copy of the course for them to practice generating reports and so on. We review the key interactions we need with learners like discussions, what's up, important dates, when to email, setup course notices etc. They need to know how to monitor feedback and they need to learn how to generate reports. For the learners, we stipulate that basic computer literacy is a requirement but we don't enforce it with any exam and so some people do try their luck. We explain the high level goals of the course in the opening module so they have a clear expectations. We try and help them in their commitment and planning with some issues in the netiquette video we've developed and we personally encourage them through direct WhatsApp and emails and lately as Lynn mentioned we are running introductory Zoom meetings to show learners on how to log into Moodle, navigate and work through the course. The next step we use the pilot and constant monitoring of the course and feedback to identify activities that work. For example, we found that peer to peer marking did not work and we had to drop this. Then assignments and assessment questions often have to be changed as different people have different interpretations. So just recently we asked in an assignment that they provide a heading in a chart using the key elements of what, where and when to describe the chart and the results as you can guess was what, when and where were in the title. Then the discussion groups may be proving to be useless and may either be dropped or discussion point revised because they almost encourage knowledge sharing but we found people just repeat the previous people's responses in their groups. Now, this is an example of a poll we've started using in the course and this helps us gauge how much intervention we need to provide and it shows us the type of users on the course. The majority of the participants as you can see over the different courses are not familiar with online training. This is another way we are analyzing the numbers. We look at the numbers that drop out by the first assignment which is roughly halfway through our course. Then the numbers that fall away by the second assignment and finally the numbers that fall away at the final assessment and you will notice here that this initial navigation commitment to the course is the hardest and as shown in the earlier graph we know most users are new to using online platform but even though we haven't engaged with the learner and you can see that these numbers are slowly dropping away it's still a very delicate stage of the online course. While monitoring our learners we have determined what our learners key challenges are and some challenges are causing them to drop off. We have developed ways to We have developed ways to deal with them. Challenges like forgotten your password or they can't see the next module as they have not completed an activity we do have an FAQ in place in the course but of course the learners do not read this. Our other common challenges are intermittent internet no access to a laptop sadly our courses are not optimized to use on a cell phone. Work commitments and personal problems like illness death in the family birth of a child. The final step we want to share findings falls under the evaluate and change. We use self-evaluation questionnaires at the beginning and the end of the course and it's really for them to gauge if learning has taken place. You can see the results are showing that there seems to be some improvement. If you look at the percentage change from blue to orange for example the last question is 57% that did not know what this question meant in the beginning of the course and only 5% at the end were battling. All the courses ask for final evaluation and just to note one question that is not clear in this graph is possibly this one here and it reads activities are helpful in gaining a clear understanding of the topics so that's the topics at the end there. We've compared the results across two courses here and it's useful to compare across the different courses and try and pick up the common problematic areas even if it's only relating to one or two people. Like the question on discussion forums which is over here and you could see we asked if it was useful and a lot of people were starting to find that it wasn't useful and they dropped down a bit on the side then. So we'd like to end with feedback from our learners showing their positive responses. Facilitators were supportive at all hours. Videos were clear and helpful. Useful to know about the data quality tools. Enjoyed working with pivots and charts. Creating my dashboard has been exciting. Visual impact is far greater than reading. Exercises were very good. Looking forward to see more improvement. Oh my goodness, I can't see this thing. Of my daily capturing and support that I got from facilitators in class was really helpful. And finally it's a well-designed course. So with commitment and engagement from both facilitators and learners these courses are working and with requests to run more courses means we might just be on the right track. So thank you very much, everybody. Thank you. Thank you very much, Sandy. Now we're going to hear the last the final presentation of the session. And once again, do not hesitate to ask any questions regarding the session on the COP. The link has been added to the chat box. So don't hesitate to ask any questions. And even after the session we'll make sure we answer. So that the discussion can continue. So now it's Arthur is going to present the data use for immunization in Zambia. Arthur, you have the floor. Thank you so much. Can you hear me? Yes, now we can hear you. Yes. You can't see my screen. Is that right? We have your screen. You have my screen? All right. All right, I'm going to talk to you briefly. Looks like we've got only a very little time left about a project for using data for immunization using distance learning material in Zambia. Arthur, sorry to interrupt. So sorry. Maybe you want to put the presenter mode? Sorry. That one? Is that right? Alice? No, not yet. I'm seeing mine perfectly. Oh, what do you see? Are you seeing this little shots down the side? We see the whole poor point. No? No, still not working. Maybe I have to stop share and re-share. Should we just do it? Yeah, I think so. Arthur, just click on slide show. Okay. I've got slide show there. On my screen, it's perfect. Arthur, can you find the tab called slide show? Then you just go slide show and then view the slide from the beginning. Just go to tab right at the top there. Yeah, I'll go to slide show. Along the bit? Slide. Just go up and next to there's yeah, and I go along to the right. Yeah, stop, stop. Go back, go back a bit. No, no, no, no. Slide show. Yes, no. Composite. Yes. In the menu at the top, Arthur. In the menu, yeah. There it is. It is there. Okay. And from the beginning. Okay. Are we there? Are we there? You need to click from the beginning. It is on the beginning. Yes, but it's still the point. All right. Maybe I stopped my share. If you feel more comfortable, yes. I stopped share. You click on from the beginning and then you share again. I should have stopped my share now. Yes, Arthur. All right. New share. Yes. Now you can. Let me go to from beginning on my screen. Is that right now? I don't see your screen. You need to share it again. Good. Share screen. Is that it? Yes. Go from the beginning because you are at the end of your slide. It's not on presenter mode, but that is fine. Just. It's the wrong one. Sorry about that. Zoom. This is crazy. New share. It happens. Yeah. Are you seeing that? Yes. This is the slide from the beginning. That's fine. You can you can stop presenting. Okay. Sorry, but. Sorry. Okay. Thank you. Okay. The project is in Tinsali. Any of you have been in a district in Africa, Tinsali is kind of on the lower end of the scale of developed districts in Africa. It's remote rural. It's 12 hours from the near from the the capital. There's only one staff in the entire district have been trained in DHS to nobody was entering data. And yeah, we had a proof of concept for Gavi to try to get an improvement of data use for immunization in that one district. We did our assessment in March this year and we found a pretty well-funded immunization project with good outreach, regular supervision, but EPI data that had never been looked at or analyzed. And one of the fundamental things was a monitoring and evaluation framework that was pretty chaotic with Excel spreadsheet being used for planning, paper being used for supervision, and DHS to being used for routine data entry. And a ICT setup that there wasn't a single computer at any facility, none of the facility staff had ever been exposed to computers formally, though they knew how to use them, and no facility level data entry. So we came up with a great plan, a kind of traditional face-to-face learning plan that would have two one-week training sessions driven by expatriates flying in and flying out and one local nurse in the facility for supervision and follower. Then came COVID. This was literally a week after we'd written our inception report meant no travel, no face-to-face training, no supervision, and most of the staff had been assigned to COVID duty. What the hell do we do now to try and get our ducks in a row? Fortunately, we have... Arthur, sorry, can I interrupt? Can I share your PowerPoint? Because you've given it to me because we can't see you going through the PowerPoint. So I'm going to share your PowerPoint, and you must just tell me which PowerPoint I should be on. I think it's on three or four now. I don't know which one you're on. You can't see my PowerPoint at all. No, we're only seeing the first one. So I'm going to share the screen and go through it, and you just tell me when you want to move to the next one. Okay, I'm going to obviously start it on four. I presume you're on four now. Then that came COVID. Oh yes, now I can see it. Now you're clicking on it. Okay, now carry on. We weren't seeing each slide as you clicked. Do you want me to share or do you want to just carry on? I think that it works now. All right, COVID came along, and we needed to get our ducks in a row. Can you see those ducks? Yes, I can. You can, all right, okay. Okay, I mean, it was a real body blow. What the hell are we going to do with no training, no supervision? Fortunately, we had the HISP South Africa DHS2 courses that Sandy and Lynn were talking about, that we were able to convince Gavi. Fortunately, Gavi was amazing when this came along and just allowed us to totally reallocate our resources to adapting the existing course. Then we adapted this DHS2 fundamentals course, changed its emphasis, not very much, just a little bit. We used the Zambia database, which I think was crucial so that all the data elements and the indicators were ones that they knew already. We introduced more of the information cycle and we used examples from immunization because this was a Gavi immunization project. We had to develop quite a lot of new material, basically around the information cycle, data use for EPI and get examples from Zambia that were usable. What was very useful was this reaching every district plan, which is a WHO plan for monitoring immunization, which was extremely useful. It means we could focus on that. The next thing we did was to train local trainers. As Sandy and Lynn have pointed out, they're absolutely essential. We selected them on basic DHS2 competence, four of them local and three of them at higher national level, and we put them through exactly the same course as the trainees were going to do. This was important for us because it was a test of whether we adapted the materials properly. It was an orientation for them to course curriculum and methodologies because none of them had ever done distance learning before either, and we thought it was important for them to really understand the problems, the many problems that participants were going to face because the TOTS had faced them themselves first. The other thing that we had to get their heads around was this distance learning approach. We had to teach them how to learn and how to mark assignments, a lot of mentoring and coaching techniques, and also this is something that they really did extremely well was hands-on support to the participants. So we've run the course, we've just completed it. Well, we haven't yet completed it yet. It was a six-week course and there were many constraints. The human was probably the biggest. This whole change to distance learning that Sandy was talking about before, the weak computer literacy, and one thing which was very real is that during this distance learning there are no financial incentives. So people who are used to learning, to getting something in their pocket from leaving their facilities and coming in for training, now suddenly do the training and they don't get any money in their pockets, which is tough. The technical side, there were no computers at the facility when we started and thank God for Gavi, they came up with a computer for every facility. There was no internet in the budget of any facility and we were able to overcome that with project funds. We had big problems with setting up a separate training database which reflected the Zambian database and we're still having problems with this damned EPI app which we don't have all the data for. We've only got 40% of the data, no logistics data. The third major constraint was time. It's been five months since we got permission to start, so six weeks to develop the course, six weeks to train the trainers and we're at the end of the six weeks now to train the health workers. But we have done, I believe, a huge amount in the last couple of months. There's been an amazing enthusiasm at district level. We've got more than double the number of trainees that we had in our original plan. This is fantastic. The entire district health management team, when they heard that it was possible to enroll on the course, every single member of the district health management team enrolled on it. The thoughts that facilitators were absolutely unbelievable. Stars of the whole show, without them, we would not have been able to do it. They've been giving this hands-on encouragement to the staff and providing what Sandy and Lynn were talking about, a human face and somebody to turn to for technical and human problems. The other thing we've been really lucky with is getting good high-level support. Gavi, I hope there's somebody in the audience from Gavi, but Gavi was amazing. They just said, all right, fine. Take your half a million dollars and do whatever you want with it as long as you come up with the results. National level within Zambia has been really good. Both the EPI unit and the M&E unit have been fantastic. And of course, at district level, the leadership of the district health director and his whole team has been fantastic. They've all been on the course and they're all really excited. They're starting to talk about Chinseali becoming a digital district. That's how excited they are by this whole process. On the technical side, we've got a whole lot of computers and internet and they're entering data for the first time. And for the first time in all of Zambia, we're actually starting to monitor the immunization using DHS2. What have we learned? I mean, in red, it's the only thing in red in this whole thing. This is what we should have been doing all along. I mean, we've taken far too long and I say, thank you, Gavi, for teaching us to go on the right path. Because if we want and if global organizations want to have global reach, we just don't have enough. Even PEPFAR, with all its millions, doesn't have enough trainers and facilitators to do this training. And yeah, what we need to do is train local trainers, keep it simple. Maybe others can get away without it being simple. But at our level, we need to really keep essential and reduce our expectations, make everything enormously practical, and base the training on real skills that are necessary. We need to develop high quality online material that is reusable and recyclable in many different contexts. And as we said earlier, we need to convince the higher authorities that this is important. Yeah, it's not plain sailing. There's a massive attitude change that is needed. This whole question of the team approach that PEPFAR also mentioned, absolutely essential. We need to somehow get the whole change of mindset, discipline of distance learning, and getting staff to take responsibility for their own data. It's all a very steep learning curve for everyone. I've learned more in the last five months and I've learned in the last many years about material development, changing teaching methodology and delivery systems. And all of this, everybody sitting in an office without any human contact and motivation is extremely difficult. Have we got time left? Actually no, Arthur. Yeah, all right. We won't go forward. How are we going to go forward? But I think there's developed local distance structures and slow expansion is essential. Then I'll wrap up, have a great day and thank you for entertaining us. Yes. Thank you so much, Arthur, for this very, very interesting and inspiring presentation. It was a pleasure to hear. So the session comes to an end now. Thank you to all of you who were among us. Do not, if you have any questions once again, do not hesitate to go on the COP. You will find the thread in the chat and you can ask your questions. Even after the session, we'll make sure we'll reply. And I will also kindly ask all the presenters to go on the COP and to reply to the questions related to the presentations. And thank you to the presenters for being available and for these very nice presentations. Thank you to all of you. And we will probably see you tomorrow. Thank you. Bye-bye. Au revoir. Au revoir. Au revoir à tous.