 Hi! Welcom back to this next Ola Sly Sesiad. It's great to be here. My name is Emma Procter-Legg, and I'm the chair for this session. I'm really pleased to be joined today by Myrcella, David, Sarah, Debbie and Kirsty. This session is E-Mora, a collaborative online assessment project in midwifery education. The session is 25 minutes in total, with approximately 20 minutes for the presentation and five minutes for the questions. Please do use the chat to dyma'r cwysbeth ddechrau a gwybod. Mae'n gweithio'n gwaith â Kirsty. Rhaid i gael yr wych. Rhaid i gael ei bod yn gweithio'n gweithio'n gweithio i'w morau. mae'r pwysig, Kirsty, David, Martella, Debbie, a Sarah. Yn eich bod yn rhywbeth, dwi'n ymddiad, felly, mwy o'r gweithio i'r cyffredin sy'n ddych chi, dwi'n fwyaf i ddweud o'r gweithio. Yn ymdillwch, mae'r gweithio'n gweithio eich gweithio'n gweithio'n gweithio, yn rhain o'r Llywodraeth, Selfi Aulwch, y rhai yw i'r gweithio. If you move to the next slide, Sarah, I'll talk to you about what we were doing. So, we're all midwifery educators and ourselves here today are more on the learning technology side. So we've got a mixture of university learning technologists and pebble pad as well. And we're working with academics from various universities to take a very large sort of practice document that the midwifers would go out into practice and get all of their competencies signed off and put it into paper. Now, the Mora consortium, they actually wrote this document as per the professional standards. And it's a standard document that's about 250 pages. So all the universities who wanted to use digital for reasons less paper, it's data's more secure and all that kind of thing. So we really had the challenge of taking this enormous document and putting it online basically for the students in line with university regulations, the professional regulations and something that can be used by practice assessors out in the field, out in hospitals and community settings. So we're going to talk a little bit about that, but also about how our way of working was impacted by COVID and how that changed the practice that we were doing. So I'm going to send over to Sarah now and we're going to have a look at a quick activity on Mentimeter. Thanks very much, Kirsty. So indeed, so our first activity is called making it fail. An interesting take on how you think about these larger collaborative projects might work in reality. So if you could find your way to menti.com and I believe the URL is going to be in the chat and enter the code when you get there, 6348-0258. And we've got four options for you and I'm just going to come out of the screen in a moment to make sure everyone's got in. The code again is 6348-0258. And we're going to go to the poll and let's have a look and see what's happening. So that's very interesting. Already people have got a very strong sense that it's actually the collaboration between the multiple universities that's going to be the most likely barrier to success. I'll just wait to see if there are any more options coming in. No one's gone for anything else yet. That's really interesting. Yep, so anyone who's worked for the Nursing and Midwifery Council or other type professional bodies might have a stronger view about how they might perhaps become a barrier to actual learning. Okay, we've got a small vote for the pandemic. I think possibly that might be enough time for everyone to have gotten to Mentimeter. Don't worry if you haven't been able to get into it because we've got another activity for you to work on next. So we're actually going to go back to the slides with this clear view that a lot of people think it's the people and the communication that will be the biggest factor. And if I just move on to the next slide, we're going to actually take this conversation further into the Padlet now. And we're going to ask you to go to this Padlet, again QR code or link on the screen, so that you can put in specifics that you think might contribute to these four different areas. So given what you've said, that you think it's the Cross University Collaboration and Communication that might be the biggest factor for failure, perhaps that's going to be the most expanded comment here. So just hoping you can find your way before I move away from this slide because we've got the QR code on here and the link again should be in the chat. No one's saying anything, so I'm assuming that we're good to go over to the Padlet. We'll have a look at this one here. So let's just spend a moment just thinking about what you think might be too much of a challenge to overcome in terms of working together with multiple institutions. And I don't know if you notice the eagle-eyed ones amongst you, but there were nine universities that were part of this collaboration. There were others that started off thinking about getting involved, and there were also others that have come on board after the core group have done all the work, which is really nice because they get to benefit from it. But the majority of the effort and participation that came from our group of collaborators was from nine universities and mainly three of us from Pebble Pad as well, who were very committed to the project. Okay, so we've got some nice information coming up here. I'll just pick out a couple, but these are going to be picked up by my colleagues from the other universities as part of their discussions. So okay, university didn't invest in the systems required underneath the university regulations, and it might be too long and cumbersome to achieve. Thinking about that professional body regulations, too prescriptive, absolutely, that can always be a barrier to actual engagement. If I just look on the fly-rider of the pandemic, we've got some difficulty with working remotely with different partners unable to meet together. That's a very interesting observation. And then coming back to this sort of cross-university in collaboration, lack of agreement, lack of staff resource, people don't share, that's so interesting, and then working across different technology and platforms, et cetera. So I'm going to leave it at that point, and I'm going to hop back to the PowerPoint here, and I'm going to hand over to my colleague David from Sheffield-Hallam who's going to break down some of these factors. Thank you Sarah. So yeah, it's really interesting seeing what your thoughts were there, because we had very similar thoughts as we were going in as well individually as institutions. So the key factors and what we did really was, so we had the pandemic, so as everyone's been working over the last 18 months with that over them, but the pandemic meant that we had to move online and not just our group, but our whole institutions were moving online, we were having to make adjustments, and obviously from a technical perspective, doing the support and the training for that, but also then having to move online as a group, so we were learning individually as well as a group. And then of course, they're moving something from paper to digital, that obviously, you've got the platforms and which is the best to go with. Some of the institutions already had Pebble Pad and those that didn't were happy to get on board because it met our needs perfectly. So moving on to the next set of factors that we looked at, that moving from paper to digital, one of the key things was how it needs to work also with the professional body requirements and just to speak for a moment on those, I'm going to hand over to Debbie who's our expert on that. Okay. So my role in this project was purely as a facilitator supported by Sarah Copland and Kelly Shaw from Pebble Pad, but I did have a subject specific knowledge in relation to midwifery. So I was a midwifery teacher and worked as a midwife. And this was really quite helpful in working with the project. I must say the project manager turned out to be Martella who's going to speak shortly. Very good job she did of that. But midwifery education as you probably all well know is bound up by professional requirements governed by the nursing and midwifery council. It's very strict and very rigid and that was really picked up in the padlet that it did provide some challenges. But it was critical that we produced safe practitioners from assessment in practice. So we had to make sure that these requirements were transparent, user friendly as far as we could because there was a prescriptive element to it and that they were adhered to and we were able to report on those to aid the academics in making sure these people were fit to be part of a midwifery register. Thanks David. So finally, one of the final parts that we looked at was obviously the requirements of the different participating universities because we all have different requirements and those come down to some institutions who are needing to do grading as part of the assessment so that had to be represented. There's then the fact that it's placement based so depending on the placement patterns of the different institutions how would we represent that in one electronic document and the recording of hours and again how our institutions like to record the hours that students have done it's a very key part of the qualification. So we had all of those factors that we had to jiggle and work together. So I'm now going to look at a diagram that we felt best encapsulated the way that we worked together or the way that we came together. So right in the middle you've got the e-mail or our group and then you've got the basically where this all begins which is with the midwifery team and the academics within these institutions. They've got these set of regulations that they've got to work towards but their expertise is in those regulations and midwifery and teaching they've been used to doing it paper based. So they came to us like myself a learning technologist within the institution who understood the technology that could then give them their options and what they could do with. So within the institution we worked very closely the learning technologist with the academic and together came and represented at the meeting. So I can say you had that overlap so within the institution there was the technical expertise needed and also the midwifery expertise needed. Now with so many institutions and I know this came up in the problems that you could see beforehand all the institutions and all the different voices and ideas and everybody wanting it to be the best for their institution. Well, the third cog in this wheel that was perfect was the Pebble Pad team. They chaired the meeting they brought their expertise both midwifery expertise as well as the technical expertise and with them chairing it it was able to pull it all together keep everybody focused and on track. Now I'm going to hand over to Marcella who has been pivotal as well in taking this project forward by sitting with a key person a lot of the paperwork and bits and pieces and organisation of it. So Marcella. Thank you, David. This was the hidden and unseen work in the background that kept the project together and on track to complete. There was no expert or detailed project management but there was the use of the non-tools to agile and project management. All members were volunteers so there was no obligation but just a genuine wish to help and cooperate and learn from each other. The resources were plenty in terms of knowledge so it was a matter of allowing all this knowledge to be shown. Next slide, please. This part is going to show a few of the things that we did during the project to make it successful. So first we discussed the goals and we set out the common ground. Those were the factors that were discussed by David earlier. For example, the timeframe. The portfolio needed to be completed before the following academic year, September 2021 when they were coming out now. The participants and roles that were played and the outcomes to be expected so we started with the common ground. Next slide, please. We set out our priorities in the first few discussions and we knew that the resulting product had to be tested to exhaustion so there will be little issues once completed. The ePortafolio needed to be flexible and respond to the requirements and the features will be agreed by the team democratically. So next slide. We met weekly online as a group and we divide the project into tasks. The weekly meetings were recorded in case something was missed and there was need to go back and check what was agreed. But we did not do minutes per meeting. Instead we used something that we called the mapping and working document and that kept track of our progress and the decisions agreed. The progress was tracked per meeting so initially there was a document per week. Then a document per fortnight and then a document per month. We finished with several versions of this document but the last version reflects the final document that was agreed. This is a sample in the image of the development document and it's about the table of contents. It has three parts, the original PDF with some bullet points of our agreements and how we interpret those questions and also the Pebblepipe view, how we come to show that. And a final column is a progress column but it says agree in there but the previous versions will have more notes in there. This obviously is just the final version. Next slide please. To do the agreed work we selected three developers and we also did tasks, satellite tasks that were added to the build later on. So we completed the easy tasks first and we got used to and comfortable with working together and then we did the complex tasks, the complex ones later at the end. Some solutions became sort of a model that we replicated in different sections so effectively we standardised solutions to use in different parts of the portfolio. Next slide please. We used some non-solutions but we also tried new ones. So there were no wrong questions, no fears. We asked what happens if we do this and we did a lot of testing. So solutions covered all the requirements, some other solutions were a choice of the least bad option. We spent most of our time discussing and trying to fix and trying to come with better solutions. So we spent a lot of time on the imperfect solutions. The portfolio is going to end up with 400 assets at the end when a student has completed an academic year but we haven't seen that yet. Next slide please. We did however have the great luck and privilege to have an early version of the E-Mora trial by the University of Sheffield-Halland with a spring cohort. The students and the staff and the practice partners gave us some really good feedback that helped us improve the portfolio development. We were able to do individual tests with the support from Kelly, a technician from Pebblepad and our hosts here, Debbie. We tested student parts, tutors features, external assessors, accessibility. How easy it is to use with the tests on the different platforms in Pebble Plus, in Pebble Pocket and Atlas. Next slide please. But the real test will be the completion of the academic programme in all three universities, in all the participating universities. But that won't happen until at least the summer of 2023. By then perhaps we are going to be working in the next E-Mora. I want to mention that the creation of the help side section that is a collaboratively updated by all the participant universities is a testimony to the willingness to help in co-operate by this group. This side will be permanently available on-line to all the users of E-Mora regardless of university. And just as a summary, just to say that we started the project in July 2020, we trial a first version in January 2021 and we officially completed it in July 2021. And now I am going to hand over back to Christine. OK, I just wanted to summarise a little bit. Obviously it's interesting to read the comments on the Padlet about the kind of challenges we had. But we had a lot of surprise outcomes really and it might not be things that you're necessarily surprised to see. But things that we initially thought were going to be difficult, actually the solutions, if you like, were a better way of working. So with Zoom and Teams, we were able to work very in a very agile way as we all know. And we got to see each other's pets, which is always quite nice. And I think the things, having the ad hoc meetings was nice too. So we had little subgroups working on things and we could just have very agile, let's just meet up for a few minutes now. We mostly use Zoom, I think, to be honest, because we didn't necessarily have all the same platforms. But it was very beneficial. So, for example, if I think I remember, I was working with my colleague Sean and Marcella on some of the testing. So we got together, did a quick testing documents and then we could easily meet, perhaps the next day, with the wider team. All the meetings we could record in that way, there were people who couldn't come. And they did actually use those recordings. And then as Marcella was talking about, we actually got a record then of why we made decisions and why certain things are a certain way. So in a few years, if we get new staff coming to the universities and say, what have you done this way for? We've got a clear reasoning there. And I think one of the biggest things was the cost of the project. As you know, travel costs a lot of time to travel. Refreshments, all that kind of thing. If we'd have all had to go and maybe converge on Pebblepad or wherever, then there is a cost involved. And I don't know that that's something that's really been factored in, I think, for this type of project for our own institutions. And also just to mention, particularly during the first lockdown, as people travel and some people are locked down on their own. And people look forward to the meetings and had a nice atmosphere about it, it's quite chatty. And the other thing is that if we had a colleague who we wanted to come to the meeting just for a few minutes, it was quite easy to patch them in. I think Kelly, who's been mentioned a few times, was quite often as the sort of fan to technical Pebblepad knowledge would be patched in for a few minutes to talk about something, but she didn't have to kind of take time out of her day to get on the train or get in the car and go and see us. So overall, it worked probably better this way than if we'd have done it in a traditional sort of way, very task focused, okay? And that's our view for the last year. We're not responsible for their reunion though, I don't think. And just a reminder of who we are and who's involved in the wider project. Okay, thank you very much. Thank you. That's really interesting presentation, really fascinating to hear about the project and the scale of it. There aren't currently any questions in the comments, so I'm just gonna ask a couple of questions if that's okay with you. So I'm really interested to know sort of the absolute scale of what you're involved in. So how many students were in the trials and then how many is it gonna roll out to as it goes live? Well, I can answer for the trials. There was about 90 students, which as far as the students were concerned, it was not a trial, but they have had a good experience. We've just gone through the assessment period with them testing the final sort of stages of the documentation and so far, the feedback's been really strong. So it's so far been successful. I think we'll have about 150 starting to use it. Yeah, and that's just one university. So if we multiply that across the spectrum, it's gonna be quite a few. And then over the three year program as well. Yeah, and the good thing is for the people out in the practice, so the practice assessors who are midwives doing a busy job, they'll have to just get used to one way of doing it then. It will be common way of doing it for them, hopefully. Yeah, and hopefully it will be an easier transition after the last sort of year and a half that we've had. There is a question that's come in here from Lisa Gray. She says, great presentation. Thank you. I'll be interested in your thoughts on what you feel have been the overall benefits today in terms of this approach to assessment, redesign for students and staff. Who's gonna start with that? Yeah. David probably is the best part of the call to go for this one. Yeah, I can say obviously, but when it comes to the assessment design, the benefit of the fact that we came together as the institution, so I'm from Sheffield Hallam and Derby is quite close to us. And the fact that we collaborated in putting this electronic documentation together helped because again, when it comes to placements with the different healthcare trusts, there's a lot of overlap. So those that are learning how to use this electronic documentation out in the placements, when they learn one, they're learning everybody's. So it was mutually beneficial. And again, with the way that students might transfer or move areas and things like that, the paperwork that goes with them, the electronic documentation is the same and continuous. So that's been really beneficial from an assessment design perspective of the fact that we've collaboratively worked together because otherwise people might be familiar with the software, perhaps they may have seen pepper pottings before, but then the way it's been put together might be completely different. So this cohesive approach and the time benefits, I mean, we've not been able to demonstrate it or anything, but our support guides are amazing. You know, when you're working a technical field and you're putting together support guides, it takes a huge amount of time to put, you know, eye to needle, everything that the student might need or want as far as the part. But when there's nine institutions working on the same set of guides, it, you know, it took us a ninth of the time to produce much better quality support resources. So things like that have been huge gains. I think just to add to that as well, talking to midwifery academics, it's loyal to their profession. And we are, you know, in a sense, I suppose we're quite close kind of competing institutions, but the midwifery academics and our team and the collaboration is working for the good of the sector. They want to get good midwives out in practice who are trained because that's what we're all here for at the end of the day. So I think that ethos of maybe a healthcare ethos came through this project quite nicely. Thank you. I think that's a really great place just to end because we're basically out of time now, but thank you again for presenting. It was really fascinating to watch. Thank you. Thank you. Thank you everyone. Thank you Emma.