 There you go. So next is Steven, the way you go, Steven, thank you for the final acts of this coming before the shutdown. I've started calling it the graveyard shifts. For the people watching online, the QR code links to the file on Google, so you should be able to get to it there. I'll start off by saying that this research or this work was financed by the European Union. For all those Brits who are jealous of us remaining in the European Union. We got some money from them. During the pandemic that the European Commission decided, you know, we give a lot of money to higher education to kind of kickstart anything that has to do with online learning. And so within our institution, we wrote a project on use of digital video in practice education so during internships and stuff like that. That's what I want to talk to you about. A lot of bullet points but that was because I followed instructions by all on usability and accessibility. Normally I would have had pictures, but they said don't use too many pictures. So that's the balance between accessibility and boringness. So there's bullet points for you with a small map, and there's a text, an old text to the map. The situation is we're in Flanders, Flanders is the northern half of Belgium, where people speak kind of Dutch. We've got 13 University of Applied Sciences next to five regular universities and universities of applied sciences, they have, they can award these kind of degrees as you can see here. We tested this project in both level five and level six education so both associate degrees and bachelor's degrees. All of these are professional bachelor degrees so we have a difference between academic and professional bachelor's in Belgium. And this is situated within the professional professional bachelor. The location is a bit of the area indicated where our campuses are situated. We've got about 15,000 students within five faculties. So what did we focus on? We focused on the practice aspect of the programs. For instance, if we have an associate degree, one third of the package that students study has to be practice oriented. Usually workplace learning but it can take any form. So that's why we have the spectrum here, where on the left hand side you can see, you know, trying to simulate a realistic work environment whereas on the right hand side you have the real work environment. So students going to a work environment and doing an apprenticeship. So the project covered all sides of this spectrum. The stakeholders in this video registration. First of all, the students. Sometimes when they were in a simulated session, they would have peers who would be acting as if they were students or pupils in the school. They could be patients that could be workplace colleagues. It's usually supervisory staff from the workplace and then someone from the university who would also be involved in this video. So instead of just giving students a video camera and sending them on to the road, we would say, you know, there's an assignment for you to do with this video camera and fits into these learning goals and the ideas for you to observe either your own action or someone or your students and processes. Usually we would combine it with some kind of evaluation scheme, so they would know which competences would be evaluated on the basis of this video etc. We define the number of potential functions for video registration. If you're not in the same room as where students are performing actions, then you, the video allows you to observe the actions after the fact. There's evaluation so it can be formative or summative. There's feedback. So based on the video that the tutor has observed, they will provide you with feedback on the things you've done, things you've shown, and then a very important one is reflection of self protection. So students looking at the video of themselves and then saying okay, next time I think I will do it differently because of such and such. So all functions can be performed by a number of stakeholders, so the student himself or herself peers, I'll come back to that later, could be workplace expert or college staff. So the idea is, if you have digital video available about the student performing certain actions, you can have a number of stakeholders comment on give feedback on that so it doesn't, it's not limited to the people who are in the same room at the same time. Well, the other, the other issues are, you know, synchronous first asynchronous during the pandemic we had a lot of workplace learning which happened, which was observed through a synchronous video connection. A number of schools for instance in the teacher education department only allowed live streaming of classes and not recording of classes. We thought that once a class has been recorded we don't have any control over what happens with it, but they were happy enough with the synchronous streaming of a class, which was that observed by a teacher from university at a distance. There's quite a number of issues that come up with the difference between synchronous and asynchronous. So what was the project. It was used last week. So we had two year development project, and the research question we started with was, how can digital video registration be used more effectively in practice based education. So during the pandemic we were forced to improvise. We had to have workplace learning we had to observe students in the workplace environment. And so we did we improvised and sent students away with their own laptop or a mobile phone or whatever they were had available and we were thinking of what have we learned during that period that we can retain within the processes of the university. So we looked at this as a educational design research project. So we did a bit of desktop research we interviewed teachers who had used video during the pandemic, asked them you know what worked what didn't work. And then we chose, we put out an appeal on the internet and said who wants to participate in the project to study the use of video for practice based learning. We had quite some interesting reactions so we decided to start with 11 course modules during two semesters. The next one is a list of these courses at the end of the presentation. If you download that you have the full list of all all the courses and the context in which we used the video so she's a very wide range of faculties all five faculties participated. It's mainly professional bachelor so PBA stands for professional bachelor, but it's a number of areas dental hygiene nursing business management, child and youth studies. There's an associate degree on electromagnetic mechanical systems. I was hoping for more of these associate degrees. For instance, we have an associate degree on on maintaining windmills for instance, I can imagine that you as a tutor from a university would appreciate that the student takes a GoPro camera when he moves on to the top of the windmill so that you don't have to climb the stairs and see what the student does on the top of the windmill. But they didn't, they didn't bite they weren't interested. I was hoping they would guess that's one of the most practical applications that I could think of. But maybe they weren't ready for it. So in the department at the time teacher education, we had two teachers who really went into the project and I'll come back to those pilots later on. So the first pilot was this program on child and youth studies. It's kind of social work program. And the students involved were a third year students so final year students in a campus complex so they were in a meeting room with the teacher available. And we use the camps 360 degree camera. I don't know if you know this owl. It's a camera that offers 360 degree view on top and then it zooms in on the person who speaking. And the idea of this assignment was to simulate a family conversation. And because we had the camera in the middle of the table, they could zoom in on the different people speaking so you could really look at the, you know, the body language of the one doing the simulation. And you can also see on the 360 degree camera. So these are the three participants you can see them sitting here, but they had fellow students who are standing a bit back and observing the simulation while it was happening, but it was also recorded. So what the tutor would do is they would she would give them an assignment and say this is the conversation that you need to simulate. So there's a parent and the child and there's a problem that's happening there and there's one coach available and you have to mediate the conversation so that was the exercise. So what the tutor would do it, she would record the session and then play it back immediately on the screen like this. So she would have to perform the, the activity. Take the recording put it on her laptop and then show it and then pause at certain points and say, you know, this is what I saw what did you see. So she would use that in the live session immediately after the simulation exercise. In fact, we got from most of the students was at first. They were really worried about seeing themselves on video, because they hadn't used this before this was in the final year and this was the first time they were using video recordings. The second was, they felt that it was quite confrontational to it would have been confrontational without the video because they would have given been given feedback by their peers and by the teacher. They were really looking at themselves and video and really seeing what exactly they did, which, you know, made, which was a cause of some feedback was very confrontational for them. But towards the end they've done this for full semester towards the end of the semester they said we learned a lot more than if we hadn't if we wouldn't have had to video. But that's pilot example one pilot example to was not a simulation. The first thing you see with the toothbrush in her mouth is an actual patient. The students were asked to, in a clinical situation, actually record their interactions with the patient. Of course the patient had to agree beforehand. The idea of this assignment was they, this is dental hygiene so it's not actually a clinical intervention they're just showing patients how to, you know, perform dental hygiene exercises. And of course the assignment was to practice their motivational instructions to the patients that I had practiced it in the classroom with with Pierce they had practiced it with tutors. The year before that had an exam on that and now during the clinical activities they had to really practice it on a patient. What the tutor also said was, you know, you record this you upload it on our learning platform, and then the other students to Pierce were asked to comment on the video recording. So what you see at the bottom there. Those are markers indicating that there's some feedback by Pierce. So there was a kind of double loop learning there as well. A rather small group of students who did this so their feedback was also positive but not as positive as before, because they felt you know, not only was this the first time they had to use video but they immediately had to use it with patients, which made them very anguished about you know, and it was part of actual assessment, not just formative feedbacks which was part of their actual assessment. Another example is within the teacher education department, teachers of secondary education in flambas. They, the students need to choose to course. Well, how do you call them to specialties when they study to be teacher. So you can be teacher in economics and history or English and French. So they always have to choose to courses for the three year period. They were the economy economy students, and they were simulating an exercise on campus that had to record themselves during what we call a micro teaching. It's a simulation where someone does five or 10 minutes teaching in front of a group of peers. And the peers can have an evaluation scheme or they don't. They depends on the on the teacher at that stage. And what happened was that are the tutor from the college from university provided individual feedback to the students based on the video. The students presented this to the peers the peers would just give feedback, which was not recorded and then the tutor would look at the recording and provide feedback, which combined both the peer feedback and her own feedback and then the assignment for students was based on the feedback that you've gotten during the simulation and from me later on reflect on what you can do next time. So they didn't use a platform for this. The whole recording was put on the platform and students have to re look at it and then reflect on their own function. For example, I want to show you is the second theory cannot economy students. They were asked to take the same setup so the iPads and take it to an actual class during an internship on apprenticeship. Some schools didn't allow us to record students or pupils others allowed us to record them from the back. So we could put the iPad at the back and then record the backs of the pupils but not their faces. And instead of the class periods in Belgium are 50 minutes instead of recording the full 50 minutes. They were asked to select a certain part of the class in which they had, you know, focused on language oriented content teaching. It was a very specific assignment for the students not just, you know, record the 50 minutes and reflect on what you did during those 50 minutes. But how did you handle this language oriented content assignment and reflect on that. The advantage of this was that the tutor wasn't allowed wasn't, you know, expected to watch 50 minute videos for all of the students but only the fragments that the students themselves selected. And we recorded a number of data during this these pilots we had focus group interviews we had surveys. And what we learned from the data was that in some of the pilots the technology acceptance which we measured before and after was lower after the project. Because they expected you know I can handle an iPad I can handle a camera. But once they have to go into a classroom set up the iPad, and then get ready to start teaching, whilst the pupils were all running around in the classroom. They gave them a lot of stress and they felt, you know, I don't really feel safe and comfortable using the technology so that's, we were hoping you know, after the pilot that technology acceptance would have been higher. But it turned out for those pilots that was lower. Those settings where students were asked to share their video with teacher from the university. There the students felt safer, more confident than when they were asked to share it with their peers as well. Because they were afraid some of the video would get out of the closed environment of the university and end up on the internet, whereas they expected teachers not to do that so they felt, you know, they wanted to share with the teacher I'm willing to do that but not share with my peers, especially not digital video in which I am practicing things and can be made fun of. For us very important was that the self reported self reflections course were very high. So they indicated that using video their self reflection have become much more thorough, much more in depth. So we must say that those who responded to the questionnaire post pilot all said you know the experience was positive for me. Same came out more or less about the focus groups the teachers that we involved in the pilots were quite enthusiastic wanted to continue and expand the project. And mainly, I think that's the final point at the gradual introduction of video. But in teacher education those students that started in the first year, and then went on to the second year felt much more safe using the video because they had used it in a simulated environment first and then taken it away to an actual teaching environment. They also felt that they might be less frustrating for them, because we had a number of glitches at the start. We were given an iPad. I wanted to upload the video found out after three weeks that somehow the system had reset itself and wiped away all the videos. So three weeks worth of video recordings were deleted because someone within the learning management system had said, you know, this is a, this is an iPad it's on the road and we have to make sure that it's clean. We said, you know, we see the positive results but somehow using, for instance, iPads on tripods was quite cumbersome and we wouldn't want to have something that's easier to use. We had thought to avoid having them use their own smartphones because we said, you know, recording video takes quite a lot of space on your phone. And we can't expect every student to have one or two gigabyte available on the smartphone. So we said, we'll hand you to an iPad from school you can borrow. But they felt that it was too cumbersome because they had to use a device that they weren't used to. And also, the management of the university expected us to come up with, you know, using video observation and feedback cuts down on the number of actual visits that teachers have to that our teachers have to do on locations to watch the students perform. And all the teachers said no way, we need the location visits to actually see the student in action in a wider context than just on a video camera. I'm thinking that the management won't be very happy with that because they can't save money there. Lessons learned. We noticed that the context of the workplace determines what we can do with video. There's privacy, there's client confidentiality, we have patient data, for instance, in the nursing degree we were not allowed to film any patient at all. Not even discussions between nurses in handing over from one shift to the next, they were not allowed to be filmed because there was sensitive patient data available. In the dental hygiene case, patients felt that this wasn't really sensitive patient information, this was about dental hygiene. And then the patient said, okay, you can fill me because it's not, you know, it's not about whether you have cancer or not. In the nursing department we only filter film simulations within the classroom or on dolls and those kind of, you know, skills that they need to practice minors in school contexts, as I said some schools. Just told us not to film weren't allowed to film at all. In the technology pilots. We asked students to take their smartphone and go on their internship and film what they were doing with certain machines and certain technology and the company company sometimes said you're not allowed to film because this is sensitive company company data. We cannot have this getting outside of the company. There's a number of issues that we need to look into, but they're different for every context that we want to fill in. We can't come up with a general rule and say you can do this and this in that context. The second lesson was that we found that it's very well suited for supporting learning of manual and technical skills that they need to practice. For instance, the example of the windmill maintenance. There's a number of steps that they have to go through in maintaining such a windmill. And the way that they practice first within a classroom context and then on top of a mill supervised by by staff of the company that they're doing the apprenticeship and so those skills are very well suited for video learning because in fact, our university teachers to be standing next to them while they're practicing now. So we asked the students you know practice it and then film that practice which you feel is the best you've done so far, and then send it to me and then I can look at into your view of what your best sample is soft skills. So communication skills as we as I showed in the first pilot, you know this group exercise family conversations. Business to business pilot where students were the assignment was to have a one on one sales conversation with potential client. So that kind of skills, practicing in simulated environments and also in real environments are really very well suited to be recorded and reflected upon and then reflection skills themselves. They also always indicated that their reflection had become much better. That's quite a steep learning curve, both for the teacher and the students. So we will, we have advised the board of the university to allow a gradual scaffolding so start in the first year and simulated environments under supervision of the teacher and then second and third year give students cameras to take with them and use during their apprenticeships. Students need clear and precise assignments or what they need to record or which fragments they need to send into the teacher. The example I told you about the economy class. They were told to film themselves while doing something about language and some of the students just said, you know, okay, I've got 10 classes this week that I'm teaching class number five I'm doing something with language and I'll record that and then take the box I've done that. Whereas the idea was, you know, you have to practice these tools in every class that you teach in the next 10 weeks, but there was no way to control whether students have done that. They had recorded only one example of this assignment. So your assignment needs to be quite clear and precise in order to get the maximum value of the video recording. This was the team that worked on this. Yeah. And this was the funding organization so the European Commission through the Flemish Community Government. That's the people who worked on it. And some related work you can find links to with the UI, if available. And then we're up to question and answer. Thank you very much. Any questions for Steve. All right, thank you. Great. I mean, I really enjoyed the pilot's very, very informative. I was wondering what kind of requirements and equipment they had in terms of editing because I suppose that was something else that might have held them back. Yeah. So the ones that used iPad just use the tools that were available on the iPad because the editing we asked them to do was just, you know, take the 50 minute class and just select one or two fragments and save them separately. So there was no real editing. No, you know, just collect fragments and select them and put them online. Yes, exactly. Any other questions. Another really interesting talk. You mentioned about technology acceptance and that I can well imagine once they realized actually what was involved. You said it was dropping. I just wondered. So with your findings, what were the questions and I suppose, does it mean were you trying to say that it was that they felt less confident because they'd realized, or that they were choosing to reject it, or that they weren't sure how to apply it? We used the Utah instrument. I don't know if you know Utah. Utah. I'm not sure. I think it's the Venkatesh, the user acceptance of information technology model. So it's a standard instrument that we adapted and we used it in the pre and post test and we found in the post test that the scores for, especially technology acceptance have dropped in a few of these pilots. So on average within that group of pilots of students in that pilot, it dropped. So it's based on a standard surface. So looking back in the focus group discussions that they felt, you know, it was a larger barrier than they had expected. Any other questions. Sorry, can I have just one quick question for me, just in terms of the feedback that was being provided. So two part question, one was kind of, did they get some instruction about how to provide feedback with a rubrics or was there any sort of training in advance? And then secondly, was there any, was it kind of a digital feedback process using a shared environment or was it all through the LMS? Yeah, I'll go for the first part of the question. In most of the pilots they had a feedback form that they had also used without a video. So they were used to using this feedback form in kind of peer feedback situations, and they were asked to do the same and then give it to the teacher. We were hoping to use, we used Kaltura to upload the fragments and we were hoping to use the commenting option within Kaltura. But we wanted to have the feedback based on the forms on the observation forms that they had and Kaltura doesn't allow that. So, to the Kaltura people here and that media learning, and they said they don't have that, but you can have preform annotations, as I was, as I showed in the dental hygiene, but you can't use an existing form or rubric to give feedback. You can't upload it next to the Kaltura. You can't use something within your own learning environment to somehow integrate the Kaltura video in an assessment form. And I applied for some funding in-house to build that, but I turned it down. We're looking into that because that's what the teacher said, in order to stimulate the flow of feedback, we would have to have this rubric or whatever within the same environment so that students don't have to look at the video and then mark something down on paper and it's not readily available so far, unless you know of some tool or technology that's available, I haven't found it. And I think, you know, with my Kaltura hat on it, I'm very disappointed that we couldn't support you there, but I think we rely on the LMS to provide those rubrics and once the content is uploaded, but I take your point. I think that's a frustration. I've got to show your colleague at the media learning. You can suggest it to the development team, but he doesn't think that it will, you know, there's not really a market for that. Yeah, yeah. And we talked to the Blackboard people who, which is RVLE and they said, well, there's no priority for us either. There you go. Okay, thank you very much indeed, Stephen. Thank you.