 Yna. informed to be arde, and we've got two really nice presentations coming up the next hour first is going to be Matthew and Charlotte and then we've got Stephen and then we 30 minutes each. If any of you have questions, please just put your hand up on our triangle to use that we've got the audio for the online people. Okay. So, ond, I'm Charlotte, and I'm starting off, and then I'll pass over to Max, and there'll be a bit of interchange of mics and stuff as we go through. But, yeah, we're looking at kind of the jackson video production to enhance user engagement. So with what we're looking at is our users are staff rather than students, but it can be implemented across to students, obviously as well. Not if you just want to move on to the first one. So, we create a mandatory staff course every year. So, for the past three years we've been doing a staff course that they have to complete every year before teaching starts. So it happens to kind of join the summer, we're just coming to the end of this year's mandatory session. And within that we put videos, we have content, how they're supposed to be using the VLE, how they could be using the different tools for teaching and all the different elements that they could be using for that year and any updates. The current 2023 course has been live for two months. It runs for about that length to two and a half months for them to complete it for that summer period. And then last year, obviously we're coming just towards the end. So last year it was 650 plus students, which are academics within that course. They're mainly using desktops and laptops rather than a mobile device. So just thinking about the differences of maybe students might be more mobile device kind of element. And what we did was last year, as we were creating this course, we looked at last year's course content, the video content that we were using, and analysed that content basically. So had a look at their user engagement, where the kind of dropouts were, and different kind of elements of where it was placed within the course and what people were looking at. And now for this year's content, when we were creating at the beginning of the year, we created that kind of data driven content plan. So what works or what seemed to work, what people were looking at, and maybe how we could change it to experiment a bit with that content. So Matt's going to have a look at some of that data from last year's, as well as from this year's and how we kind of compared between the two and what we've kind of done. That's what this is for you. So hopefully that's given you a bit of context anyway. So just to give you a bit of an idea on how we approach this. The main thing I want you to get from this presentation is just the process. I don't really hold all the answers either. It's just the idea of analysing the data, looking at the parts in the video where the dropout happened and just trying to come up with a conclusion. So just to give you, you've got two points on the graph obviously we can see a little bit of a dropout there. Those are the bits I'm interested in. There is a general dropout you're always going to get between the start and the end, but that's not really good for me at all. There could be internet dropout. There could be someone's knocking out the door for a parcel. There could be children next door setting the house on fire. I don't know. It's one of those things. It's just during the summer. There's loads of different reasons where that dropout could occur, but those ones are the most interesting points just there. So I would calculate percentages based on that as well. So look at the starting point point A. It was actually got to that point and then on point B look at the differential between those two numbers and I think on this example it was about a 20% dropout. So it's just looking at those and seeing anything significant and looking at and trying to figure out why. Which inevitably meant that I had to look at video, like a 10 second part of the video over and over and over again and try to figure out what was going on. And the average completion rate is actually it doesn't help me at all. Because if you actually total up the average completion rate and you think it like finishes around here, there's no dropout I can't do anything with that data. It's a misleading stat. So trends and phenomenons. Again, this is like one of the hardest ones to it. I'm doing it and when I've written up the reports on this again, I've done like a 4000 word report on this if anyone was really into data and stuff by all means let me know and I'll send it to you. But I kind of felt like I was like tin hat theory at the end of it like am I going insane am I seeing this thing it's repeating over and over again and one of the things that I keep finding was every 25 to 30 seconds there was almost like an attention hurdle. I've been across all the videos within the two years, it was like, and I try to look at stats and statistics around. Okay, well, do people's attention have a certain point and then they drop out and there's some people saying 47 seconds some people saying 30 something but essentially what I've come to a conclusion of is that's repeating number so you might get them past the first X amount of time but then you have to get them past the next hurdle and then the next hurdle. That's going to happen over a video so you can kind of see that in this example here, very, very slight obviously I've got a bigger version that you can do. In this version, I think it was that was about 6%, 5% and somewhere around that like relatively quite low that was like a relatively good video. This one you can see something a bit more substantial. And I kind of went into this as well with preconceived ideas that okay shorter content is going to be better that longer form content, because of the attention span X, Y, Z, everything that told me about the attention and economy and everything like that but ultimately some of the shorter videos actually have stronger dropouts than the longer ones. It's literally down to delivery, it's down to content, there's a load of nuances there that's going on that I have to kind of uncover and try to figure out. So that's kind of the pre context but the why this is the bit where I'm going to go into this and basically say this is my preconceived ideas on what I think might have happened now. In these examples that I'm going to show you, I made all of the videos and edited them all kind of packaged them all together I've got reasons why that I think they've caused a dropout but again I'm happy to share this content after the fact and then you can watch them and then you can figure out what you think happened here so one of the videos that we do every year is essentially Keith McLean here he basically introduces the course and basically says look welcome to the course. We're going to get a lot out of this it's normally just like a prerequisite to a bit of context right so we do those every year the first video was just a single camera. Just on him second year I did two cameras cut between just two angles and this year and just it up and put some cutaways in so as you can see what they look like here. This is the cutaways now. This is the viewing statistics of it we've got a semi repeating pattern it doesn't happen too much here and the reason why on this 20 to 32nd point here is not like a very visible dropout. I tend to find the first video and of course tends to have the most amount of views, because people are all enthusiastic about the course at this point and then normally here. They hate the content from this point onwards it's a bit more of a task. So it happened a little drop out there but that one is more substantial. At that point. It's these clips here now generally how I tackle film production especially with creating a cutaway package video just a context a lot I do a lot of video work that's my background creating a lot of content shooting and filming. I would normally film like a wide shot, a mid shot and the reverse that and then get like a bit more of a scene a bit more action in the scene and you put that together within 10 to 15 seconds. And it just keeps the viewer just a bit more engaged and giving a bit more context. Now, during that point there is a repeat of shots. I think I've actually shown the scene earlier on but essentially I was scraping the barrel of cutaways at this point I was, I should have really filmed a new scene. It was just, you know, we had like 12 videos to edit by a certain amount of time I didn't necessarily have time to go and reshoot a new scene but to stop that drop out. I mean I've come to the conclusion where I should have just, I should have gone out and filmed a new scene. Essentially, it was just as basic as that the cutaway package was terrible it was boring all the shots kind of looked similar and the same. And yeah, it was boring. It was visually not great. So that's kind of some of the reasons. I think that we kind of do year on year out and I'm going to learn from that. And next year I'm going to make sure at those intervals at those 20 to 30 second intervals that something there's not an excuse for people to drop out. I'm always going to get drop out rates as well. I think you're always going to get some people at those interval hurdles that will have a reason to drop out but if I can minimise that drop out as best I can. That's what I do. One of the things that we did this year that we didn't do last year is we did. This is a one minute video. Hopefully we'll play it for you in a second if it works. And it's a concept video with one core message. Now this sits within the course just on about organising your blackboard folder. It's a simple message. That's all it is. Just make it, just organise your blackboard folder and it will be better for everyone. That's just the core message. And we did it in a metaphor of labelist tins. And student here he takes out a tin of mushy peas instead of beans and that's kind of the core concept. It's very simple idea. Let me see if I can play the video. I want you to try and guess where the dropout is. Hopefully. Does this work? Okay. Let's just pretend that we know what it's playing. I was hoping to try and click on this but it doesn't seem to be doing it. Is it right if you can try just while I waffle on for a few moments. Yeah so obviously the way we tackle this and where this sits in the course the content is completely different. The other stuff is more like documentary where this feels a bit more commercially fictional. It's playful. It's playful. We've got a bit of music underneath. You can see the shots that we're messing around with there. Obviously we're cutting logic out of the process and it was just something relatively quite easy to play around with. If you scroll down hopefully, oh now everyone can see my terribly written notes. Nobody read it. Let's just blur it as much as we can. Take the play out. Now I did make this unlisted just before. Okay great. I just hope there's audio. In our kitchens, we organise things to make them easier to find. You might organise your cans of other similar items and benefit from the clear labels which help you find that can of spaghetti you need. However, imagine if you were just staring at sea of unable cans. How would you find the one that you were looking for? Well for many students, searching for the right resource within a module can feel just like that. It can be confusing, time consuming and just downright frustrating. But there are things that make the search so much easier. For example, when resources are organised into clear categories such as by week and subject or assignments. When resources are organised clearly, your students will be able to find what they need quickly and easily. No more hunting for that can of spaghetti in a sea of unable cans. Just kind of noting now, I was trying to avoid the cliche of students with tin of beans on whatever on toast, but yeah, anyway. We did this and I'm going to, well, where would you just think where would you drop out at that point, right? And then I'm going to. Okay, first can. Yes. Okay, now. You're close, you're close. So basically at that point in there that's within the 2025 to 30 second mark is the steep drop here. We've come to the conclusion that essentially by this point, the video has done its job. It's like people know, okay, great, I've got the concept. I don't need to watch it anymore. And essentially this part of the video is just hammering that fact tone, you know, just just a little bit further. Now, if I was to do this video again. I wouldn't change a thing. I wouldn't, because if I made a change here and made a 30 second version. I wouldn't say that people watching towards the end, but there's still a considerable amount of people watching till the end. So those people there still might just a little bit extra that visualisation to remember those key points. So it was an interesting one. We analyzed it over and over again and yet it was about that tin. It was a tin pouring in the first tin pouring in that kind of was there. And basically that's what the conclusion we came to. We just analyzed that point over and over again. So, congratulations to Rachel online. She actually said after the peas. So, oh, great. Again, it's just like you start thinking about it in that way and start analysing content in the audience perspective and information up to that point and where, you know, because again, it's very important to think about that all attention economy and like the way I think about it is to say, you've got a student going on a course or online course and they've only got 10 tokens worth of information to absorb that period of time and what tokens are they going to do. What tokens are they going to dish out or what are you going to. Why are they going to spend them on essentially so. Yeah, this video did work surprisingly very well even though we had near 21% drop out but again, I'm happy with that. That's fine. Okay, so this one was really interesting said this one. Last year we had we saw this phenomenon where we looked at all the content and anytime this is going to sound like a negative point and then I'm going to bring it back to the fact that's not negative. It was a really hard one to articulate this, because the phenomenon we saw was every time any of our academics in the video content last year mentioned our students there was a drop out. I immediately think, is that a bad thing like they're watching as soon as our students have mentioned I'm not going to watch anymore. Right that was the initial preconceived idea that I potentially had at that time. And I was like, hang on is this like a really bad thing that I've just found here like that that's the, that's the thing that's coming coming around but what we actually found was. We kind of merged that theory to what if it's, what if it's presumed knowledge what if every time we mentioned our students that they already know what those academics in those in that content is going to be talking about what if what if that's it. But then it finally evolved the theory to and again, I can only stress this enough, this is going to be an ongoing process I might never have the exact answer might change every year but these videos we decided to get the student voice in the content. So rather than academics around the university talking about different things that they're doing with their students what if we get the students in it get their voice in it. And they work surprisingly well. They actually had some of the best views and with very, very minimal dropouts again round the 25 to 30 second mark. And they have a range of different, well, range of different reasons why I think might have had the dropouts towards this one towards the end is where the person feels like they're wrapping up they're saying like. It feels like they're wrapping up a point and that's what I tend to find a lot of the dropouts happen towards the end if the presenter in the video is kind of saying like. Thank you for watching whatever if it feels like it's going down those longer lines you're going to get dropouts another reason is more of a technical one as soon as you have an end frame. That's longer than five seconds people is going to drop out as soon as they see that logo come up it's like nope don't need to watch it. It's just that's it. So we analyze a lot of points around there really. Yeah, it was just an interesting one we the point started out. Maybe it's a very negative thing against our academics but actually turned out to be they just wanted to hear, or our ideas that they actually wanted to hear from. I don't really necessarily think this is appropriate saying but from the horse's mouth in this in this case but hopefully that kind of summarizes that a bit more putting the students voice in it, rather than academics talking about the students. Basically, I think in terms of where we're going forward. We're going to look at all this content and analyze it further. And it's just going to be an ongoing process where we try come up to a conclusion we try figure out tweak a few things here and there. But these are the things that I'm basically going to talk about now so. I know we saw the keynote said don't put full of points on the slimes and I'm really sorry I've done that. Yeah, this is kind of the process that we kind of went with. And we designed the content to start out with, and this was the year one we just kind of went into it filmed a lot of stuff chucked it out there and hope for the best. So we created it deployed it. We actually do a review until the second year, like I didn't even think to look at the analytics and look at how people were engaging with this content it was like a task given to me to make the content and chuck it and open sticks. And it was only after the second, the second year where we actually started seeing these trends. I, if you are in a position where you look at video content analytics or anything like that and you can start seeing these very visible dropouts, I would try analyze and must you can to try figure out why those dropouts are occurring, at least as a keep yourself just a bit informed in terms of the execution of the video production I suppose. So yeah, we did a review second year, and then we made some changes. But again this is an ongoing thing. The exact same thing. Next year approach it completely different. One of the things that I tried to do last year was to try and keep consistency with content, right to try and make them all feel part of the same package like it's all part of the same brand or part of the, all the commercial side of me that would probably a brand would want. But that's a bad thing, like viewers start anticipating what that content is going to feel like it's like you want to surprise them. Do one that feels like a documentary do one that feels like more like a metaphor. Do one that just feels like, I don't know. The ones that we do with boxpops with students and stuff where it's just candid grabbing some stuff if you keep the viewer constantly, not anticipating what that content is going to feel like that will also increase engagement. It's just, if they anticipate what's happening it will drop off. I also found his pace so how they present. We've got some excellent academics within this one video. And it was them talking about this new developer Derby site that was just going to be a resource that they can use with their students. Now, in all honesty, I didn't think this content was going to go down that well, it was the way I filmed it was interviews to cameras. The second camera angle was handheld so they had a bit of pace to it as well but they general passion for what they were talking about and how they applied those pedagogical practices into their learning and teaching was the thing that was contagious while watching it. It was like it just kept you going it just didn't feel like it was necessarily a bad thing to watch that's all I did just to camera angles and cut it through I thought I thought I was going to have major dropouts in that video. It was one of the longest pieces of content we did it was like two minutes 20, and then the other one was like almost three minutes, and it had the biggest engagement, and it was just a really simple thing I thought I'd have to chuck in loads of cutaways. I don't know animation halfway through dragonfly and I don't know something something exciting visually but this was the. This was, it was just a thing I couldn't really I didn't really plan for. I think you just repeat the question. So how was I the approach to that one what surprised me. Okay, so I think the previous year I shot a load of interviews. And that was it the most exciting thing about it was maybe a couple of times to put audio underneath cut between the two camera angles. And that's it it was relatively visually quite dry. And that's the thing that surprised me. It was the thing that kept them going was their passion for it and they're just and they were so animated. You know, they were just genuinely passionate about whether we're talking about so. Yeah. How they present is a big factor. And I think that's with all content as well just imagine. It could be the most interesting subject in the world but if you have somebody very dry stiff. It's going to. Yeah, it's with anything. I mean it's kind of similar to what the keynote was saying this morning about how they present in front of people as well. There's a way that you can get people to switch off. Yes. Okay, this one's an interesting one. I might not get through all of these apologies, but I'll try to try to rattle them off as quickly as I can time for a single call to action messages. So it's very similar to that metaphor video that I made that was. I know it was one minute of that same 30 seconds tops, but I will make rules and break them all the time just to let you know. It's one of those ones where within those 30 seconds that one point is probably enough cognitively to process. If you overload it too much. That can be a point where people drop out as well. We did videos where it would be somebody talking about something that's coming to the new blackboard or whatever and their applications of it. And then I'll cut to a screen sharing thing of showing them doing it through blackboard and the video up until that point the pace felt more documentary feel there was cutaways whatever and then eventually you just get to this. Kind of stagnant. I didn't really do anything else than just a screen recording. And then that was at a crosspoint of the attention hurdle 25 to 30 second it was a killer. It was just plummeted the video. So I think the main thing is with our students and in this case our phenomenon is between 25 to 30 seconds that might be a completely different environment for you. You might see something completely different. But again, I'm not trying to get across the the rule of 25 to 30 seconds is gospel. It's just the approach that we take. Just checking on time. Five minutes five minutes. Great. So context is a very good point in this if I think if we just randomly surprised them with the video on the organizing your blackboard content. I think we would have got a less of a drop out rate but because the context of where we put that video was on organizing your blackboard and they've already read some of this content anyway. So it's just geared up to the fact that this video is kind of going to be around that point as well. So, but then it goes down to a design point of view don't really want to chuck around a video about different content and somewhere else. It's kind of weighing up that balance. And I think the main kind of point I want to go across. And then I've got a little bit of time for questions is. If you can make changes, you can think about some of the decision making that you can make with cutaways and at those points that you're going to put them in the video but ultimately you don't want to make a change that and then alienates the previous people that watch to the end. Because ultimately then you're just making a change and there's just a new group of people that are going to drop out and then that makes sense. Hopefully I've articulated that in the best way possible. So some of the changes we can, we can make and some of them, I would keep the same, even if we are just going to you're going to get dropouts, no one's, no one's going to make a perfect video I don't think. Unless you do please let me know and then I'll steal the secrets. But yes that's that kind of hopefully I've kind of answered all the questions Charlotte have you got anything else that you want to add just before and then we'll leave a couple of minutes for questions. Charlotte's just responding to a couple of questions we got online, which is great. Anybody else got any questions from Matthew and Charlotte. Yes. Sorry, can we just do. Well firstly thank you thanks for being so honest that was really, really useful. And can I get your report please. Yeah of course. I mean it will go through all of the content specifically and I'll break down in the report where the dropouts happen, the percentage of the dropouts and what I think and believe each dropout is the reason for so. Yeah it's a bit more in depth and yeah there's a summary at the top actually just need to read that bit that's fine. Thank you very much. Dropouts then. Yeah. Yeah, again thank you very much. Very interesting. I want to rewind all the way to the top because one of the things that I find very frustrating ease. You know you create something and then you can't send people that all these nice content in your case people so it in my case I think people wouldn't really necessarily see it at all. Is that how did you in essence get the buy in to be able to design something that is to three months long and you know stuff actually go there. Yeah, so I mean this is where we're incredibly fortunate right so let's just take. I'll get to your question but let's just take a normal lecture experience and maybe you've got at best 100 students or plus depending on what institution you have, and you drop some content in a module. So if you really look and you watch this by the end of next week, you're not going to get a consistent approach on viewership. It's just one of those things where at best you might the data won't be accurate I don't think, and people will watch that in a very different way whereas we were in a very fortunate approach of this is semi policy driven that they have to kind of watch this content and basically up skill themselves. And because of that we have a lot of students academics going through this process and watching that content so it made the trends very clear, and it was just a really nice. Yeah, it's policy driven, give us the data we need, and then I can actually make those decisions as well because another thing you can't do is maybe within a lecture you play a piece of video in front of a load of students you don't necessarily know when they're dropping out mentally. Like that's a hard, it's hard thing to kind of get so very, very fortunate position to kind of use this data in a way to create a decision made. So yeah, policy, sorry, I think I've answered the question. Okay, we're sort of out of time I think unless there's a burning final question from anybody. Okay, well, thank you very much indeed Matthew and Charlotte.