 I'm John Wilson. I'm the CEO at Agenta. We're a technology company that focuses on education and learning. We build, manage and operate platforms for education, for video collaboration. Externally, we prefer to work with what we feel is ethical industries. Obviously education, teaching, learning, healthcare. We feel that we can really contribute to these industries by creating exciting platforms, easy to use platforms, secure platforms that people can utilise. What we feel is one of the most important things for Scotland to boost economic growth is investing in rural areas. By investing in broadband in these local areas, we can attract more talent, we can attract more companies and we can drastically improve the delivery of education and learning within these schools, within disparate regions, within Scotland. One to the first afternoon session of the last day of ALT. Hopefully you're here to see two sessions. The first one is What Lies Beneath, Reflections on a Community Consultation and Ethnographic Research on the Implications of the Use of Technology for Teaching Practices. Thanks for making that so easy. From Donna Lankler and Laurie Fitz. That will be followed by two sides of the same coin, exploring student and tutor perspectives to develop a best practice model of online learning. That's Osmond Javade from Manchester Met. Over to Laurie and Donna. That's your introduction. There's so many words, that's all you need. We didn't want an abstract, we just wanted a title. Yeah. Oh, all the mics are gone. We have to be there. Okay, I was going to move around. Good afternoon. We appreciate you coming back after lunch on the last day of ALT. I know three-day conferences are challenging for me. They're probably challenging for lots of people. So I'm Donna. I'm Laurie Fitz. I'm from JISC. I'm an anthropologist. So we're going to talk to you today about... Yes, that's why I wanted to move around, Mike. Oh, this... I can do this. Awesome. We're going to talk to you about a project that we've been working on for the last year. So we started doing this research in 20... What year is it? 2017. 2017, we started doing this work. And this is the first time that we have been able to talk publicly about it. So we're very excited. Yeah. So what we've done is we've been working with the sector. What we did originally is we had the co-design process. And a lot of people in this room will have been involved in that. Where we talked about the next generation digital learning environment. Thank you, me too, for putting that back on the bingo card. We looked at what the next generation of digital learning environments might be, and we asked some questions. We asked what would an environment do for staff and students. We asked what kind of learning experience would an environment provide. And we asked what learning and teaching practices aren't currently supported within the current crop of learning environments. Now, we went through several months of consultations with the sector. We talked to an awful lot of people. I think we had something like 7,000 Twitter engagements and over 100 blog posts were written about the next generation digital learning environments during that period. And not just from the UK. We had engagements from the US, from Australia, from Europe, especially Finland and Sweden. I'm not sure why. But we actually then started to pull all of this together into a retrospective of that particular aspect of next generation digital learning environments. And we started to sort of draw out some of the themes. Current good practice was one of them because there's an awful lot of good practice that people are already doing. People know how to use the environments they've got, right? We started to see the emergence of large enterprise approaches. Lots of people were talking about the fact that Microsoft and Google and even Facebook had started to get involved in the learning environment space. We started to look at the rise of the individual in terms of their practices. Seeing lots of people like academics setting up their own, for example Slack channels for small cohorts of students and things like that. We saw, of course, analytics writ large across the whole thing. But we saw analytics in lots of different guises. We saw it in terms of what's going on in institutions and that reporting. But lots of people were seeing the benefits of analytics and how analytics can help support some aspects of learning as well. And there were various other emergent models that we'd started to pick up on. So we started looking at that and I say we had lots of work around that. These are some of the key things that came out of the report. The emphasis on the administration of learning and teaching. Innovation in learning technology we found tended to happen outside of the virtual learning environment according to what we were doing. We also started looking at the changing behaviours of academics and the changing behaviours of students. And that's where this report emerged from as a result of this consultation. We started looking at, well, what are the teaching behaviours of academics? I just want to emphasise something that we didn't do the previous slide. This is amplifying and advocating rather than discovery. So this is a piece of research and we're going to tell you the things that we think are going on. But we're under no illusions that this is something that has never been said before. So what we're trying to do is boost the voices that are coming through in our research, not pretend that we've discovered this for the first time. So I'm so bad at titles and my apologies to Becky for having to read the title. This is the working title of the report. This is all the stuff we're trying to do. We wanted to move away from a sense of talking very specifically starting off with digital and we wanted to ground it in the behaviours of people who were teaching in the sector. The pitch that I made for the project was, you can't talk about any sort of learning environment until you've talked to people about what teaching is. And what teaching practices are and what teaching means to them. So this is what we did. So over the course of the year we conducted 11 interviews. It's across the range of disciplines. So we tried to get as wide a range as possible within quite frankly our convenient sample. We put out a call to people to see who would talk to us and we were really pleased with the amount of diversity that we got. We had people who were senior in their institutions, people who were very new in their institutions and it was HE and FE and Post 92 and Russell Group and Red Brick. We tried to do all the things as best we could within our sample of 11. We took the interviews that we did and we processed them so that each individual thing that a person said became a standalone statement. And when all of those were printed out, it was about 1500 things that people said about teaching for those of you who are in love with numbers. And the analysis was not just done by myself and Laurie but also in conjunction with a group of partners who we brought in to help us think through what we were getting a handle on here. And so you will have access to the slide deck at some point but these are the interview questions that we did. And so what a contextual inquiry is that it's basically a very specific way of dealing with structured and semi-structured interviews. And the idea is that you interview people within their context so that you can learn in depth about what they're doing and why so that you can be there in their space talking to them about their practice so that when they say, well, I tend to do this thing or I learned about it in this context, you can point to the surroundings if that helps you. You have this photo, what does this mean? See, you've got these books arranged on the shelf in a particular way. What is that doing? So it is in context. And so ethnographic in that sense but not sort of long, slow, deep ethnography like a classic anthropologist would do. And you can see it's very sort of wide-ranging questions about what is teaching, where do you teach, how did you learn how to teach, how do you talk about teaching, with whom do you communicate about teaching and your practices? What's the role of research in the role that you have? How does that inform your teaching? What sort of support do you get for teaching? You will notice, but nowhere in these questions do we say what kind of technology do you use when you teach? We never ask people direct questions about tech. We started off asking them about teaching and I think that's important and it informs the way that people were willing to talk to us about their practices. So just so you can see what it looked like in the analysis phase. So what I would do is we would interview them sometimes as a team and sometimes myself alone. I would take handwritten notes. So I was synthesizing at the time that I was interviewing them. I would then go in and transcribe my handwritten notes because after about 24 hours I can't read my own handwriting anymore. And then I would end up putting them in a spreadsheet. So each, as I've said, standalone statement would be in the cell of a spreadsheet and then this is the code of the person who we were interviewing. And then these thematic codes came later but they ended up in the spreadsheet. So this is what one piece of data looks like for this project. So what we did is once we got all the data we printed out every single piece of data and then we cut it up into small pieces of paper and put them in plastic bags. That's our research method. We then brought together people in the sector, teachers, librarians, researchers, people from GISC, people from other NGOs. Instructional and designers, technology. And we said let's get together, let's get in a room and let's look at the themes. Now we could have used a piece of software, we could have used something like Envivo to sort of try and elicit the themes and actually have an algorithm do that. But what we found is by doing it this way we got people engaged with what was going on. And here you can see a few people just sort of going through. You can see this is just one wall of four walls of the data being analysed. So the big post-its are the large buckets of codes and then we would have smaller sub-themes emerging and we would move things around. We would have these things taped up and then we would sort of visit and revisit them. We'd say well I don't think this piece actually belongs in barriers. I think it belongs over in students. We started very early in the morning and we finished very late but we were very kind. We gave them a sandwich lunch. But all of this sort of, every piece of data, we didn't finish it, every piece of data had actually been put into a theme or a sub-theme. And the themes just kept changing and emerging so that as we sort of progressed through the day rather than just having a simple output saying oh look, all these are the big keywords we wanted to have something that was more contextual that actually meant something to the people that were in the room. So for you social scientists out there we were doing grounded analysis. We were not starting with a sense of the themes that we wanted to find. We looked at all of the things and then saw what emerged from what we had. Go through that and sort of pulling that out. So these are the, now there are a lot of themes. You could see on that previous slide there are actually more themes there than I've actually put on this slide. What we wanted to do for this report is get the first draft report and a published report out to the sector. So we decided to focus on these five themes for this report. We still have masses of data for smaller sub-themes as well. And we picked these themes specifically to respond to the issues that came out of the NGDLE community consultation. So we felt like these are the ones that resonated best with the things that people were saying in the sector in the previous piece of work that just came out. So it followed on from the stuff that we were getting about the actual technology, the actual digital learning environments. So once we focused on teaching places where people actually do their practice, students, obviously, change, innovation and risk came through massively and I don't think we totally anticipated that. I did. Hashtag by-roll. Organisational support for teaching and of course technology. Technology came through almost all of the... Everybody talked about technology. So this is very similar to what I say to libraries when they want to know what people think about libraries. Don't actually ask them about libraries. Don't ask them about technology. We didn't say, tell us about the tech you use. They just told us about the tech they used in both positive and negative ways. But it just came out because it's embedded in their practice. So I'll just run you through some of the kinds of things that are sub-themes within these themes. The teaching places theme had a lot of stuff in the conversations around what they don't have access to. We don't have enough of the kinds of rooms that I want to be teaching in. So they might be in a place where they have a lot of lecture halls available but they don't have as many rooms where their students can get up and move around and interact with each other. They don't have flexible teaching spaces and so they're stuck often with spaces like this even though they might want to do group work. Access is not just for them but also for students. So not all teaching spaces are classroom spaces. If students want to be in an environment where they're teaching each other for instance and can't have access to classrooms and don't have access to less structured learning spaces like lounges or collaborative learning areas, this is a problem. And they were aware that students can't always get to the places that they want to be in to be able to learn. Offices showed up as teaching places. Offices are places where they interact with students one on one, where they have confidential meetings and so this wave of open plan offices and hot desking in academia is having a direct impact on the kind of work that academics can do with their students in safe spaces. And so we had people coping with that lack of access to private spaces by scheduling. So they might be sharing a space and so some people would say well I'm going to meet with my students on Tuesday and you can meet with your students on Wednesday and I'll make myself scarce. So they were using calendar to sort of adapt to the fact that they didn't have enough physical spaces. Other places here is off campus field trips going to sites for ecology research and things like this. So not all teaching places and you guys know this, not all teaching places are classrooms and it was really important to elicit that the range the landscape of teaching places that are relevant and then what the implications of digital are for access to, for enhancing for making these spaces work for them and especially for hacking spaces. Students, the notion that trust is a huge part of what they're trying to build when they're in these teaching spaces so the trust is linked to the spaces because often what they would say is I really value going on field trips with my students I really value going on co-curricular activities with my students are being in cafes and being in places where I'm not the sage on the stage because then they'll talk to me about everything. Assessment is frequently the tail wagging the dog so a lot of their practices around teaching are informed by assessment. This is important came through the sense that the hesitancy and what they put online, what they put into these digital systems might put them at risk. If they capture all of my content, if they have all of my lectures then maybe they don't need me anymore so the political situation of precarity and higher education is having a direct impact on the extent to which people want to engage with technology. Somebody did actually say to us ed tech will replace us. This was during the strike as well so it wasn't some sort of abstract notion It was a real they actually said it visceral. I won't go through all of these but again these are the sorts of things that we got. Organizational support or lack thereof there's not enough time how can they build a network, how can they find people that they want to talk to about teaching when it's not built into their contracts and the barriers they found were generally not technology ones as much as people bitch and moan about VLEs the barriers that people were talking about were other people. I think the other thing is one of the things that we picked up during the interviews innovation seems now to be writ large in every institutional strategy. Show of hands, how many people have got innovation in teaching in their strategy? It's important to distinguish that digital and innovation are not the same thing but it seemed to default in some of these cases. We had people talking about really traditional things they were doing in the VLE as content depositories and again this is familiar to you, right? That's not innovative it's digital but it's not particularly new or exciting. So the implications of what we've been doing this one very quickly there's a disconnect between what we provide institutionally for teachers to use what they actually want to do and what's possible. What we mean by that is is the institution provides a tool maybe says use this tool they then start using it but then it becomes invisible to them what they could actually do beyond that thing and it becomes something that stifles the innovation. So they're trying so hard to do what they want to do with the stuff that they've been provided that isn't actually a good fit. They're trying to do something at what they are presented with that they never get to a point where they can actually imagine what's possible. So if these circles started to overlap more people would have more access to the possibilities that are currently not visible at all. And the invisibility would disappear. And this came through very strongly as well and maybe we've heard some of this week especially during the keynotes but when you mandate technology it's not the same as supporting teaching. When you say you will use this tool it's not the same as saying I am helping you in your teaching practice. What did Tracy say? I make sure I've got a shell in the learning management system for each of my classes because that's what they told me I need to do but all of her active teaching practices were happening outside of the mandated institutional presence. And mandating technology is also not the same as innovation. This is shot through everything and I think this is one of the most important themes to sit with when we're thinking about why and how people make the decisions they make around teaching. And it's not just about trust between instructor and student although that was a big deal. If the people working in your institution don't trust you and or trust the technology that you are in charge of as an instructional designer as a learning technologist they're not going to engage with you. We all know that when you start delivering teaching it's not just the teacher it's the staff developers it's the educational developers it's the people that are involved in curriculum design it's the people that are involved in timetabling it's the learning technologists when you're trying to provide an holistic experience what we found is that at any point on that circle the trust is broken then things start to go wrong. If students go into a system that doesn't work they never go back If faculty have that experience with somebody who is supposed to be helping them they never go back If they try to upload a piece of content to a system and it fails they never go back. So this needs to be kept in mind all of the time I know it doesn't surprise any of you but I don't think we talk about it enough. That's us. This is us and we have time for questions. Thank you so much for fitting that into 20 minutes I feel like we could talk about that all day We've got 10 minutes for questions and we're going to have Becky choose I'm going to read that Do we have the not anything online that will be in the room You can do the me too thing or we can We have Roving Mike if someone wants to That's awesome Here we go The Mike's Running Woman It's An Camel from the University I noticed when you were talking about spaces that people had a variety to what were all physical spaces didn't you introduce did none of them talk about online or digital spaces They did, they did talk about digital spaces and we didn't have it up on that slide but people talked about discussion boards people talked about social media as a teaching space and they also talked about digital spaces that they would not go into because they wanted to make sure that students would interact with each other so we had one lecture who had a space set up where her students knew she was present and she would be there as a teaching presence but then she encouraged them to set up a separate space that she was never going to go into because then they needed to do different things over there so absolutely digital is a place in these as well So we put physical spaces but even in the language that people were using talking about they were using words that you would describe as a physical space so sometimes when they were talking about spaces you have to check where they are talking about But the same things around structured and unstructured spaces institutionally supported spaces and non-institutionally supported that's true in physical as well as digital spaces so the themes cross cut whether it's digital or physical Thank you I think Catherine had a question and then we've got some coming in online Excellent So the first one we've got here is you mentioned the main barriers were human not technological What were the main human barriers? Trust was a big thing So a lot of it is if those people are not in your network you're not going to talk to them about your teaching So if they didn't identify anybody on their institutional network as somebody that they could talk to about teaching they would try to find somebody that they went to a graduate school with or somebody that they would meet at a conference or they would just sort of resign themselves to the fact that they didn't have a network or they would sort of wish that they could do that The other thing about the question was the barriers that were human it was sometimes a physical barrier as well it was the fact that they couldn't get time with somebody they couldn't I don't want to play to the audience but if you're in a university and there's only five learning technologists and you want to do something really innovative with learning technology come and see me at the end of the day when I've done all the jobs that I'm supposed to do Another barrier was quite frankly administrative higher ups who didn't see the point of engaging with technology for the sake of education So there wasn't any space being made by the leadership to be able to engage in these sorts of things Catherine Thank you both Alluding to what you said before about you expected that notion about change to come up that theme So I'm not surprised at the names of the themes but there's a lot of richness in what you're identifying within each of those subtleties that I think are really important So there are people in institutions who are aware of these and are trying to fight these saddles so how do you anticipate this resource being used to support those That's you So I have hopes but I don't know if that's going to come to pass The purpose for the report from my perspective of JISC is that we wanted to know how people teach There's lots been written, lots been said We wanted to do it in this way to actually hear the voices of the teachers rather than hearing second hand voices or we get lots of market research for example from vendors but we get lots of market research telling us We wanted to go into institutions and listen to the voices and for JISC to be able to say this is what teachers are telling us so that when we start looking at things that we're doing next we've got work on the intelligent campus we've got analytics writ large at JISC and we've also got some work coming up in the next generation digital learning environment when that work starts to progress and starts to develop things that can be used in institutions then we want this report to be able to influence that work but also it's going to be available it's not going to just sit in a desk this is going online it will be available as a resource and the data will be available as a resource for other people to interpret So my primary intention actually was to get these voices to JISC's attention and part of the reason we did the initial analysis was to get people from that org into the room and touching the data at reading the words of the people who teach in the sector so that there couldn't be any what we think they say this or I've heard the people who teach think this but they were confronted with the words of practitioners and it is my hope that that then will be kind of an antidote to the assumptions that often run through some of the policy work in the sector and some of the things in behaviour Okay, so we've got about three minutes left and you've seen up there one of the questions is, where can we read it? Well, where can we read the important fault? We're writing it up now we're doing the implications Some of us have done all of the writing Some of us have to write some recommendations The recommendations are being written now we are looking at releasing a draft for the community to engage with as well because once we've done this and we now have a draft report at the moment we can now release the draft reports get more feedback from people that are in teaching places and give us more feedback about what they are seeing and that feeds the richness and it keeps the dates coming but there will also be a full report republished this academic year as well I wrote all of that We've written most of it The first draft of it was a 40 page report and now I think we've whittled it down to about 25 So we've got a couple of comments and questions about academics and students sort of related, so one is their research on academics or students not going back if the tech fails I assume that means not going back to the tech and the other one that might relate to it is was there any disconnect between student and teacher perspectives Well, it's not a student-centered project it's a teacher-centered project so if anybody would like to work with me to get the funding to hire me to do the student learning contextual inquiry I would be delighted to have a conversation with you about that so that wasn't the focus of this particular one When it comes to the tech failing yes there is research we've just done if one person tells us that they went to the BLE and it failed that's research because if it's happened to one person it's happened to more than one person we keep asking ed techs the same question Well have you got research to back this up? How often do we turn around to a lecturer who stands in the front of a lecture theatre and say have you got evidence to say that what you're doing is effective? Yes, we've just been out and spoke to academics who said it failed so I never went back and how many said it academics said and how many times does the stuff have to fail before we give that message credibility like how many people have to decide that they're not going to engage with technology on campus because it doesn't serve them or their students for us as a sector to respond to that as an actual need 400? 600? So last one and this is kind of an interesting counter that you've just said I think did you uncover any scenarios where digital was being used to increase trust? So yes I'm thinking about how in the sense that it proliferated the number of places where you could communicate with your students there was one practitioner in particular I'm thinking of who was very mindful about his social media presence as a way of breaking down barriers between students and academics because sometimes it's hard for academics to frame themselves as people right because they're teachers they're authority figures and so he very deliberately curated a mode in particular social media accounts to make connections with his students that he couldn't otherwise make in formal structured classroom environment and that worked really well but it then bore fruit when he was doing the fieldwork so when he was actually then interacting with them in social situations and in fieldwork situations that actually was manifest. But he was super mindful about it and he was never under the impression that just because he has a social media account that that got him in good with his students. It was about what he did with his presence online that made the difference. It was carefully curated and he was aimed at students. I'm afraid we're out of time. That's okay. Donna and Laurie's contact details are up there if you want to chase them up for more information. Thank you very much. So if you came in partway through that next up we have two sides of the same coin exploring student and tutor perspectives to develop a best practice model of online learning. That's not really got much to do with my presentation but I just liked it. It links to a theme of the findings that came out which we'll talk about in a minute but I just like this one. So I'm Osmond. I'm an e-learning manager at Manchester Met and I'm going to talk a little bit about some research that I carried out for the Faculty of Health Psychology and Social Care on one of the online programs that they run. So I'll just give you a brief background to the study itself and then I'll go through some of the lessons that we learned. So the program itself was an MSc Psychology Masters program which is a 12 month program and it was designed for non-psychology graduates. So what you found was that the students who were particularly coming onto this program were people who had already gone through the university system, mature students, very often working full-time or part-time alongside these studies, had young families saw very different demographic from what the department was used to in terms of the students that were studying on this program. And as I mentioned it was completely online which was again a new thing for this faculty. They've never studied on a completely online program before and then to add to that after the first run of the program in 2015 they ramped up the numbers from 13 which was essentially a pilot all the way up to around 100 students and as you can imagine that had huge implications for the delivery of it, the student experience and it was at that point where I kind of carried out this study looking at the staff perspective, the student perspective, what things were what things are like, what things were working as a means of improving the program moving forward and from that the programs really kind of gone from strength to strength after tweaks year on year, it's now one of the most popular programs, online programs that the university delivers. Just very quickly the kind of approach that I took for this study distributed two terminally surveys so these were free text comments, didn't really want to presuppose what students were going to be saying, letting them talk in their own language about what they thought was important and this was a really, really rich data set so we had comments which were like over 300 words sometimes for each question and the questions were on all aspects of the program and as it says there are 350 comments for each survey had a really good response rate around 40 percent then after analysing those thematically we then kind of went in to do a focus group with these students to really young pick what do we mean by these themes what is it that you kind of mean by this and really get a real deep understanding of what the key themes were that was important to the students for the online program and then taken that to staff to say this is what students are saying on the program this is what they like, this is what's not working what's your perspective on this what was your intention behind doing something like this have you thought about this and really that kind of dialogue between staff and students was the real focus of this research and then from my perspective we're working as a learning technologist to look for opportunities for intervention to say well how can we look at this and build on the commonalities where they are and if there's disparities those are behind this why are staff thinking about certain aspects different to what students are thinking about so before I do go into the themes of what came out the overall themes I thought it would be interesting just to break up the session a bit and ask for what you think might have been important on this obviously there's going to be a lot of people with the experience of either being a student or teaching online programs and blended programs but I thought if rather than asking questions at this point under me too things just ask maybe put a few comments on things that you think might have been important and maybe then we can just have a look and maybe compare and contrast with what I thought what we found on this study if I just give you a couple of minutes just to do that and I think hopefully that will so if you can just put up on the screens the comments that would be really good if you maybe if you just talk to the person next to you or just put up some comments again just from what you think from a student perspective what was important what things are important to consider or from a staff perspective what are important aspects to taking into account if we just put that on the screen and also if you see a comment that's particularly pertinent give it a like and then hopefully we can get those ranked and the ones that are most popular will show up on the screen so there's quite a few comments around community community which I think which always comes up this is always an important part time obviously I think time both from staff and student perspective that's something that always comes out I think some of the comments are coming through the generic comments that you would assume for a student any student but they're amplified for online students or things about not being able to submit online or not being able to submit at all I think those kind of things anxieties really are amplified when you're online because you don't have anybody to talk to of a physical space to be able to drop something in I'll just put that up there these are the things that students said that was important for them and a lot of them are being covered here but I think one of the interesting things there about a community it did come up in the findings but it wasn't a distinct theme in itself it was always in relation to things like interaction or wanting to feel that social presence but the community itself because these guys were working full time and loads of them had families and busy lives I think some of that took a backseat and the thing that really focused their attention was I need to get my assessments in on time and I think sometimes a community thing didn't play as big a part as you might do for an undergraduate course where you really need to feel that sense of a community study but yeah the time one is something that we're going to come back to in a few minutes but yeah I think particularly from a staff perspective that was something that really came out that staff didn't have enough time and that needs to be acknowledged of the amount of time that goes into developing an online programme but what I thought I'd do now is just go through a few kind of many case studies of some of the findings obviously wouldn't be possible to go through all of them but these are the overall kind of findings of what the students found important and again I think a lot of them are covered here things like support, academic support pastoral support that covers things like timeliness of communications which I think was one of the comments here as well interaction of learning so again that interaction with peers and students so I mean it's all and I think with a room like this some of these things might seem obvious in common sense but I'm in an institution where some of these things you know aren't very obvious to certain people so I think part of this conversation needs to be that the right kind of people need to kind of hear this information I guess and that's something I'll again talk about in a minute so the first thing I wanted to talk about was the individual unit areas so students talked about the fact that they were getting inconsistent units within each one of their learning spaces they were having inconsistent information it was labelled differently sometimes assessments were located in different spaces and because these guys were time poor we all know that you don't have enough time to be able to get your assignments in on time never mind search for contact details or trying to find the weekly activities that week and that often that important information was sometimes missed but from a staff perspective they kind of countered that by saying well there was a lot of information that staff had to give to students things around exceptional factors or the elective units or all that kind of information had to be given to students and because they were in online and they were in that week they had to give all that information in one go another thing was about some of the cheaters talking about the fact that they felt constrained by having to have this consistency with other units they weren't able to teach in a fluid manner as they might be able to in a face-to-face environment so some of them relied on this kind of almost serendipity of when you're teaching some of the things just kind of the penny drops but you weren't able to kind of build that into an online session some of the cheaters felt and having to kind of front load all your resources at the beginning of term really made that difficult now from my perspective trying to take a neutral role in this it was really obvious that all of the staff comments were relating to their own unit areas and that kind of carried over from the way that it's taught in a face-to-face environment but in Moodle each unit area has his own unit leader who has ownership of that unit but they take care of their own unit but they don't necessarily think about the whole programmer as a whole and we all know students see a single programme not individual unit areas so my role was to kind of think about that whole programme level thinking and I did that by kind of incorporating Moodle template to try to bring in that consistency across the different unit spaces but also leaving them flexible enough for different tutors to be able to build on their own style of teaching into it and make it easy enough for their own subject matter to be incorporated into those templates with the idea of just enough information and just enough time to stagger the information to say you don't need to know about assessments on your first day or Harvard referencing or whatever it might be and we picked points in the turn where we thought this would be more pertinent for the students to be able to access that information and obviously make it available and signpost to it but not to bombard them with all of that information at the outset. Now these kinds of things that have happened here they're ongoing issues inserting a template is not a quick fix you can't just say everything's done now templates can be broken they can be ignored you can imply a consistent template inconsistently I think it's an ongoing thing of being able to kind of work with the tutors to say well this is what's working here that's not what's working there and that more iterative approach and similarly with that idea of just enough information, just enough time just coming different things to different tutors so again just working with them really to get that overall cohesiveness to the program and really thinking about what the online journey should look like feedback on formative activities so students talked about not having enough personalised feedback or wanting to have that kind of validation of their opinions and just making sure that they were understanding the material correctly as they were moving through the program as you can imagine staff were concerned about the amount of work that would be required responding individually to 80 or 90 students on a weekly basis and they also talked about the fact that some of these students were carrying over their experiences of undergraduate studies over to level 7 masters level and where they needed to be a bit more independent in their approach but I think ultimately what's happened here is that students, well staff were in a situation where they knew a group of students very intimately 13 students and then that same pedagogic model was used for a group of 100 students where it was unrealistic for them to be able to know them kind of students very well being able to engage with them in the same way and I think that really speaks to the people who are in charge of resourcing the program so the scale up of it really it was assumed that staff would just do more of the same, the program had been set up in the first instance, the resources had been created now we can just let it run and really when you scale up a program from 13 to 100 you really need to either increase the staff resource appropriately or you can choose to change the pedagogic model to then fit the resource that you currently have but mechanisms like discussion forums if you're using that with a group of 10 or 15 students that's unrealistic that you're going to be able to use those kinds of same mechanisms on a bigger program with the same staff resource so I think that was one of the issues that so ultimately the approach was taken to kind of modify the way that the program worked to introduce more automated ways of feeding back so things like quizzes where you had that personalized feedback for students but also workbook activities where you had that focused interaction with a piece of reading but importantly expectation management both from students of how what it takes to come on a program what kind of interaction you can expect for staff as well so this is what online learning looks like particularly for those outside of the department that are unfamiliar with this type of teaching there's sometimes that assumption that well you can just put it online and leave them to it or that it's a cost saving exercise and again I think one of the big draws of this program was that you were able to reach students who previously couldn't have come to university full time and studied on a program like this and I think treating it as something where you can say money or time I think it's probably not the best way to look at it so those kinds of messages need to be articulated and just very finely webinars this was part of the overall interaction theme but things that we've kind of come out in some of these comments as well to do with the social aspect of learning and students talking about the fact that they weren't able to they felt sometimes isolated from the university and these webinars were really really good tools of fact that they could see other students they could talk to them in real time and really had that human interaction but the fact that they didn't go far enough or some of them weren't as interactive as they would have liked and staff talked about the fact that there were a lot of students that they weren't able to have those kind of interactions as personalised as they would have liked sometimes there was students who would dominate discussions which made it difficult to again have those very interactive kind of discussion type sessions but ultimately this speaks to the same point in the fact that you scale up the program and then you need to kind of deal with the fact that interaction needs to change you can't have the same kinds of interactions based on a previous pedagogic model and also the fact that the other thing that came out of this was that staff acknowledged that these webinars sometimes weren't as engaging as they should have been but they didn't have the time to plan so I mean you need to have an assessment plan, you need to plan for these kinds of interactions and if that's not acknowledged by the powers that be of what resources required for delivering this program then you can't expect them to plan these sessions in particularly for those who are new to this type of delivery so staff development sessions were put on and we talked about how we might be able to deliver webinars to larger scale numbers but again if staff don't have time to either attend or to implement that it's questionable whether there's any merit in delivering those staff development sessions one of the things that we did do was to develop more webinars around assessments and that was to counter the fact that some students felt quite vulnerable online, they weren't able to engage if they felt uncomfortable with the fact that the knowledge of these webinars being recorded and their lack of understanding as it was to be broadcast to the whole group and so if they were talking about something that was more personal to them and they had more experience around they'd be able to open up a lot more and facilitate those discussions in a more engaging way so just very finely some of the lessons that we took away from delivering this programme that overall programme level thinking is key I think and my role in that in putting those templates together working with academic staff trying to sell the idea that these templates are a good idea and also selling them in a way that's beneficial to staff as well so we are saving staff time you don't need to populate your moodle areas every single year that's an important aspect of it trying to win the hearts and minds of people you're working with as I think Llorian mentioned as well, it's not about imposing anything on anyone the scaling up again there's that option of having either you maintain that staff student ratio and you scale up or you fundamentally change the pedagogic model and it's unrealistic to be able to scale up without having one of those two options and the misconceptions again workload is always a concern for staff but as long as the as long as it's understood what online learning is then that resource can be built into workload models and how we want this online learning programme to work if it's just going to be a case of putting all the resources online and leaving students to it then essentially you've got a another version of a MOOC and that's not what I think a lot of people want and you're going to eventually get the drop off rates that you might get on a MOOC as well so I think that needs to be articulated for the people who are in charge of resourcing these areas and these programmes to kind of articulate this is what an online programme looks like and this is the resource that's involved hopefully that's been useful that brings my presentation to a close and I'm more than happy to take questions Thank you so much Osmond I've spent most of the last 20 years working for the Open University and so much of that was familiar although I think you've got some really helpfully articulated comments both from staff and students and it really ties in with I think what Amber was saying in her keynote about we keep talking about this stuff because the practice is continually evolving So do we have any questions in the room? I'm looking hard at my former Open University colleagues who must have something to say about this No, they're all hiding now I mean this is an interesting one at the top that resonates with me from my experience why did they have to have it already at the start was that due to students having to be able to self-pace their studies? Yeah I guess so, I mean I think part of it was their conception of what online learning was so it was solved to them in a way where everything has to be ready beforehand and that kind of I guess was carried over from the way that they taught this programme to the space environment and where they already had all the lectures in place so yeah I mean there's no reason why it had to be ready week on week but I think that idea of consistency ties in with the fact that things are organised in a certain way so if you had certain weeks that were certain unit areas which did have all their content ready together then it was assumed that every single other area would have to do the same thing. Okay so what changes would you make to the pedagogic model and might students be able to cope with that? So as I mentioned we introduced a lot more kind of automated feedback but we also made use of the forums where discussion was happening so they had another personal professional development in which they were split up into their peer learning groups so we directly allowed the discussion there so although we made some of the communication a bit more didactic in terms of question and answer some of the other forums that were actually being used we kind of shifted some of the discussion there so I think it's dangerous to say well we can automate all feedback because I think that's not going to be possible and we don't want to do a programme like that but I think it's to make use of those areas where discussion was already happening to kind of introduce and say well this week's topic was on XYZ maybe this week we can have a separate discussion forum within your forum group to just highlight some of the issues or some of the things that we've taken away this week so I think that's one of the things that we did and the webinars again I think people love that that real life interaction so as much as it's possible to have that we had things like having intro videos again it sounds like a small thing but just having a face and a person saying hello welcome to the new unit this is what we're doing I think when you're online I think it helps a lot more bring a bit more of a human feel to it so there's another question kind of related to that did other synchronous models come up such as Skype chat or Google Hangout to encourage free conversation so we did we ran our webinars through Adobe Connect so there were the students had their own groups as well so they would be communicating with each other in social environments things like Google Hangouts where they would arrange their own meetings but apart from the webinars and the kind of programme committee meetings and things like that with student reps we didn't have any other kind of synchronous communication and I think that was probably more kind of interesting of the student body than anything else because we had people in different time zones and so I think it was very difficult to try to get everybody together synchronous way We've got a couple of questions about kind of peer-to-peer engagement so was peer assessment considered and are online students more resistant to peer feedback so in the dissertation unit we introduced an informal peer assessment peer assessment activity where you could look at each other's proposals and I went down really well I think some of the issues where they were probably that probably weren't particularly individual to your mind delivery was more to do with the students were comfortable with the fact that they were giving other people their access to their their assignments they weren't sure about how to assess it but equally we did get some good feedback about people being more comfortable with the assessment criteria in terms of their resistance I don't think they were any more resistant than I would have expected to do this Okay I think you mentioned that you were working in a particular discipline so is there a kind of template you had from that scalable to other areas? Yeah, I mean there are now a few online programmes in that faculty and this was kind of seen as the template in terms of how delivery was worked so the original model was really heavily based on these weekly formative activities and I think that was unrealistic given that resource so it was tweaked although there was some version of that it wasn't a direct discussion forum and a webinar every single week and I was tweaked but a version of that is now being rolled out and used on other online programmes in that faculty Okay, someone's asked about Jelly Salmon's E-tivities model was that relevant, helpful, used? Yeah absolutely that was I mean when I talked about that staggering of information that was one of the key drivers of that so really in that first week we just focused on getting them online whereas previously in the first run we kind of looked at this is all you need to know about assessments this is all you need to know about everything else reflecting on what Jelly Salmon does in her model just looking at the first week as well getting them online, getting them comfortable with engaging with us we're going to look at how to communicate this is the emails that you need to know and really making sure they're comfortable online and then moving over to the academic side of things and I think that's reflected in Jelly Salmon's model as well So it's come up at quite a few sessions over the last three days about getting senior management teams on board so someone has asked how were your key points moved by the leadership I think really well I think I had a really good relationship with them I think a lot of people probably have a similar role in terms of being a learning technologist and I think one of the things that is a good thing to have in your toolbox is that good will that you have with a lot of people because you have so many people so many things so when you are in a position where you articulate something that might be a bit of a difficult conversation in the right way and you're not kind of there to add any work to anything but you're just looking at things in the right perspective but yeah I think it's always a work in progress even if people do acknowledge it there's always a port that comes in at some point bought this, bought that but yeah I think and as I mentioned the program did increase and that part of that increase was because it was supported and it was kind of acknowledged that the usability of the workload that staff were putting into this needed to be addressed so that was acknowledged to some extent The last question has just come in I don't know if you have an answer to this are online tutors born or made? Well I'll have to say made because I think it's quite a blessing I'll have to say born yeah I mean I think I think online tutors are now no different to if you can hold a good teaching session I think it's a very few manner tweaks I mean that need to happen for you to do that in an online space so you know a good teacher will be able to facilitate a boom and facilitate discussions but when you do that online obviously there's some technical knowledge that needs to go into but essentially is the pedagogy which is leading it so I think the question probably could be are tutors born or are tutors born or made because I think the online side of things is a very small step once you acknowledge and understand what the online space is like I think that's a perfect place to finish so thank you again for your excellent talk and thanks again to Donna and Laurie so you've now got a half hour refreshment I'm John Wilson, I'm the CEO at Agenta we're a technology company that focuses on education and learning we build, manage and operate platforms for education for video collaboration externally we prefer to work with what we feel as ethical industries obviously education teaching, learning, healthcare we feel that we can really contribute to these industries by creating exciting platforms easy to use platforms secure platforms that people can utilise what we feel is one of the most important things for Scotland to boost economic growth is investing in rural areas by investing in broadband in these local areas we can attract more talent, we can attract more companies and we can drastically improve the delivery of education and learning within these schools, within disparate regions within Scotland