 Good afternoon depending on your sense of time. Welcome to week six, the enhancement and the evaluation of TEL. This is the final week of Octel and in a sense we've come full circle. We started by raising questions about what TEL can achieve and this week we try and wrap that up by saying what can it achieve and how do we know? How do we evaluate the initiatives that people have gotten involved in? We have three presenters for you. I'll shut up in about two minutes time and pass you over to Mark Kerrigan, then James McDowell and then Keith Smyth who will be talking about initiatives in TEL at different levels. One based on a particular technology, one based very much a module level and one based at institutional level. And the idea is that we've asked these three speakers to talk about their enhancement, about their evaluation and then we'll throw it open for discussion at the end of the session and then keep the discussion going through all the formats that we've been using on Octel so far. My name is Peter Hartley and my function is simply to introduce the session and then hopefully keep the discussion going at the end. Two fundamental questions, what can we achieve and how do we know that we've made a difference? And so those are the questions that we want to throw at our speakers after they've presented. And this was the brief, these are the speakers and as you see three levels, three different initiatives all of which have been recognized as successful across the sector. And this was their brief, they have 12 minutes at a time. What's the initiative? What was it intended to achieve? What has it achieved which is the enhancement angle, both planned and unplanned? And finally, how have you been able to evaluate it? And how do you intend to do that in future if that is still kind of in train, if that is still ongoing? So we'll ask our three speakers to take their 12 minutes airtime. We'd like to ask you to post your questions in the chat box and we will relate to those after the three 12 minute sessions. So without further ado, I will pass you on to our first speaker, who is Mark Kerrigan. Okay, this video is not working well. Hi, it's nice to meet you all and thank you for the invitation to talk to you today. What I'm talking about today is some work that happened at the University of Greenwich. I've currently left the university so just to let you know I'm cutting out based at Angularask in World Work. I'll be one of the faculties here. I'm going to turn the studio off, it's really not working very well. So the projects that we had at Greenwich really was the use of iPads in the curriculum. And it was really led by the provost chancellor for the faculty and what he wanted to do was to look at the way that laboratory teaching was delivered and how this actually relates to the professional environment that the students are going to be going to. And the idea was to align that practice happening in the university so that that's happening in the employability sector and supporting students transition from university into where they wish to be when they graduate. So it was quite a big project. It was one across the school and really what I want to do today is just talk to you a little bit about that as A to Z and talk to you about some of the things that we did, some exciting and unexpected things that occurred, the evaluation that's taking place and also a little bit about the future of the project as well. So I started off with this image here. This is just really helping to contextualize where the project really was. So this is an laboratory class and then the iPads are just there. So this is the environment the iPads are being taken into. And one of the questions that we always get is why iPads and I'll come along to that as opposed to different mobile technologies. Just as a reference, just to show the other people involved in the project, this is a big team project and a lot of people were involved. As I said, it had very strong buying from senior management and then it also involved a lot of other staff as well. And quite exciting as we're speaking to as well later on, we also involved students very heavily in this and had a student change agency working with the academic team to really help move this project forward. So thinking about the rationale for what was actually happening. So the PDC's vision of the show is to align practice with industry practice to support the students as they move to university. From a university point of view, a university like many universities in the sector is really focusing on digital literacy and digital competencies and skills. And the iPad is all digital technology in general is a very good vehicle for these sorts of activities. There's also a growing use of mobile devices by the students and think about where students are coming from and where they're going to also need to think about aligning their learning practice, their learning modes, what we're delivering within the university and that's another cool drive for this. Supporting student improvement is obviously there as well and very strongly linked to the student experience as well. What it is is deliver a very good student experience and the clever and strategic user technology aligned very carefully to professional practice and also managing the student expectations with a very good way of doing this. And as I say it was linked to the vision of the faculty too. So what do we actually do? So the project itself was actually designing to loan or level first year, first year the project is now in two years. The first year was learning all undergraduate full time first year students and like that. The project then moved into its second year so those first year students have graduated to level five and then all new level four students have an iPad as well. So there we have all undergraduate students in those two years is the aspiration for this. And now we have quite a few hundreds of these devices around. And as I say the first target really was really changing the way laboratory teaching is delivered. And thinking about the School of Science as laboratory because of multiple disciplines as well. So as Labarge teaching physiology, biology and also say for example chemistry and also exercise science. So it's a very large discipline focus which was very exciting from our point of view. Looking at how each different discipline responded to having a technology available to them and the sort of things that they could do. And that was something I'll come up a little bit later on. With a big goal of really re-engineering our teaching and learning to support our students. So an important aspect of the project for us really was recognizing what a success is going to look like for us. And the project had a sort of three year vision for it that in each year is associated with the different sets of activities. And really important for us to make sure that the technology was actually worked and engaged is not expecting immediate advanced transformation for everyone. Recognizing the fact that different people have different capabilities and that there's a longer game to play. So this is a diagram here it's published in that the case study we did for USISR as well. And it's looking at three different levels where an iPad was even used at a displacement level and then moving through enrichment and transformation. So it's these four three different levels of activity that we're able to see ideally sort of looking to the long-term look at the complete transformation of practice within the university. We're also happy to be interested in displacement as well and aligned to what people felt comfortable and able to achieve within the goals of the project. And that was an exceptionally important thing to teach when talking to staff and it helps to get people on board and build confidence around this. It was saying we recognize what we want to try and deliver. These are the goals of the project, how we work together to move on that journey. So another part of the project was really looking at the continuation of learning as well. So the project targeted sort of pre-activity, in-activity and post-activity. And then importantly again for the project was aligning it with the technologies can't be available. So the iPad itself was actually linked to the VLE. So students could do their work on the VLE for the iPad. Inactivity and post-activity recognized what students produced the laboratory reports written in Word. So what we're very conscious of was that the talent device had to fit within a learning pattern as well. And so this is sort of happening that people adopted. So they were getting creation of pre-activity, which could be completed in the talent, then inactivity within the laboratory. And then post-activity which supported the Walker-Afric construction of the laboratory reports, and then delivery of those as well. So another important thing for us as I said going back to something I mentioned earlier, really was around the concept of digital literacy. And whilst the iPad was very much used for thinking about changing the way laboratory teaching is delivered, it's also an excellent vehicle for helping support students develop their digital literacy. So the university at Greenwich, for example, is really focusing on developing this for both staff and students as part of a just project, the LNHE, Juliette and Tony developed a critical digital literacy model. And the iPad project is now beginning to think how it can align with this sort of work. So yes, it's enhancing the teaching laboratory and science in general now, but also the development of digital literacy. And the graph on the bottom is just showing how the learning of different social tools can actually help and link together so you're mapping digital skills through a curriculum, which is quite an interesting piece of work to move forward. So other things that we did, as I say, we had to develop a whole series of support for both staff and students. So the framework was put in place for staff, again, recognizing that different disciplines had different requirements, and also helped even give the confidence that there was actually a structure behind the project as well. So the rules and regulations for staff and students to engage with. For example, we ran quite a detailed project pilot. Apple was very helpful and loaned us some tablet devices to be able to do this as well. So we were able to test and see what was going to happen before we actually went forward and launched these things in September. Other considerations for us were insurance, loan agreements with students, which was, again, some of us had a lot of time constructing and managing students' expectations of who the device actually belongs to. One of the things we decided was the device was not going to be locked down by the university. It was a loan to the students, and the students could use that device as it was theirs within the university's rules and regulations for general IT. So in other words, there was a sense of it was the students how they wanted to learn, how they could customize it and engage it with and tie it down to them. Regarding apps as well, we did give them the first year an app toolkit to students. We wanted to provide a sort of level of confidence whereby these are sort of apps that you could use. So we recommended around about five apps that students had to have, and we provided a little of funding to purchase this and check that they were actually bought. And the idea around that was it helped staff build material with the confidence of a series of apps were available, and students, if they were less confident in using a digital technology, there were some apps there which they knew they could be used for their own studies. Then, as I say, we recruited some student changers to deliver these things as well. So these students worked for the project team to help us move things forward and engage with the student cohort. And I'm just going to go to the slide to change. And one of the things was on the conscience of time, they created an app rating website. So they took apps that were available, and they did a little review and rated it against various bits and pieces, encouraging other students to submit ratings as well, which was very exciting to see. And the idea was these students were acting as a catalyst within the student cohort. They created the app review site to engage in research and dissemination with us, too. And as I say, there were a whole series of rules and there to put in place, particularly the barteries, practice can be essentially dangerous clothes. So one of the things that we had to create and test was to make sure the iPads were safe to use in the lab, particularly the chemicals and biological hazards around as well. So all the iPads, for example, were contained within a plastic sleeve, and they were tested against various reactions to make sure they were safe and sound. So that was exceptionally important for us to make sure that we complied with the health and safety within the laboratory and also given the student's ability to use these things within the environment as well. So from our point of view, I think about evaluation. Evaluation is continuously ongoing for this piece of work. Our students were given a couple of questionnaires doing twice throughout the project, and it was good to see that they did enjoy using the iPad in the lab. And it was nice that they were taking pictures and thinking about recording data, photographic data in the laboratory is exceptionally useful when right and important. It was great to see that these sorts of things are being used and also beginning to take video as well. So it was nice to see thinking about how the iPad is being used. It's actually being used for the device itself and some of the hardware functions, which was good to see. And then, so we decided to change. It was nice to see these things did actually help support collaboration with peers and communication. So it really is a good vehicle for that. But more importantly, and finally, the final slide for me thinking about evaluation. One of the things I was quite interested in looking forward to exploring further, it was talking about increasing their engagement with the program. So students that have a strong relationship with the program, which was not actually a part of the original project brief, and also supporting transition into university. Again, this is not part of the original project, but students reporting these sorts of things helped there. And the final one, which I really did like was graph D here, which is after using iPad to my studies, I have started using the other aspects of my life. So from my point of view, that was an exception to what we've been doing in the curriculum and how these devices are being used to support teaching and learning in the laboratory is then radiating outwards and impacting other aspects of students life beyond the institutions. And thinking about my 12 minutes, that's where I'm just going to stop there. Thank you. Thanks. Thanks very much, Mark. We will proceed straight on and we'll, I'm collecting questions for the end. So if I now hand it over to James Vidal. Okay. Thank you, Peter. Just to give you a little bit of background on this particular set of slides, I inherited a module on a foundation degree in computer games technology, where overall, there had been very poor retention, low rates of progression, and generally a very sort of high level of disengagement. At the time, when I was sort of looking at this, one of the things that I identified very quickly was that there was a problem with documentation in this particular module, and that students were spending a lot of time doing the game development work, but not getting around to starting the documentation until it was too late. I was also aware that I've got very high numbers of students with dyslexia and or asperger syndrome. So at this point in time, I was looking to bring in formative feedback in some way, but also to sort of create the space for that formative feedback. So it involved really sort of moving away from assessment of learning and more towards for learning by creating this space. The first thing that happened then was to develop video-intensive learning, sort of video tutorials in order to create this extra time to spend on the formative feedback with students. This series, the series of tutorials were embedded in an e-portfolio system from Mahara, which is where I was also trying to sort of foster a community of learners kind of approach to what was going on there. Once I managed to get the tutorials in there, I then started looking a lot more at video- enhanced feedback, and in particular development of a system which drew students into a conversation around the feedback. I was trying to get conversations going using video from both sides, so that asynchronous video was sort of used in a loop, which I'll come to shortly. The third aspect was video-enounced assessment, where initially I tried to encourage learners to create their own video vignettes, as I've described them there, in order to get them to look at how they had used the portfolio system, how it's related to their work, in a sense to move towards documentation. I wasn't able to immediately implement the video drivers, which I mentioned there, but I did mention that in the subsequent year. So moving on, individually, the instructional tutorial videos. The development of these was very much underpinned by aspects of the competitive theory of multimedia learning. I've got a couple of quotes there for Mayor. Students learned from animation and animation, from animation and on-screen text. Again, that tied in with the dyslexia side of the class. And similarly, that's when words were written in spoken text rather than printed text. So this was talking to and explaining what was going on while demonstrating on the screen. And we do have a video, hopefully, which we can link into here. Hopefully that gives you an idea there. I've played it through in the background. I think there we've probably already gathered that wasn't my voice. One of the key things we're doing anything with video, and in particular audio, is that if you have a point where you've lost your voice or you've got a bad cold, it doesn't work too well. So there you've got the voice of one of my postgraduate researchers doing the narration for me. I'll move on to the next slide when we start looking at video feedback. Now the principle here was very much derived from the conversational framework. It was very, very much a dialogic approach that's been taken. The idea was that learners would submit some sort of evidence of work, whether this was a problem or something that they were trying to demonstrate to me. The system would then notify me directly to my phone or to email if I'm a secondary PC. I could then create my own feedback. That would then notify learners. They would get a very quick response to a problem. Now what I wanted to do was not be purely diagnostic and solving problems for students. I wanted them to think about what was going on. So I tend to ask a question at the end of each one. I'm going to jump straight to that diagnostic feedback link for you. If you have a look at that one, see that's about a minute. And then there's also a link to the formative feedback for those of you who want to go and see that. I'm aware that we're already into about five minutes, so I'll be back in about a minute or two. Yeah, that's a bit of a comment about static version. Then we've got, you know, motion sequence. But what we find in yours is currently something like that. So you basically plug in your spotlight time over the first time. So if that stays in there. Okay, hopefully that's given you sufficient time to at least let a flavor of those two. As you see at the end of the diagnostic feedback, the method I was to ask a question, ask for evidence of engagement with that feedback to see it come back around. With the formative feedback I was doing there, I was asking students feedback about two weeks before the final submission, really to highlight something particularly that might be improved on. Obviously there's also a name there to praise anyone for any particularly good work. The idea being that students were submitting their videos. I watched the video. I then give feedback and around we go in the conversation. Okay, moving on to the video announce assessment. Initially this started off as summative self-assessment as I'm saying there. I want the students to demonstrate to me what they've been doing in Mahara in order to link this in with the work that they developed. So it was very much sort of a to and fro between, well, here's how I was documenting using the blog to here is the development of the asset and then just going backwards and seeing the development of the work as it went on. This thing to consolidate learning that certainly made the narration much simpler and for students affected by dyslexia and asperger syndrome it did seem to make a difference. However, this was an area which I looked to change later on. There were some practical considerations which came up while I was looking at all of this. Distribution of video is always one of the key ones. Recording and storage is one aspect. Yes, you need to be able to be enabled to make these videos. However, how do you distribute them? My approach was to use Camtasia and I directly uploaded these files into Mahara and then they had sort of private feedback so students could only see their own feedback. Students themselves tended to go down this screencustommatic YouTube and embedding route which worked very well for them and also cost free for them. Some of the early benefits. Well, I mentioned that we have very poor retention progression to start with. The retention rates were absolutely turned around. We went from 15% completion to 85% retention in year one. That's made such a difference to quite a lot of students there. We also, if you want to look at the evaluation side of this, you can see that if students suddenly get in distinctions or merits from having previously only got passes, this is also impacting elsewhere in their studies. Certainly a lot of the evaluation that we got showed that students were taking this and using it elsewhere. Inclusivity was another aspect which was very key to this. Certainly for both Asperger's syndrome and dyslexia, students were doing much better in general than they had previously been doing and were certainly engaging and not dropping off the course. So that was making a difference there. I talked about the evolution of this. The idea was that rather than simply ask people to use text in the blog, which was still disadvantage in those with dyslexia, but we moved to weekly video blogs. So that each week students would make a video short for you, video usually, highlighting what they're doing in response to the tutorials, highlighting any problems. They would then get feedback from myself and then the conversation would continue week on week. So it cut out any issue of those who didn't want to raise a problem by video, perhaps missing out on that support. Two minutes to go, so I'm going to move on. So the integration of these activities from the first area you saw how we got the feedback loop. What's gone from there is that the instruction tutorial videos each week have been inserted into this loop. Again, I'm getting feedback requests there. Students themselves are also showing me through video assessment what they're doing. I can then give them more feedback, point them at extra tutorials, and away we go in that conversational framework again. Okay, moving on. The impact on the student experience. As you can see from this qualitative feedback, there were some very, very positive comments. The usability of the feedback was key. The timeliness of it was also something that was there. Controls mentioned in terms of being able to use the video tutorials. There were a lot of different areas there. Interestingly, there was one student who initially objected to use some tutorial videos, citing the cost of fees as a reason. But once we explained why we were doing this and how it enabled me to spend more time on feedback, that student also joined in and hopefully got a little benefit from it. We're into our final minutes, so I'm going to go on to some of the findings. Certainly, there is transferability of this approach. I've also used this and I've asked colleagues around the institution of how this could be used in other scenarios, in particular with written work, a video of how something is marked internally, a grade mark has worked very well, off-screen assessments. In a sense, a little bit like we just saw there with the iPad's sort of a Greenwich. Way distribution is an issue for mobile. There are apps that can help you there. However, it works best, as I've found in high-frequency, dialogic formative feed-forward. Once again, it's moving away from assessment of learning. Inclusivity, incredible results there. We've had a lot of students who've gone through and done exceptionally well. It's also worked out very well for internal moderation and external examination, which would lead me on to my any questions, but Peter, I think you're probably going to need to hand over. Thank you very much, James. That was example number two. What we're doing for folks who've just joined the session is we have our three speakers who are talking about interventions at different levels. First, we have Mark Kerrigan talking about iPads in science based on work at Greenwich. That was James McDowell from the University of Huddersfield talking about how he's turned around a particular module. We've had an initiative based on a device. We've had an initiative based on a module. We'll now turn to an institutional enhancement initiative. Hand over to Keith Smyth. I'll be keeping a note of the questions that you raised in the chat box and we'll come back to them later. Thank you, James. Over to Keith. Thank you, Peter. Thanks very much. Thanks for the invite to take part in today's webinar. The third example is an example of an institutional initiative. What I'd like to talk about is our implementation of the three framework at Edmmer Napier University. So what is the three framework? To give you a bit of background, the three framework originally developed within the context of a cross-institutional technology enhanced learning and staff development project where Edmmer Napier worked with two partner FE colleges across a range of levels and discipline areas from, you know, January apprenticeships through to health related masters to try and help practitioners across the three institutions find ways to use technology to support learning and teaching that have the focus on students being actively engaged as possible as curators and creators of their own educational experiences and which also wherever possible had an influence, sorry, had a big element of collaborative learning involved. The framework in its original form was really there to guide the decisions about making use of technology and learning and teaching. And in 2011 when Edmmer Napier decided to introduce a new institutional wide strategy for use of technology and learning and teaching, we returned to the three framework and then introduced it as the new benchmark for use of technology across the institution. So just to give you a very brief flavor of what the framework's about, it's a framework that's based on an enhanced extent and power continuum. We're at the enhanced level, we're talking about adopting technology in relatively simple ways that will get students to actively engage in various learning activities through to extend where we're talking about further uses of technology that gives students both individually and collaboratively at least some key decisions to make about the tools we use, the types of activities we engage in, what they'll produce as a result of the learning. And then at the empower level, we're talking about uses of technology to underpin I guess higher order individual and collaborative learning that reflects as closely as possible how knowledge is created and used in the disciplines that the students are preparing for and also how knowledge is created and used in the real world that we all can live in, hopefully most of the time. The three framework is a framework based around learning and teaching activities and there are categories for about 20 different learning and teaching activities with examples of each of the three levels brought up to correctivity. So the one that's on the screen at the moment is around encouraging early engagement in key concepts. The example at the enhanced level is having students take turns to define one or two key terms or concepts each week to then include in an online class glossary with the idea being that those taking the turns are defining concepts that might be coming up in that week's lecture or seminar. Through to at the empower level the equivalent type of activity being the use of online resources such as collaborative spaces, links to video clips and so on that students use in case and problem-based learning tasks. So the three frameworks based on learning and teaching activities so that people can think about how to use technology based on decisions about the types of learning and teaching activity they're engaged in. So there are various examples in the framework that relate to things like use of technology in lectures, engaging students in field work, engaging undergraduates and post-grads in research like activity and we introduced this as a means for staff to think about what technology might mean for them in their context. An important point about the three frameworks is that they continue themselves. The three categories aren't mutually exclusive. You could have a course or module that combines activities at the various levels but as a general rule we provide guidance to our staff that suggest that you'd probably do more things at an enhanced level with a group of new first-year undergraduates whereas if it sounds of you know honors level students, master students you almost certainly want to be doing things at the empower level. That's nothing to do with use of technology, that's just to do with the level of the learning and engagement. So we introduced this in 2011 as a new framework for the use of technology and the expectation that was set was that every module in the university would make some use of technology at at least the enhanced level and our thinking there was that technology can be used in a meaningful way in any discipline area to add to the quality of the learning and teaching experience. As well as presenting examples of things that enhance, extend and empower level for various types of learning and teaching activity, we developed a framework in collaboration with colleagues across university so that we could provide mappings within the framework that relate to different discipline areas. So we see on the screen examples from nurse-related provision, evidence-based practice, substance misuse and the whole idea here is that we can give illustrative examples relating to different types of learning and teaching activity but through the module mappings, colleagues can see how that might look in their own discipline area. We've also used the three framework in curriculum design and our staff that undertake our own masters in blended and online education experience the framework and practice because we've used it to structure progression from the first module at the PG3rd stage of that program through to the third module. So we've used it at that program level design context as well as within individual modules activities. How did we implement this? I think this is key to how we how you might evaluate anything. We aligned it with the move to new institutional VLE in 2012, Edmund Napier moved from WebCP to Mudo. So we introduced the framework in tandem to get people to think about the decisions we wanted to make around using technology in their modules as we migrated them from one VLE to another. We also embedded the framework within various professional development events. It was the theme for our 2012 staff conference and also for that year's learning and teaching awards and those who put their practice forward for one of the learning and teaching awards had to exemplify what we were doing at either the enhanced extent or power level of the frameworks. So it was really important for us to bring the framework as a sort of way of thinking about learning and teaching very close to what people were doing in practice and use the framework to exemplify and inform that practice. We formed some teaching fellows special interest groups around technology and enhanced learning and we also opened up a range of sponsored places on our own masters in online education so that staff could engage with the framework as a design tool and experience it and make informed decisions about how they could use technology in their own practice. What has it achieved? Well it's been two and a half years since we introduced it, we introduced it at the end of 2011. We have seen over time an impact on practice across many areas and schools and faculties. We've seen lots of things emerge at the enhanced level. Activities like the one I described where you get a new group of learners and get them to take turns each week to define key concepts that are coming up and contribute to an online glossary. That's proved very effective. Similarly use of shared kind of bookmarking resources. We've seen an increase in extend level activities so a greater number of modules over time featuring things like student led seminars where students define an online seminar activity. They provide the resources throughout, they provide the context and the questions and lead their colleagues on their modular program throughout a pure led seminar activity. We've seen lots of activities emerging at the power level like using online resources to engage learners in the professional communities that they're ultimately going to be preparing to join. One of the added benefits of introducing the 3D framework for Edmund Napier was through the fact that we introduced it as a creative commons tool and since we published it in late 2011. I think there's now about 30 institutions across and beyond the UK that have used it in their own institutional strategies or their own institutional staff development or their own approaches to how they are evaluating tail uptake. I might come back and say a bit more about that. How have we evaluated it? Well as I mentioned I don't think you can really separate this from the implementation activities. So as I said we aligned it with the move to new institutional VLE so you could see in practice the decisions that people were making about the things we do in Moodle based on the guidance in the 3D framework. We brought it into the theme, it was the theme for a 2012 staff conference whereby staff were encouraged to present papers and sessions on how the 3D framework was informing their own tail practice and that brought forth a range of case studies from the usual suspects in the university but also the people that were perhaps doing really good work under the radar and not necessarily shouting about it. In terms of how our staff are responding to date then we have had over 90 module mappings submitted for potential inclusion in the online version 3 framework from our staff and over 30 case studies in our learning teaching and assessment resource bank. We've also been able to see uptake of the framework through coursework that staff on a PG start learning and teaching are undertaking and through teaching fellowship applications which are increasingly showing applicants own engagement with the 3D framework as part of their own practice. However, and as we move forward to concluding with the next minute or two, one of the things we're really interested in is what the University of York have done with the framework. They're one of the institutions that's adapted it and they've used the framework itself as an evaluation tool to get staff to articulate what they're doing with institutional VLE and other educational technologies and at this moment in time we're looking towards what York have done which is presented on the screen here and how we can put that into our own future plans for evaluating the uptake of the framework and its impact within Edmund Napier. Couple of points I'm going forward. Edmund Napier has just moved from a module to a program focus approach to curriculum design and development. The 3D framework is being embedded in that guidance. It's being used in the current redesign of our own PG start learning and teaching in higher education so we can use that program as an incubator of good practice and technology enhanced learning and it's also been key to several recommendations of a recently concluded digital futures consultation we've undertaken and that's bringing forth a number of recommendations relating to how we further embed the framework and how we evaluate its uptake. My final point we have been very fortunate that a number of the institutions that have implemented the framework themselves have come back and shared with us what they're doing so we're currently in the early stages of planning a cross institutional evaluation of how the framework's been implemented and adapted. And that will include of course an evaluation of what we're doing and what we'll do with it into the future at Edmund Napier and I think that's my time up. So thank you Peter. Thank you very much Keith. If folks would now like to start chipping in questions I will start kicking off with some things that have emerged in the chat box as we've done along with a few who would like to add in additional ones and if you've got Mike you can always raise your hand and we'll come to you but I will if you put stuff in the chat box then we can deal with those. As a starter I've left that slide that last slide of Keith's on because I'd like to throw a quick question at Keith on that final point this cross institutional evaluation I know you're in the early stages of that but how do you think you'll be doing it? Yeah that's a very very relevant question and the honest answer is we're working through that at the moment but we have seen for example as we just discussed the way that York have used the framework as an evaluation tool rather than the tool to guide thinking and so we are probably going to have a strand of work that is around replicates what York have done as a way of looking at uptake of the various kinds of activities that are sort of referenced or recommended in the framework. We are also though and I don't want to take us off of too much of a tangent a strand of this work will also be focused on what makes something like the three framework adaptable and usable as an open educational resource and part of the research that we'll undertake we'll be looking at the decisions people made around why it was appropriate for their institution and what made it so adaptable across really quite a broad range of use rates from tell strategy through to staff development through to quality frameworks. If you'd asked me that question maybe another couple of months down the line I could give you a much more detailed answer but that's kind of where we're at at the moment. Okay thanks Keith while you've got your mic open could you want could you respond to that question that's just come up on the chat box from Peter does the 3e framework encourage a tech driven approach to pedagogy? Sure we certainly hope that it doesn't Peter because the framework is organized around learning and teaching activities from lectures through to field work to undergraduate research and it's also the framework is also wherever possible platform independent so it doesn't talk about tools in webcp or middle it talks with things you can do with technologies and our experience to date and this is some of this is anecdotal but what we hear from other institutions who are working with it is that it's the learning and teaching focus making decisions about learning and teaching to inform how we use technology this appeals to them and that's one of the big factors and then making use of it and adapting it for their own purposes. I just I just pushed the slide back to our three speakers slide just to remind you of the three initiatives and our three speakers. We had a question early on which I'd like to direct at Mark about how you manage the loan and the maintenance of iPads which is obvious an issue for anyone who's considering putting devices in. Mark do you want to say a bit about that? Yes sure happy to. Regarding the loan the first thing that we have to is make sure it's very obvious that the iPads did remain the property of the university and that was exceptionally clear in through conversation and discussion with it that being said we also told the students you can use it as you want so the exceptionally transparent in ownership is very important for us and then regarding the loan itself looking at university insurances depending on the threshold level for a claim that that's an interesting one to its law but the students were given basically a document of expectations and an agreement to sign that should they lose it or damage it there there's a cost for replacement there which was then signed and then goes on the student record which was stored in the faculty so for us it was really making sure that ownership was very obvious, usability was defined as well and students had to agree and sign something around there. One person did or I think one person had to split a police report regarding something that they lost something and then we had a separate process within the faculty to help support students around there as well so stock devices in case students did lose theirs or break theirs and that's roughly how we managed the loaning. Okay while you're on mic so to speak can you afford to continue with this initiative given the cost of iPads? In an earlier comment on the chat box I referred to a talk from Simon Thompson at Leeds Met he gave a very interesting keynote at the Solstice conference last week where they decided to go with the Google Nexus and one of the issues there was cost. I mean that is an excellent question and obviously I don't work at Greenwich annual so I really can't speak to their future plans. I think from our point of view looking at loading the device and then how that fits into the bigger agenda of sort of bring your own device bring your own service it's an interesting question and to be honest I really couldn't speak to that. What I'm hoping to be able to see is that over time the universities develop the confidence and understanding to support multiple devices across multiple platforms so the underlying literacy there therefore the need for large-scale purchasing may not necessarily be there although it counts rather less its inclusivity and also a pattern of experience and how to reconcile all those different arguments and considerations is a very big discussion for debate within each institution. Sorry I can't offer any more on that one. Could you just comment to respond to that comment from Marcus in South Africa have staff were they loaned iPads or given iPads as well? Staff were loaned as well from our point of view it is exceptionally important to develop staff buy-in so we said any member of staff who's involved in level 4 teaching so going back to the beginning of the project would also have a device as well and they were given all of that device so they could then give up their own teaching and ideas around that so yes staff will also learn the device within the same product. Okay we've had a couple of questions queries about the app review site that you mentioned can you tell us a bit more about how that was run and how successful that was? Yeah this is something we didn't ask for so we had a group of student change agents and it's something that they actually created they created the process themselves the reviews themselves so from our point of view that was exceptionally exciting to be able to do this in research and it's just in the case of about students confidence from moving from apps recommended to them to actually go and discover their own. I think at the end of the project this all had about 25% of students who were still only using the original tool kit so the idea was anything that could support students and engaging in greater apps was great for us so it was student created and student run so actually about themselves the change agents I think did mostly initial reviews and then the idea was then to continue opening and encourage any student or member of staff to submit reviews and which links to the some of the dates that we got whereby staff and students are engaging in recommending apps to each other so we really wanted to encourage that debate between between staff and students and recognizing that it's a partnership around app discovery and app engagement not sort of a didactic approach so you must find and use this once so that that's where it came from as continuing use I'm hoping it's continuing to be used but but again I couldn't really speak to that last part. Okay finally Mark could you just comment on that use of iPads for image capture can you can you do that with those trans those transparent bags in place? You can indeed they're designed to have a camera in them and so we are seeing students do that. Some students if they need exceptionally controlled photographs with permission may be allowed to move the camera outside of it but no generally speaking you can take a decent up image from somewhere in the bag without a problem at all. Regarding image capture we also did have the exception to clear on the rules around image capture because students as long were told very clearly that they could not take pictures or video of other staff or students within the laboratory without their permission and the consequences of doing so were actually quite yeah and severe again going to big concepts of digital literacy and sort of reasonably surviving responsibility. Thank you Mark. A couple of questions for James I'm sorry that slide should say James not Jim McGill that's a typo on my on my behalf so James I wondered you did mention that you thought about translating this approach to other subject areas and I wonder if you'd had any experience of that or whether your institution has picked it up or whether you could just comment in more general terms about that. Absolutely I run an institution my project where we looked at how to adapt video and house assessment feedback in each of the seven schools. One of the disciplines was the in art design and there's a course there on textiles and fabrics fashion that sort of area where students were required to make a jacket within a studio environment. There the students were able to use mobile device to sort of show their work off and comment and similarly the academic was able to walk around with an iPad and point out exactly where there might have been a problem with stitching or how this would relate to some particular theory connection with fashion and that really ties in very much also with the use of iPads and mobile devices. Certainly in applied sciences the case there was of students being asked to basically write an academic paper around a predefined data set and there were requirements for there to be a specific kind of an abstract and introduction to present data in a particular way and this was then marked within grade mark with a rubric being dragged onto the screen. A colleague who did the work there had also trialled audio feedback alone and said he found that students engaged better where there was video. I think that was a quote was he the video gave the students something to hang the audio on and it really sort of tied it together. Certainly I found that when a visual demonstration of the similarity index in Turner's team was demonstrated and students saw here are the eight places where this work has been misreferenced or plagiarized from that did seem to make quite a difference to the way that they went about in the future. There were other examples around the institution I mean that's too few straight away. I don't know if you'd want to know more there was also some work we went on in the business school where international students made their own videos as if they were giving the presentation. Now some of them did face to camera or face to webcam work. Others chose to use screen casting others chose to borrow shall we say bits of cartoons from things like Family Guy and then put their own audio over and got quite creative with it. It was very entertaining although perhaps not entirely strictly within the laws of copyright. Yeah perhaps we've lost quickly over that one I think depending on who is listening. One final question for you James I think it's probably our final question for the afternoon. You mentioned the particular advantages of the period for students with Asperger's and students with dyslexia and I wonder if you'd say a bit more about what you think created that impact. Certainly the frequency of dyslexia on the particular course was very high. It ran at a minimum of 25 percent and went much higher than that in certain years. As I mentioned earlier on one of the issues was to do with documentation in the course and that was kind of at the root of why students were doing so badly on a core module which was effectively worth 40 to 120 credits per year. 30 percent of the module marks were available for documentation and yet students didn't write anything. Now this wasn't simply dyslexic students but it did seem to mean that the students with dyslexia appeared much further down so the ranking within the class than they did after we had allowed them to go on to document their work using video. Similarly when it came to students with Asperger's syndrome the opportunity to give a presentation without having to stand up in front of a class for peers meant that there was immediate engagement with this. I found that I think it was something like 36 percent of the videos made within the class of 25 students. 36 percent were made by students with a learning difficulty and that's against the background of say 25 percent of the students affected in that way. So there was evidence there that this was something which was appreciated by the students and they engaged with it very quickly. In terms of enabling the conversation for students with Asperger's to be able to converse and get into dialogue around feedback the asynchronous pause that goes with trying to make a video but not doing it safe through Skype. These I think really were advantageous because there's the opportunity to back away to think to reflect on what's going on and occasions to actually reflect on their own thinking processes. There were some very interesting videos where students began to solve the problem as they were making the video to demonstrate it and it was a great one for a monster of Asperger's who then started to self-analyze as to why he'd been thinking, the way he had, how he'd needed to change what he was looking at. In fact it was a metacognitive stop alert which was beginning to come through which was great to see. Okay thank you very much James. Well on that note of students making really effective use of this technology and enhancing their learning I think we shall draw this session to a close. You'll see from the chat room that Caroline has revealed that the code work for this week is Santa. So I'd like to thank our students presenters very much particularly for keeping so wonderfully to time and given the code word I wish you all a Merry Christmas and we look forward to seeing you online over the next week or so. Thank you very much and goodbye.