 I'm not saying much. It's going to welcome each speaker onto the stage and let them speak. We have five really interesting sessions today, so we'll keep the time to question somewhat limited, but hopefully we'll be time at lunchtime if we want to follow up further. So for the first speaker, I'd like to welcome Theresa, who will let you introduce your topic. Thank you very much. Thank you, and thank you all for coming along. My name is Theresa McKinnon, Principal Teaching Fellow here, so I haven't had to come very far. And what I'm going to talk about is a project that has recently finished, and I'm just going to welcome up on stage as well here. This isn't daunting enough that you have to come up on stage, but two of my colleagues from the library. So can I ask you to introduce yourselves reasonably near to the mic? I'm Kate Courage, and I'm the Academic Support Librarian for the School of Modern Languages and a couple of other departments as well. My name is Tracy Dicks, and I work in a teaching and learning team at the library supporting the Academic Support Librarians with embedding digital literacies and information literacies within academic departments. Thank you guys, and you'll be hearing from Kate and Tracy, who have been very helpful supporters of the project and people involved in the project. My main focus really today is going to be on the takeaways. The support of this experience of a project, and I've project managed lots of different projects in the past, might be useful for you to take away and might be useful for you to try out. So you will see more depth and scattered around randomly on the ends of the rows here. You'll get a little bit more insight into the details of the project. If you scan the QR code on either the postcard or the leaflet that's out at the end of the row. Please do feel free to move along, grab yourself one of those. If the remnants from last night are anything to go by and you're now feeling actually I'm not really bothered about listening to people and watching screens, you can at least browse the outputs using the QR code and see what we produced as part of this project. The School of Modern Languages and Cultures is a recently formed school. I'm a language teacher. I work within the language centre and we're now part of the School of Modern Languages. So we have different and diverse histories within that school and what I wanted to do was to make sure that we had a way of surfacing some of the things that went out and involving students and other stakeholders in which sort of things now we needed to take forward in terms of teaching and technology-enhanced learning. So the name of the project was digital learning and teaching languages in the digital age. So I'm going to tell you a little bit about how we went about producing those takeaways that you can browse. My role within the project apart from project management is also a background as a seamalt of somebody who wants to embed digital literacies within the experience of language learning. So I was looking for ways of discrete contextual skill development for all the participants. Now I think probably most of you are familiar with this. Whenever you embark on a project, it's worth doing a little bit of a SWAT analysis and we sit down and or I sit down and do that and think very carefully before we actually embark on the project around what sorts of things are going to come up and this is what came in terms of qualifying these things in terms of no knowns. No knowns really were the resource. I knew we wouldn't have the resource in terms of time or budget that we needed to achieve the goals that myself and the staff team had decided that we needed to achieve. So that's a case of sort of okay are we going to live with that and we decided that it was worth doing and therefore we were going to live with that. The project itself was initially written as two separate projects. One focusing on learning and one focusing on teaching. We were approached by those evaluating the project who basically said well we like the ideas, we can only give you half of the budget, how much are you going to deliver and we decided as a team we wanted to deliver it all so we rewrote the project in a way that would hopefully enable us to do that. The supporters of the project are we here. The Warwick International Higher Education Academy this is a recently formed body. Very much part of the unknown knowns in as much as the body of we're here and I'm a founder fellow of the organisation is very recent in Warwick. So there was very little awareness of its existence as you've seen if you walk around campus this is a huge campus. Lots and lots of diverse activities going on. So in terms of unknown knowns we didn't know what impact that was going to have on the outcomes of the project as to whether people were aware of we're here. Hopefully we've done a dissemination job today by putting that literature out and do feel free to share it and pass it around. And there is some auto tweets going on from Warwick language which will send you some links as well if you're interested in finding out more about we're here. We're here is all about increasing the focus within this very research focused institution on teaching and learning. So it's the ideal space to support the sort of activity we're going to do. In terms of the unknown unknowns well I was going to be working with a team most of whom I'd never met before. The timing was going to be problematic because actually the timing of the project was going to be covering exam periods so how much student engagement we would get during that period we knew was going to be difficult. There was going to be a great reliance on remote working in order to achieve the outcomes of the project. So let's just show you the individuals involved. It's a very small team. Thanks to we're here we were able to offer these students a small amount of money. Say a small amount actually for these students it was above minimum wage. So sort of ten pounds an hour for participating and producing to contribute to the project. And I don't think without that we would have had the commitment and the involvement that we had. But in order to achieve that in order to be paid they did have to go through some hurdles. They had to register with unit temps they had to find their way around an online system in order to claim their funding. So there were quite a lot of barriers in there for them to overcome and of course that involved a lot of to one support as well to help that. So in fact we ended up with just one launch session one opportunity to meet face to face as a team. So across this team you can see students, some of whom are SMLC students some are from sociology you can see Kate and Tracy up there you can also see myself and then you've got some students from sociology some students who had heard about the project and were interested and wanted to get involved. But I knew that as we were going to be doing a lot of this online the immediacy of communication was going to be really important so the models in my head were very much around Jilly Salmon and how we actually engaged people and bought enough confidence from this small group of people and a meeting that was really very short. Let me just pass you to Kate. Thank you. As a librarian Tracy and I were really delighted to be invited to join this project I think because of this morning's keynote I don't have to do too much of a hard sell on what the library can do in terms of working with academics so thank you Jane for that. But our main mission particularly in our team is to support teaching and learning and encourage students in their ability to find and use information and this is just a more innovative way to do it but one of the many ways that we try and work on that. Tracy and I on behalf of the library were able to step in at key moments in the project and collate resources that we already had in most cases that would help the students proceed with their research. At some points this was getting started with copyright bringing in creative commons at other points it was bringing together resources on research methodology putting them online in the space and at the time when the students needed them. We also joined the team and offered support face to face and online with students as they navigated the resources and found the information that they needed to create their own outputs and so in some ways it seems separate from the library's core mission which is often seen as being about books and reading materials but it's completely core as I say to what we do in trying to support students supporting their research and their learning so we really enjoyed the opportunity to work with Theresa's team and particularly working with these student researchers I'll pass over to Tracy So I'm coming at this from more of a digital literacy kind of perspective and we learnt a lot about the students from this project and I think part of the reason it was so effective was because Theresa worked really hard to create this level playing field between the staff and the students so our first meet up was in a very warm room in the school of modern languages and the staff and students were all sitting together we were all engaging in David White's visitors and residents activity that was produced by the Disruptive Media Learning Lab at Coventry and we talked over coffee at the Humanities Cafe I mean it was so effective that I didn't even realise till later that I had mistaken one of the academics for a student I just thought he was really outspoken but I think this went a long way in showing students how much their perspectives were valued on the project and more so than staff in many respects because we wanted to learn about their experiences with digital literacies this also partly took place over the exam period so when students and academics were preoccupied with assessments and students' output started trickling through Kate and I were able to support the project by interjecting on the discussion forums and just making the students feel that their thoughts and their ideas were valued and I think that helped to keep the momentum going it certainly did and it was really really important that the sort of balancing of roles was established and that the students felt that they were co-researchers on this project it was very important for me as well that I needed to step back I was facilitating and not expecting particular results from their outputs so we used these three spaces a Moodle course which actually is within our Moodle that's optimised for interaction so we had the Blackboard collaborators for weekly meetings online and drop-in points and they were used by those who were interested in doing such things we also had the Mahara which is a private space and actually encouraging people to use Mahara to pull their research together and reassuring them that it was private until they decided to share was really important so we were hoping through that to make them think about privacy and public space and what they can do in different spaces and also to move them through that whole permissions aspect of contributing it was very important that I didn't influence that other than taking their ideas on board so the project design for this I drew on a way of describing learning in the 21st century Hurtagogy can I take a quick pan round the room Hurtagogy yes Hurtagogy if you google it there's a community of practice out there it's a way of describing now in this information age how we can much more easily navigate and find resources and how actually to optimise the learning as a result of that so the students are encouraged to explore and jump around and create stuff, collaborate connect but also reflect and reflection is the vital part of this process and once you've created something that moment where you then decide to share it is probably the moment when you reflect the most so before you send me that email and say ok you can add my pages to the group you're probably thinking am I really happy with it and I think it was actually fundamental in terms of the quality of the outputs that they produced as a design Hayes and Kenyon 2001 started to look at this and Steve Wheeler whose opinions I respect greatly describes as the form of flannerism which being a French language teacher I'm particularly drawn to a flanner is somebody who saunters around observing sounds a great job doesn't it I'd like to be a flanner, please employ me as a flanner what we were encouraging people to do was to saunter around digitally and observe what they saw and think about it and share their thoughts with us so finally I think the crucial outputs for us and outcomes were the time and as we know time is money and we didn't have enough of that so we just had to stretch ourselves we were the elastic resource but the time to reflect and the time to question the ability to connect these students as well to thinking about digital capabilities so clearly JISC's framework for that was helpful but also Belshaw's essential elements I think there's some great work there that we're able to draw on chickering in Gamsen as well principles for undergraduate education you'll see all of these links being tweeted on my account so if there's something you want to investigate I've done quite a lot of work in those sort of areas it was interesting to me that none of these students who are all learning languages in some context or other in fact the only students out of the entire group who would thought about using social media for their language study were the ones who were studying with the language centre so they're not specialist linguists that's the space where we do tend to encourage them to think about how useful it is to follow French tweets and how much possibility there is for their language learning there so those were key outcomes there's obviously more details on the Padlet Board and we'll see you as well through that QR code the where next so it would have been great to have a follow up and we will try and have a follow up meeting now face to face with them and to give them some feedback on the overall outputs and maybe move them further forward there are this was the little pack of cards that Tracy referred to these are available under creative commons licence on the DMLL website and translatable it's a great little game inventing a story we did as a team game and it really did get us talking and we're linguists so we like to talk so we had much more close collaboration as a result of this and I got to know the individuals in great detail we used an autoethnographic approach to get them to reflect on their thinking about their relationship with technology and that surfaced some wonderful things so that was really useful we're hoping to embed and spread the process into the institutional digital skills plans and I think Tracy might have something to tell us on that very quickly yes so in terms of where next in terms of digital literacy at Warwick a couple of us from the teaching and learning team at the library are working with the school of cross faculty studies on a certificate of digital literacy and I know that Amber's academic technology team is also developing a digital capabilities course so there are these ripples of activity and we've also borrowed some of Theresa's activities and ideas in planning these outputs and hopefully with knowledge sharing and these projects that are running will be able to take it to a more institution wide process at Warwick thanks very much thank you for that stay there in case there are any questions and we are going to have I think Martin is going to run around with the microphone because we are live streaming this any questions see one in the back oh Martin somebody right here actually a couple of very quick questions one that you did this like a very good project how does your institution like how do they see it and did you get any sort of recognition or formal informal and next question is kind of linked to it that are you hopeful that you will be able to expand it in future like you did mention but what are the chances I think well the chances I'm not a betting person chances I'm going to leave out of the question it's very much it was very much an opportunity to see what we can do and using the knowledge that I have from computer mediated communication in language learning how that could be applied as well to digital skills embedding more widely I think traces conversations with skills folk may help us you know how much these sorts of models could be scaled I think there's it's a delicate thing because it's very much about individuals and personalities and group dynamics but I think it could be done but big question mark there for me really in terms of in terms of how the institution recognised and valued it that was through we're here through the Warwick International Higher Education Academy thanks Hi there, Justin Steel, David's University of Derby I might have missed something right at the beginning with this for credit or was this sort of an extra support project okay so it's an extra support project this wasn't from credit at all but it was for money so there was payment for students who contributed to the outputs but it was entirely driven by their choices so we set up a framework the hercogic approach really essentially sets up a framework for people then to choose their own way through and engage in the learning that they wish to do okay so it's almost like an extra credit type activity but they were funded to do it that's where you categorise it I think that's all the questions we've got time for but thank you very much thank you and just moving on to our second speaker I'd like to welcome Sonja Gwsendorf from London School of Economics and I'll let you just go and get started so she's already said it but I've written it down so I'm going to say it again I'm Sonja Gwsendorf and I'm a senior learning technologist at the London School of Economics and I want to begin with our conference theme so themes are useful frameworks they give structure and purpose order, form and substance they're constrained against which our thinking can grow and prosper like a calamitous against a garden wall themes are frameworks and they work by keeping things in as well as keeping things out which means we can ask why are we connected concepts, connect, collaborate and create I would add the notion of care for a start but not because it has been excluded but rather because it is implicitly already there and it deserves to be made explicit care is the name for the realm within which connections, collaboration and creativity can unfold in the first place in care we can more properly examine the question why for example why at this year's conference we care about connections, collaborations and creativity why I care about the visual and why anyone cares about anything at all so today I take care of my work through the lens of connect, collaborate and create so this is what I promised in my abstract that I would reflect on a rather loosely defined design project this is what I promised and I intend to keep if not exactly to the letter this promise or rather not to all the letters perhaps only these ones that I will use my project at the springboard to consider the visual that I will wonder why and how aesthetics, the beautiful and good design matter to us and consider how if at all it matters that learning technologies bring their attention, their care to the visual also so clearly the why is more meaningful than the what and the how why does good design matter equally clearly without content the what and the how the why would simply sit there annoying and intrusive like a child that wants answers but doesn't tell us what it wants the answers to be about so I hope you will appreciate that first I will do some mundane presentation context also known as this is what I did and then I will try to keep this as short as possible so in March 2015 Peter Bryant the head of our department came to me and said I want you to lead on the design project that's what he did get some graphic designers to make us some visuals choose some filmmakers to work with and celebrate our e-learning innovators put together a pool of young student artists so I was to create a creative hub that the LSE academics could draw on for all aspects of their teaching and note that the teaching dimension is here really essential of course the LSE like most universities has their own design unit but their remit is not pedagogical and that was virtually all he actually said to me and there's nothing more brilliant than be given a remit that is entirely vague because it means you can fill it with meaning as you see fit so I was very lucky in any case I did understand the remit to be meaningful the intention was clearly to elevate the visual to recognize its rhetorical power its ubiquitous presence and cultural social importance in short to bring it back to its proper place which it had somehow lost at the overtly overly textual LSE so as part of this we created a series of video celebrations and articles of academics who properly innovate their teaching practice with us this was a small nod to understanding that although research teaching and administrative burdens on academic staff have increased exponentially over the last 30 years recognition and reward are given exclusively to research output you'll all be familiar with that studies have shown that a thank you does a lot to increase academic job satisfaction so with the LSE innovator series we say thank you to those who care about teaching but more importantly this focus on visual creativity that I was given allowed me to reexamine the use of our VLE Moodle here was a chance to invite academics to reexamine their relationship to using Moodle in their teaching or if they ever had one a chance to persuade them to view it as a space at which learning can happen as opposed to a space into which we shovel the text badly arranged obtrusively placed to constantly shout all learning is reading so we opened a call for academics to ignite my Moodle and unsurprisingly almost no one answered unsurprisingly because Moodle is boring and its boringness is entrenched certainly at the LSE I find it boring when I see our LSE Moodle I want to die I blame this on Moodle having been misunderstood from the outset it has systematically abused over the years it has become an administrative tool at worst and a glorified overspect Dropbox Plus at best some blame Moodle itself pointing to it being cluttered by default I blame me for not having tried harder to make its potential understood now my excuse is that when I look at our Moodle I want to die through the design project I have rediscovered my will to live so luckily one academic did answer our call and we are currently redesigning her Moodle course in tandem with redesigning her teaching approach so here's a screenshot of of her Moodle as is it is a beautiful example of how VLEs are almost never known for what they are spaces online they may be but they are spaces into which we send our students and where we ask them to be so my academic knows exactly how she wants to be with her students physically to make content with each other to self-select and to research groups to communicate, collaborate, connect and create and she knows exactly how to facilitate this in a physical classroom and naturally resources are essential to her course so are reading lists and if she didn't have the web she would put all this essential information into a course guide carried those guides into the room the classroom in a box and allow the students to consult them whenever they need but as you can see online that box is as big as the classroom itself and it's loud and shouty in week 2 we discuss migration policy and it shouts this in week 3 and 4 and 5 and 6 also every time somebody goes into this classroom that box shouts about week 2 it's uncomfortable useless and ugly so here's a screenshot of her Moodle after I thought about it for a bit and I mean for maybe 3 hours I eliminated clutter I arranged the space thematically I put all the resources into one box and put a lid on it I listened to Jennifer telling me her students need to communicate, collaborate and make stuff so there's a dedicated space in which the students can do that the space reflects Jennifer's teaching approach it is cleaner, easier to navigate and visually more appealing because of it and if any of you want to argue that we can do that later outside and I've got weapons with me I'm just going to let that stand there but much more importantly why care how and why has it become visually appealing more importantly considering what Moodle is a learning technology a suite of online teaching tools why does it matter that it is also visually appealing why do we care about the aesthetic, the visual well first there are of course universal design principles easier navigation and visual clarity increases usability usability ensures accessibility clarity of design means that design elements aren't obtrusive that means they recede into the background and allow users to focus on what really matters their work, their learning and the Japanese word for simplicity or illumination of clutter is kanso or something like that and as a design principle it is to remind us to think not in terms of decoration clarity a kind of clarity that may be achieved through a mission or exclusion of the non essential that was a quote but these are I would suggest surface reasons for a focus on design they are at heart pragmatic and utilitarian which to my mind is reductive this is not to say that these reasons aren't fundamentally important but they are not exhaustive next step so there are of course proper pedagogical arguments to be made on these up under the term of visual literacy visual literacy is necessary so that we may recognize the visual rhetorical power its ubiquitous presence as I said earlier and cultural social importance the world is effused with images it has always been this is not anything new are essentially visual creatures but as Peter Felton points out and I quote living in an image rich world does not mean students naturally possess sophisticated visual literacy skills but as continually listening to an iPod does not teach a person to critically analyze or create music like any literacy visual literacy is an intellectual reflective ability that must be learned so that we and our students do not stand defenseless and uncritical against images that impart messages ideas ideologies meaning and I quote again that composition studies and indeed most academic disciplines are only now beginning to take visual representation seriously reflects a failure of many academics to understand human learning rather than a radical change sparked by technology and culture to train students to see critically and to create and multiple modes should be an essential component of a liberal education that gives away I think that Peter Felton is an American liberal education presumably is an arts humanities education the first step towards this is to recognize ourselves as visual creatures to reinstate the visual properly vis-à-vis its counterpart to the textual and yet I suggest that even with these reasons remain in the realm we remain in the realm of pragmatism and utilitarianism and in any case visual literacy does not explain why online learning spaces should be visually appealing that is it doesn't explain why we should care about the aesthetics of it you still with me which means I'm now coming back to the notion of care and I want to approach it by a two further stepping stones namely aesthetics and the visceral so the aesthetic matters to us because the visual matters because we are visual creatures we are artful we are creative we are homo farba or homo favour we care about art design beauty this caring for the visual constitutes a particular way of being that we are now some say we alone among the animals are aesthetic but if we consider the Sutton Bowerbird and his beautifully decorated structures we might want to lay such reductive categorizations to rest also attracting mates, securing reproduction and not the soul evolutionary drives all behaviour can be reduced to for if they were then so would be this talk because I'm behaving right now moreover decorating their bows is a learned skill one at this it is possible for a little bowerbird to excel at and through life getting better at it it is quite unlike the accidental plumage of a peacock for example the aesthetic speaks to us on the reflective level we reflect on it or we engage with it on that level but the visual affects us also on a more primordial level the visceral our natures are not merely composed of the intellect but also of feelings, instinct the body, the gut feeling designs make us feel good and on this level we can combine the utilitarian with the hedonistic if well designed spaces make our learners feel good then we have a duty of care to design their learning space as well online and offline not only because it will measureably make them learn better but because pleasure happiness, feeling good are goods in and for themselves and if you agree with me that this is a well out motivation for caring about the visual then it is because you care at the fundamental level which I'm saying if you agree with me you're a good person so we care because as humans we operate in the realm of care in being in time Martin Heideg explained that the being of humans reveals itself as care and I quote the phenomenon of care in its totality is essentially something that cannot be torn asunder so any attempts to very taste it back to special acts or drives like willing and wishing or urge and addiction or to construct it out of these will be unsuccessful, you can read this up in being in time page 238 or let me put it less carefully but more accessibly cares at the heart of what and who we are this extends to the realm of education of course and this sort of appeal to remember care and love is not so very uncommon at all in the revenge of the monsters of education technology Audrey Waters reminds us that education technology requires our love and our care so as not to become even more monstrous so that it can become marvelous instead and Bell Hooks wrote that the heart of education as a practice of freedom is to promote growth it's very much an act of love in that sense of love as something that promotes our spiritual and mental growth the academy is not paradise but learning is a place where paradise can be created and here is a final segue I will do this quickly which might illustrate what I'm trying to get at Paul Clay's pedagogical sketchbook an odd text that explains the principles of art design, form and function with sparse sketches and sparse a text an extraordinary little book and I have no time to describe it here I want to mention only why he created it Clay became a teacher at Bauhaus and as a teacher he reflected on his own working methods and techniques and he wrote when I came to be teacher I had to account explicitly for what I had been used to doing unconsciously care in the face of the other means to be accountable to them we care not because of extrinsic factors not because it's written into our contracts or our job descriptions and not because we are being held accountable by others Paul Clay was not told by his employers to account for his methods he came to be a teacher which meant he became accountable to his students from within himself that is what it means to take care Paul Clay, very special artist that he was did not display any special care that would be alien to any of us I suggest that his response was really rather ordinary which is why I feel no compunction in saying that in our practice this is what we do too we care, we know about connecting we connect academics to each other they're students, we connect students to students we know about collaboration we embrace creativity both as a concept for students and as an approach in our practice because fundamentally we care if we don't act out of love we don't really act meaningfully at all this is no truer of a learning technologist than it is of a zookeeper but it is no less true either I'm done Thank you very much and I'd like to invite questions is there anyone who'd like to see anyone I'm happy if you don't Hi, Steve Lotwiton, University of Warwick It could seem like a big rat hole to go down looking at ever making your mood look better and better and better picking your font, you change the colour of it slightly, you change the background colour How far do you go how far is pleasant enough I wouldn't go that far at all this is something like I noticed when I was writing this that I wrote something very similar at all 2004 it's all about the spaces and people don't seem to get the metaphor that is very simple here's a classroom, you know what to do and here's a classroom but it's online do you decide what it looks like don't spend a lot of time on it but think would you put a box in or would you every time that the students come to you and say by the way do you know what we're doing in week 2 it's week 7 already so the design principles are very simple it doesn't have to be white it doesn't have to be green or yellow and those things change and you can in fact pay people to do that for you it has to be decluttered I thought I kind of made that clear but there is a sense of simplicity declutter it for a start and Moodle does come out of the box with a lot of clutter attached to it so that's something that of course the academics don't know but I think a lot of the times that the teachers don't understand that we don't want a static website what are you doing here if you're just using it as a drop box then use drop box because that's you know you don't have to worry about the design did I answer I was trying to say not very far at all yeah that's my answer not very far at all into that rabbit hole I presume your title is in relation to David Bush you'll see the title in a minute and I'll handle that thank you good morning everyone let's just get set up my name is Alex Bears and I'm from the University of Liverpool and this is the vice chair shall I meet you? the reason we're here I just need to give you a little bit of an explanation about this but first I must mention Neil Withnell who's made the decision to actually be in Malta instead of Coventry for reasons I quite can't quite grasp at the moment I don't understand the logic behind it do you? no anyway he didn't ask us he just left us to it so we're talking about the changes the chitcha changes that we have undergone in this open online course which we'll get to but I guess the reason why we've used David Bowie as this motif for through this is because when we were submitting it was in the untimely death of that kind of great man and you know he's someone who kind of actually has changed a lot he has kind of transformed throughout his career and that's something that is kind of intrinsic to what we've been doing over these past couple of days so it's a little bit of a commemoration of that also it ticks the box that Peter Bryan has got a YouTube playlist going on at the moment so we can add tons of things to that playlist and that's part of the play which is another element of this conference ok so just to give you a quick recap this is all about the bring your own devices for learning open course which was developed two years ago by Hugh Beckingham and last night's winner up of the old learning technologies prize Chrissy Narancy and what it is, it centres around a wordpress site which is open to everyone it's participants from all around the world students and staff and it's based over a five day period where we're looking at a range of different themes, the five seas there's been lots to talk about three seas and four seas I'm going to give you five seas now so the five seas are communicating connecting curating what's the next one collaborating and there's another one thank you very much and it takes place on a yearly basis five days and it tends to be in the darkness of January and so this is a little bit of light to pick us all up it exists in these different spaces so we've got the wordpress site we've got Twitter space which has become the centre hub where all the heat actually happens and we've also got a Google Plus space where we connect and share and it's all about getting people to come in and immerse themselves in those spaces and try them out first hand OK, so I'm going to pass over OK, so we were actually thinking about making this a bit more interactive and seeing if you could guess our theme but I think we've kind of given away that one but following the theme of that's been started at the conference I just wanted to have a bit of audience participation at this point and for those of you watching on the live stream playing along too could you stand up if you've taken part and bring your own device for learning if you've got anyone who's actually taken part don't be shy pardon? yes you can, multitask man forgive the sake OK, so we've got a few people could you keep standing if you've been involved for more than one of the iterations oh you all have, you've done it more than once you've alright so you're just kind of doing it all at home oh well I just mean you've done it like one year and then come back the next year, so just for a bit oh you've just maybe been involved one don't do that, OK you keep standing if you've been involved all of the years and you've actually had something to do with running it OK this was all just a ruse to embarrass Chrissy and David sorry Sue and David and Chrissy's not here so obviously there was a golden period when fantastic people like Sue and Chrissy developed this fantastically fantastic open collaborative model you can sit down now if you can keep standing if you want, thank you very much to participate and get people connected and you know we really felt that we were understanding on the shoulders of giants but you know there was a golden period and it was quite challenging for us to go forward in that so in some ways we could say that Alex, Neil and I were beginners in this kind of when we offered to take over and run its facilities we were like oh my goodness but actually we'd all been involved in it from the beginning and we actually had been involved in every iteration and as well as being involved in all the stuff that's happening online each of us at our institutions had been involved in running face-to-face events that tied in with that as well so we looked to that for quite a while so we weren't quite maybe as complete beginners at it but it was still quite easy to fill in Yeah exactly and big shoes it was because we were feeling the pressure there was lots of pressure for a number of different reasons because it had been very successful lots of people had participated in the course but it was also as I was saying tended to be in January and that's quite a kind of dark time you know you've just spent two and a half weeks eating chocolate watching lots of television that type of thing and then you're back into work and it tended to always start on that first week back and what we tended to do in our institutions was kind of badge live events in our institution to get people involved and introduce them to things like Twitter obviously Google Spaces but also kind of the whole 5D kind of like course so we were tended to be running around in our different institutions getting all these things done so there was like an awful lot of pressure going on added to that we were running the sessions as well so you've got your 9-5 and then at night each night for five days you are online in the twitter sphere communicating, connecting collaborating, creating with a whole bunch of people it was immersive, exciting draining very hard to go to sleep afterwards because when you put your head in that stream it is enlivening it's like kind of putting your fingers into kind of like a socket so it's very kind of hard to chill out after that so and added to everything else that we've got going on in our lives so there was a kind of a lot of pressure so just to kind of give you a little bit information about what the course actually entails here are the 5Cs that have been developed by Narancia and Beckinham which I couldn't remember early on apologies for that and let's just go through quickly the days just very quickly run through so each of the 5 days takes one of the themes and people share what they have been doing so we start with connecting and that's a really useful way for people who aren't sure about using twitter or connecting it's a really good way to get people to do that I'm conscious of time so I'm just going to quickly go through there we have a blog post every day that kind of sets the scene again we just went kind of with the tried and tested method so that sets the scene there's a twitter chat at night and people fantastic the amount of connecting that happens across not just on the first day but across the week then the next day is about communicating again we have the blog post we have the google plus space day 9 day 3 was curating always quite an interesting day 3 I think because I think the way we curate things is changing all the time and the value that we put on our resources relating back to what Jane was saying this morning it's a really good way to start talking about some of those more copyright issues and other things around information literacy collaborating well that's what we're all doing the week is really about collaborating but that's day 4 is very exciting and then day 5 is kind of the end of the week we try and get people to create something I mean we've actually been creating a huge amount over the week but there's always something really interesting that comes out on day 5 I think obviously yeah by this point people are getting really comfortable with the tweet chat saying okay I'm going to be here for now I need pizza, I need wine so we get that on the tweet so we get that kind of social interaction as well so that's just kind of a whistle stops to hear what happens each night but yes I'll let Alex explain this one On the final night it wasn't a black star it was a black out Twitter went down 20 minutes into our final kind of Twitter session there was no communication at all and we were all kind of scrubbing around trying to find different platforms to find out what was actually going on and Twitter hasn't went down for a long period in time there was a period when it was going down all the time so yes that was curtailed quite short which I feel quite sad about because that is the time people have eased into the are creating really interesting objects and artifacts and sharing and the are at that stage where they're very comfortable working and communicating and sharing what they've got out of the whole experience Okay so the next steps so the title of our abstract is about changes one of the comments we've got back from the review you're talking about changes but it doesn't seem that you've actually changed very much and in some ways we didn't change an awful lot in terms of the actual structure but if it ain't broke don't fix it that kind of motto so we kind of went along with that but on reflection there are some things that we would like to do maybe to update some of the content but to change some of the collaboration on the tweet chats for example we do a story of them but they're huge I mean there's thousands of tweets so maybe we're thinking could we get people to actually maybe do some more creation and maybe do some more personal stories and get people to actually story their experience maybe something like that but that's just some ideas we have but again more audience participation what we'd like if any of you have any ideas or you've been involved in bringing your own device for learning or you're just hearing about it for the first time if there's anything that you think that would be a good idea you would like us to try then just tweet it out with the BYOD4L hashtag we'll collate that let's just try to sort a few ideas again this is just to show the Twitter community and we always have to do this because this is Martin Hoxley's Fantastic Tags Explorer which just gives you some of the insight so if you have any ideas you can all be our little team of warrior people telling us what to do Hero Turtles yes I'm going to get that one I think so that's it so it's been a fantastic experience we didn't change much but we are coming back it will be back next January the 16th to 20th of January so put it in your diaries now if you haven't been involved in it before please do join in for five minutes, for five days it's a really really good fun learning experience for everyone and again if you've got any ideas if you'd like to be involved please just let Alex know it's been a fantastic collaborative experience for us so yes I think that's about us so thank you very much questions comments fine you can tweet so this will be the fifth time and how many people have you joined I don't think you said it's kind of difficult to get a number but we have more data on the actual number of tweets so I think over the five days we got between two and three thousand tweets collectively so it's harder to pinpoint individuals because people interact in different ways we've got some traffic stats from the blog site but quite a lot of people seem to interact David's got a question it's David from Universal Work I'm really one of those people who's got a comment not a question as someone who's been involved as someone running it and also as a participant I'd say it's really really worth getting involved whether you're actually key to it really active on it it is tiring doing all five days but it is thoroughly worth it so that's my two penny worth it I think another thing just one thing we probably didn't know I think another seed that it brings it can help people with confidence and it gives people a safe space to try things particularly if you're working with staff that you maybe want to encourage them to do something a bit more with social media without their teaching they're really nice, safe, supportive space for them to dip in a night often as David said and just watch or they can actually start to interact and I think that does help with people's confidence which is really really important for all of us we're fine, we're all tweet we're comfortable on those spaces but a lot of our colleagues aren't am I allowed to ask another question Institutionally are you supported in doing this rather than taking it on as something because individuals because you're passionate Yeah I think because you have the week long events so you badge a lot of something around that week so you make sure that you are doing face to face sessions so introducing people to sort of getting people together to work on the mobile devices, think about the different apps that get shared on a nightly basis and so having that through the week helps connect and it helps that buy in from institutions as well Yeah I think Sue sorry, she has a more kind of formal route into some of their CPD MMU and accreditation very similar to Alex we kind of use it more as an informal CPD opportunity and again just because it's that time of year it's quite good people maybe have a new phone or a new iPad so it's quite a good chance for them to kind of play with it in context so yeah it's a bit mixed but yeah we've had some quite good institutional support Yeah Thank you very much Thank you and for the next because we did have we did have Nanash and and it's erroneously the stupid Sarah we will find out it's not Sarah but Sultan I'll let them do the introduction while they set up Great Well, my name is Manoj and this is Wadud Sarah couldn't be here with us but yes Sarah and I we kind of plan to give this presentation and share our experience Well we have kind of prepared this presentation in a way to match the theme but it is more like a story and not a very pleasant story I must say and not necessarily a story with a happy ending but I'll let you decide So the theme or the overview of this short presentation is what happens when a technology is pulled without a pre warning or just give you a short notice it could be turn it in it could be lecture capture system it could be your moodle host or or whatever it is just gives you a year then how does it affect you your team and your institutions and most importantly your users now we will also talk about a shared experience of a loss of technology media or media server and how we all shared our experience how we collaborated and how we overcame the challenge actually how we work together to rapidly overcome the challenge well not fully but partially at least like we shared our knowledge and then we devised a path or a platform from there we took separate ways and what have you learned so far and that's really quite important now formal introduction my name is Manoj I'm a learning technologist particularly within the team or stream of policy and change at Queen Mary University of London and hi my name is Arwadud I'm a learning officer at the University of London so I'm part of the BLE community great so we use Moodle, Mahara, Echo360, Penocto, Tarneten and many services including Media Co that was our media server so BLE they started using media server in 2010 and I remember Sarah presenting it at 2011 alt and that's where I saw what media service and what it has done for them it evolved they worked closely and it was a small company at that time so they really they were very agile they were very helpful so Sarah worked with them to to develop the product integrate with Moodle develop a Moodle plugin and then you have provided reference and based on that reference and based on some other assessment we did we chose to make it available for our medicine and dentistry students and staff and then we released it to all our staff members and students in April 2015 yep and we also helped them to refine the product because at that time they didn't have that tight integration with Moodle we wanted our users to have this experience they should think the media server is a part of Moodle they don't need to leave Moodle to do anything they can embed add upload or share within different areas so you have pretty much here we work closely to develop the product and we also helped them to come up with this sort of things like terms and conditions that if people are using it for the first time they should see terms and conditions and devise a policy and so on and so forth so anyway it was all going absolutely fine so you see the timeline is April 2015 and summer 2015 that was August everything was nice here you can see that and we were also very hopeful that we are going to add video assignment, video quizzes student because we didn't release it to students so student engagement and several other features are going to be released in the coming year in 2015 2016 including Mahara integration but instead of that we received this email from media code so you can just read the highlighted bit so this is what they told us and they said you got a year to find a replacement we offered them can we buy your plugin can we continue using it you don't need to develop it's a very straightforward thing and the answers were no for every single question we will discontinue this there is no choice so what happens these are five phases of grief but we didn't have any time for these things and very quickly we passed through each stage and we thought that we need to do something we can't spend time on this so what we did I will let Vadud explain so it wasn't quite Avengers assemble but we did have a crisis meeting on the 7th of October so we came together and we made a list of possible supplies that we can change and migrate to originally we had 13 supplies including the likes of youtube and then we brought that down to about 5 including culture, media planet history and and media sites what we did is we created a spreadsheet which I will show you so there is a copy of the spreadsheet and we looked at the features that are key to our needs and how each of these supplies meets it and this kind of formed the basis of which ones would suit each institution best in terms of their specific needs so including things like mobile playback moodle and blackboard integration what kind of media files that you can upload to it roles, the hosting also the search facilities so once the document was completed we invited the 5 supplies to come down and do a demo day each supplier had 90 minutes to kind of pitch the services to us and then we were able to then go away and think about which way which supplier we wanted to take so what we are going to do is we are going to look at two pathways that we took and we are going to explain this in more detail Thanks so in this meeting we had what like around 25 or so people from various institutions all the victims of media code we all gathered and after this thing after this combined presentation and demo and going through the shared spreadsheet we thought ok now we all have necessary knowledge, now we know the timeline and we can go and choose our own path from there on we were pretty much on our own still like we were there for each other to give guidance so we are going to talk about what Queen Mary University of London did and what Bloomsbury group they decided so at QM we kind of we love all these prints too and I tell and our all customized project management process and this is a very like I put this on purpose so you see that there are each stages PMO pre requirement design build test deploy early and for each one we have to write a report and it is a very tedious process so we followed I'll come back to it so this is the timeline so we knew about it in December sorry September so in December we built a business case functional specifications were finalized in February projects were allocated in March tender process starting the project so we started that in May and then in June we thought that you know what we can do it in house so we need a developer it's going to help us third week of August finished migration we started replacing because within Moodles we had 4,000 or so embeds so you see like we were there on the deadline 31st of August actually I need to go back and find out if it all went well or not I have been away for a while so that was our process and it was like not a easy task because we also wanted to replace the Moodles embeds so we were almost like hitting the deadline so this is one approach that you follow all the prints to process you do a lot of documentation you write a lot of report and do the actual work in the end which is not necessarily the best way to do and there is another way you can do that so what the really community did was work closely with Penocto the exception of IOE institute of education they went to the media the rest of us being Penocto customers we thought the most logical solution would be to work with Penocto and see how they can migrate the existing content from the media core into Penocto obviously it was a customer of Echo360 and they converted into a Penocto user also so with Birbeck what they did they worked with ULCC to develop scripts to automatically replace the media core links into Penocto links RVC they hired students who created the embed codes and so us what we did was we manually obtained the hyperlinks and the embed codes and we replaced those in the existing Moodle instances so most of the migration happened over the summer it was fairly pain free pain free free so us we finished that about a few weeks ago so all our embed codes are all updated I contacted the relevant departments they went into the Moodle sites and websites where those existing videos were and they updated those about any kind of problems so you see the two ways so we had to hire a procurement manager we had to hire a project manager we had to do a developer we followed all this process like a gateway the one you saw I mean that was the short pause and each has its own life in this and it's still like we're hitting the deadline whereas they thought you know what we saw the demo and we're going to go with that and by March some of them were ready to go live and they hired students instead of looking for a developer who's going to run a script to replace the Moodle codes or something so the conclusion well as the theme suggests collaboration was a key because we thought we are not in this alone like one of I think ex prime minister said we are in this together but yeah we had a very different reasons for it so yeah that was very helpful we all came together we built this platform we shared our experience and impact that's the worst part at least I know and I can also speak for Sarah that it kind of pushed us a year back like my plan was to work a lot on learning analytics and undertake some projects and do something on the assessment couldn't do any of that so that's personal but it has a huge impact on the team as well because the resources and timing are located to that project replacing that and yeah procurement now that's the final bit like if anything like this happens to you or if you have to do it would be our recommendation that you know as a learning technologist or involved in this field you know what would be good for the institution just don't get too much involved in report writing or doing like following all the process you have to do some compliance but to pay a lot of attention on the actual product or there's a procurement process called competitive dialogue just shortlist two or three based on the minimum requirements and bring them to your institution spend a day, half a day and ask grill them a lot because what we learned we had a tender document of 150 pages and trust me when it came to presentation we were just shocked to see that oh it's a whole different thing they hire lawyers or somebody else to fill the tender document when you ask them when you refer to it they have no clue what we wrote they say oh yes somebody else may have done that so these things happen so yeah just saying it with your own eyes is the best thing spending half a day for shortlisting one year seems a lot it's not and yeah and then mainly the most difficult bit was user expectations like we didn't let our users know until the last month when we knew what we were going to do and in this case we made sure we ensured that all the things are done in the background so that they don't need to re-upload the videos they don't need to re-embed the videos within Moodle we did everything in the background but that was the most difficult bit I would say and for learning experience I personally think that this failed product in phase one has taught me a lot more than five successful would have taught me personally and time will tell like what happens how does it go maybe in a couple of years time or a year time will come back here and tell you more about the impact more about the experience and yeah that's pretty much it from us but I would like to hear from you about comments criticism or something else I missed here thanks so a lot of good suggestions there is there anything else that you'd like to do differently and also two prongs to this do you think it would be possible to do things differently so obviously there are certain processes for institutions do you think you'd be able to do the things you want to do rather than follow a certain process well differently well there are two things one would be not do at all and find another job I mean that would be a lot easier but joking aside no it was very painful but joking aside I think I would do pretty much what I mentioned I'm not going to follow all the process all the report writing all the documentation because usually most of the things the moment you select something it's shelved and you have to restart the whole thing and then we start because of that we started the actual dialogue with the supplier in June what we could have done in Feb or March so that's the main thing and if you have experience just seeing the product criticising and choosing and just get on with that and I always think you know what when it comes to a project it's not the process it's the people who make it successful if you have the right people they can do it do you want to add something I think we were quite fortunate working in the Bloomsbury area with the other institutions working together we were able to draw upon different experiences share ideas what kind of needs we had if we were doing this by ourselves I think maybe we would be able to pick up all the points and the kind of key features that we require so I think that was very useful for us to see what other people were doing especially in the bigger institutions that kind of look more towards the future rather than the immediate impact that it's going to have okay thank you David Hopkins from Warrick again I think we all feel your pain here with what you've had to go through and as a comment what you've been able to achieve it just goes show that when the pressure is on you can achieve a lot in a very short space time but the question is how much of what was going on influenced your decision on where you went with your media serving the future with the decision what you went with and then the future of that the deadline was the most important bet although we didn't stick to it like we still we didn't you know that in this case we have an existing service it's running we have to make continue running it keep on promoting then we have to find a replacement then we need to do that transition it's all like going to happen in a year time it seems a lot but it was not so the most important thing was that we need to make sure but 31st of August we won't end up in a situation where we have nothing so that was the top most bet going forward now we have some sort of contingency plan in place that if this happens hopefully we won't take that long and well this is a critical service we need to promote it we want our users to be innovative and we need to enhance learning experience so we need to have media server but yeah going forward we are better prepared we know that this can happen to any of the services we are using but I think still if it happens it's going to be equally painful it's not because we have an experience it's going to be easy that won't be the case a lot of unexpected things as a package I'll probably just going to mention about Penocto itself because we were heavily invested in Penocto as a lecture capture it was useful for them to be providing the media services to us our staff and academics were already familiar with Penocto so it wasn't kind of reinventing the wheel as such or retraining people just showing them the new feature is an added button into the text editor you can just do the direct upload on Penocto itself there was nothing extra that they needed to learn so it just made sense for us in that way whereas in our case it will be a whole different thing users were using a system that was totally different and now they are going to be using a system that is completely like the process would be different embedding, adding although it has more features including video assignments or something which we couldn't do in the previous space but yeah still it will be very different for them I am representing the team I am yeah The collaboration in the part of the MOOC Okay hello everyone I'm here with my team the healthy learning media team you can give a little wave over there I'm going to talk a bit about a MOOC that we delivered last February so what I talked about referring back to the abstract was I wanted to talk a bit about how we introduced into our process of development of a MOOC a collaborative ethos MOOCs they are not new anymore there is lots of them around and I think a lot of us have been involved in them but we wanted to take our experiences really and see if we can practice what we preach a little bit we also wanted to show a few examples of how learners collaborated on the MOOC as well so very much within the themes of Oatsy so I'll just start briefly by saying a little bit about how the healthy learning media team works we're based in the University of Nottingham's Health Sciences so we deliver a curriculum to nurses midwives physiotherapists and they can raise your five Cs to six Cs because the NMC code has a care compassion courage competence communication and commitment and these are embedded throughout the nursing and midwifery curriculum but we can operate within this infrastructure so we have a local school based curriculum we've developed all the curriculum mostly flipped and trying to make the mood all look nice as well so totally with you there on that we have big research in which Richard Wendell, Heather Warrod, Mike Taylor work very hard on and this is research around from health informatics education e-learning and health education all sorts of stuff our community is the people we work with so we work with groups to design e-learning materials includes students, teachers, commissions service users, carers NGO, social enterprise, local authorities families, parents all sorts of people they've actually formed part of our community and they come in and help us e-learning in a participatory way and the fourth arm there is the open so the stuff that we develop the learning resources that have been developed by these participatory groups is open available freely for anybody to use as well as part of our ethos so these are some of the e-learning resources we design I wanted to show you just the short video clip if it's working which gives you a bit of a flavour of the resources and how we design them give it a minute the internet then oh there's no internet connection oh okay maybe we could come back to that yes it's probably too I'll set it up maybe I'll show it at the end then if it pops back I don't know what's happening there or when things go wrong that you can't plan for okay sorry that's slightly thrown me actually a little bit okay I was going to show a bit of a video anyway of how these are put together all of our e-learning resources have participation at the core of them this is actually the methodology that's been developed within the helm team over quite a number of years by some of my colleagues and it's the finished product as a result of this quite detailed process but it provides a very quality of output so just to summarise that we have the aspire framework so the learning aim are a short and tightly defined learning aim for the learning object we have storyboarding workshops these are where we involve the collaborative sessions and the people the communities that come together to develop the resources with us we have population which is thinking about the resource specifications so from the storyboarding workshop it's actually okay we're going to take those storyboards and we're going to put them into something that actually you can design from I think that's probably one of the most challenging aspects is trying to get that short and that tight and that focused and implementing the resource which is what you're going to do with it what tools are you going to use I think there's a presentation later by some of my team called gone in a flash but it's about what technologies are you going to use to make it an accessible thing that's accessible in terms of disability access but also in terms of who can actually view it and what technologies are available and that's part of the release set and evaluation all of our learning resources have feedback forms attached there's research we've been some of the team have conducted on these as well and the whole process is enveloped in a peer review iterative quality process with pedagogy right at the heart of it so this is actually a big collaborative approach in itself that goes into these finished resources that we make so what we wanted to do is open up this methodology so that we could share it with people through our MOOC for designing learning for health MOOC has anyone heard of it or been on it great, yeah it's running again in February the 13th if you're interested so you can go and have a look for that but it was the first MOOC in the university of nottingham that was delivered by an entire team so Helm is quite a big team we've got a lot of work on actually but we have people from the apprentice in our team up to the professor in our team as well actually working on this MOOC showing it as a collaborative venture so I was going to talk a little bit about our journey on how we how we developed the MOOC I wanted to show a little montage video as well but I've got the feeling that's not going to be available so technology conferences aren't they great no bloody wifi so this is a few shots of our team we are actually we're quite a close team this is a jumpers picture that keeps coming out at every single presentation we do now because it's a bit daft we like to encourage the team to take part in self development we've got a lot of us here today because we value self development and our school of health sciences also values it part of doing this MOOC was actually to improve all of our practices in different areas so I was going to show you behind the scenes that Camilla put together it was a kind of montage with some music so you can imagine it in your head a sort of team America type montage our MOOC was designed by a group and it was we worked very closely with the university staff who I'm going to mention which is are we back okay should we go for the MOOC behind the scenes built it up now it's not going to be as good is it no it's not doing a lot I'll leave it I think because it's sort of distracting I wanted to mention Steve Stapleton, Sarah Spate and Sarah Stubbings from the University of Nottingham who are part of the core MOOC team and we very much they were part of the team as well as was I think Emily Conway from FutureLearn who we worked very closely with so we would consider those part of our wider team so we have six of us educating on each of the weeks I think everybody apart from Heather today is here and we each took responsibility for our weeks content we used Google Docs as well to collaborate on the design but not just the course design but the communications the promo video which if you've done a MOOC it's a bit of a nightmare to be honest putting that together all our filming scripts everything so we collaborated on all of those and I think it's fair to say as well that there were some of us involved in writing the content and being the face of the week if you like but other team members also got involved in facilitating the discussion so they had training from people in the university they had opportunities to upskill in that area if they weren't comfortable being online in terms of discussions we had weekly videos every week so everybody was involved who wanted to be in filming or editing developing the presentations development we also had which was slightly unique to some of the MOOCs we had a project going on within the School of Health Sciences with academic staff called Virtual Exchange and involved the University of Nottingham and Birmingham City and they were developing their own learning resource their reusable learning object at the time so about a year ago last June we filmed the storyboard workshop for the entire day and we interviewed participants in that project as well and they formed a kind of part of the story so all of these people here were academics, clinicians, students they're all involved in our project as well and they actually were very much part of the course and they took part in the discussion forums and the students as well were involved they actually got involved in the filmed workshops they also got involved in being the subjects of the learning resource and they took part in the the NMC professional competencies and how you behave yourself when you're on a bus so they took part in the filming of that but they also took part in the peer review and evaluating the script and evaluating the e-learning resource so this is some of us here we had stuff built into the MOOC that helped provide our collaborative ethos to the learners of the MOOC as well so we had all these weekly things weekly Q&A all of us there waving so within the MOOC itself we had, as all MOOCs do thousands of learners all over the world but we wanted them to develop a specification for an e-learning resource but using the kind of methods that we do face to face so we had to build into the MOOC some collaborative type of features so these are the sorts of things that just off the top of our heads really popped in inspired by other courses I've done the Edinburgh EDC MOOC a long time ago and I think that was a really interesting way of running collaborative MOOCs collaborative courses so these are some of the things that we did and it was providing a kind of safe space for people to feel comfortable and also to help them link up so over the course of the the weeks we had a number of topics sprung out people to design a specification for an e-learning resource around a topic they're interested in this is a list here of some of the ones that they started to emerge from the forums so part of our job was to try and match them up and encourage them to see if they wanted to develop a storyboard together but online so we did actually have self-organised groups start to collaborate and I'm just going to pop up a few quotes around the sorts of things they're interested in so as you can see there's a they're absolutely wonderful the topics they came up with and they completely surpassed anything that we'd have thought of because they are situated in the country or the locality where the people are so we've got things like supporting refugees in Germany I'll show you some of the storyboards that they created as well so what we've got them to do they develop storyboards using different types of tools the usual kind of things and some tools we hadn't heard of before as well some of them went off and created a storyboard with their friends and family or colleagues back where they were and then they photographed them and stuck them on a tablet we had you can scroll down here but we had loads and loads of really well researched valuable e-learning specs coming out of this project and here's just a few examples so we've got leadership to health personnel in Tamarun mosquito breathing around homes which actually having suffered a mosquito bite this week which has made my foot swell up over the last two days I can totally resonate without but also with Zika virus it's very serious issues in some countries people did them in different ways really creative so I think we were just blown away really weren't we with the type of resources it was created so a very quick reflection I think the key to making it a collaborative MOOC and one that actually reflected our ethos was facilitation everyone in the team provided facilitation we established a rotor using outlook and that kind of thing and maintained it sort of day and night really we used the Yammer group as well so if for example something came up in the discussion forum that I didn't know about maybe Richard or James or somebody else knew about we could just alert each other and keep them very much engaged everyone's had different styles as well so we used the breadth of the team's experience some were very good on the technology some better on pedagogy so it actually worked out very well the whole experience was really good for team building as a big team we've had two massive projects over the last year as well since then I think all of the skills that people have learnt not just working together but the skills around the filming etc has really helped it was also good for the academics involved some of them joined the course as well they've actually now learnt about MOOCs they've learnt about what you can do online differently it's moving the school itself forward in terms of what it wants to do in terms of its own online distance learning so I think as a joint approach as well we had comments back from the team the team were actually motivated and supported to collaborate everybody had quite a lot of freedom to put into their own weeks and their own content and we even had a sort of helm card game thing, virtual card game which learners on the course could find out a bit more about us before the course started so I think I want to end it on that really I'm just going to plug the February course which we're going to be delivering again but yeah I'll see if I've got time for the video or not shall I pop it if it works never mind maybe take some questions because it's lunchtime isn't it in a minute it worked for them before the session because I tested it which one this is now volume's not on either is it sorry I think we'll forget it to be honest never mind if anyone's got any questions be happy to answer them or member of my team it's just you've done the course once did you make any substantial changes before going into the second one well we're about to do that now really so we're going to wait till the start session's finished then we'll convene well with the university's team and kind of review the whole thing I don't know if we've got any comments on that but nothing we don't want to put too much changes into it really and I think the thing that worked really well was the real time kind of interaction with the learners well thank you very much especially given all the technical hurdles you have to face thank you so thank you to all our speakers thank you everyone for coming and bon appetit it's lunchtime so some background on the big picture so this peer marking activity was created for a company called Avado is a skills based education provider and they partnered with all kinds of organizations and for this project they partnered with Google so it is a squared online was the course and it is a five month course on digital marketing and leadership so students get to walk away with digital marketing skills and leadership skills it's a cohort based course where all of the assessments are group work it's based on social collaborative and problem based learning the cohorts are huge because I had come from higher education into this corporate space so I was used to working with 25, 30 cohorts and then here I am here I am make a peer marking activity for up to 500 I'm like 500 people to create something for 500 people is very different than creating something for 25 but it was a challenge and I gracefully and happily took on so the squared online is delivered entirely online and to date we have proximately this is an approximate number and some of my colleagues in the audience might want to correct me on this but approximately 3,000 students have completed this activity to date so we've had a lot of student feedback to read through and consider and so on and so forth so my instinct as an instructor to go any questions before I move on we're all good and I like to get eye contact with people but it's kind of hard when you're so far away okay I'm going to keep moving on okay so why do we try and peer marking probably the same reasons you tried peer marking with your activities we wanted to really foster those leadership skills that we were trying to encourage and develop in our students we wanted to empower the learner we wanted to actually we wanted to incorporate learner feedback really treat the learner as a partner and make them feel like they were listened to and that we were actually trying some learner center design there was also another impetus to try peer marking Avado has a number of courses and they partner with a number of organizations and they had tried peer marking already to a limited degree of success and we really really really really really were motivated to get it right because we really believe in it and we really believe in the idea of empowering the student giving them a voice and creating leaders for the future Sound familiar? Does that sort of anybody else have the same kind of motivations for trying it? Leadership, foster leadership skills yeah I see a couple hands raising up so when we looked at the activity that was created in the prototype I'm going to go through an actual high level outline of what the activity actually was and what the student journey was when we looked at the activity and we looked at the challenges of why maybe it wasn't working three three themes emerged fostering trust and engagement and motivation with peer marking seemed to be the biggest challenges and the biggest hurdles to overcome self trust can I trust that I as a student can offer anything meaningful to my other students as a student can I trust others to actually give me meaningful feedback I was a student once doing my master's degree online and I had a assistant professor who was only one year ahead of me she had just taken the same course the same master's degree that I had and she was already hired as an assistant and she was marking my papers I was furious like angry I'm like who do you I came to this course I paid good money I want an expert to give me feedback so I feel like and here I am I have to kind of convince people like me that this is an actual worthwhile activity to learn something from it and that you all have a lot of things to say and you can help each other power of social learning right so trusting of others trusting the actual learning is this really a learning a meaningful activity or are you guys just trying to cut costs trust me it is not cheap to do peer marking properly it is not cheap it costs about the same if I may be some hold from my experience so it doesn't to do it right I think it's not a cost cutting feature okay the last thing motivation engagement how do I get students to actually want to do it how do I get them excited about it so those were the challenges that we had to overcome so the prototype I'm going to take you through the prototype and I'm going to take you through the iterations I think that'll help you understand the big picture any questions before I move on I can't stop doing that I know you don't have mics but I have to we're all good okay so the prototype which I call limited success it was built on Moodle using the discussion forum with the ratings feature implemented it was a self-selected activity so students got to pick their own assignments we didn't give them assignments to mark and it was a one to one ratio so one assignment and one individual so students only got feedback from one person some students didn't even get feedback because of this self-selection option some students didn't get feedback and that was a real problem a marking instrument to use a three column rubric with three criteria, very simple we thought it would be a no brainer no problem, three columns, simple there's no way they can have a problem with this and then the written instructions we also kept those really really simple just simple, just written nothing too fancy no other really crazy support and the actual activity itself was formative so there was no grade and that's really important too I think so you can probably start to see why we were having motivation issues why we were having some engagement issues some trust issues you can kind of start to put those pictures together for yourself okay we're going to fix those issues we're going to foster trust, we're going to build motivation and we're going to get these students super engaged so some things were working with the original prototype and that was the discussion forum and the ratings and we really believed in that because it was simple the students were used to the technology they seemed to be able to negotiate it no problem what we did change and what we did tweak though is we changed the ratio instead of one to one we changed it to five to one so every individual paper got five pieces of different feedback and that aligns a lot of research that we saw as well that it takes five pieces of feedback to replicate an actual expert tutor mark and we did find that that actually worked for us as well so we just tweaked the discussion forum a little bit we tweaked the rubric a little bit we kept the three column rubric but we added more criteria it made it a little bit more complex and that came from student feedback students were saying we can't really give the feedback that we want because that mark instrument is actually limiting us so we evolved that as well we strengthened the design so we gave more signposting we added video we gave lots of sampling we turned the actual project into a summative activity so students were getting a grade and I'm just going to give you a quick example of one of the videos that we created bear with me this did work earlier but it might not work there should be sound I'm now going to introduce you to a new type of activity peer marking you've been building your capacity to give and receive feedback in various ways during modules two three and four and we've introduced peer marking to allow you to apply the critical leadership and reflective skills that we've developed so far we want you to be a key part in deciding which teams have delivered the most compelling work once they have been marked we'll share everybody's project and their feedback so that you can see what others in your cohort have created and what they have to say about the work there are a number of advantages to peer marking that will support your learning in this final module by taking assessment out of the teacher's hands it shifts evaluation from teacher driven to a learner driven model and provides students further learning opportunities peer assessment can lead to a deeper understanding of a topic of others by practising peer assessment you can discover other perspectives on a topic which can broaden your understanding participating in peer marking you're in a better position to understand the grading criteria you can then internalize this understanding and apply it to future work and to improve your own performance which are great skills for future leaders each group leader will submit your assignment the same way you've been doing in all of your other projects you'll then each individually be assigned a paper to mark and give it a rating out of ten each rating will be averaged out in order to complete this activity you'll need to familiarize yourself with the assessment criteria ok you get the idea alright so we added this video as support which seemed to really really work which seemed to really make sure I get my presentation which really seemed to help students along so we strengthened the design we added some videos, we added samples we added more modelling, we turned it into a summative project we also added incentives so we actually picked the three most exceptional papers and the three pieces of exceptional feedback and rewarded them, we invited the recipients of the exceptional papers to actually present their paper in a live webinar then the students actually voted within that live webinar for their favorite paper and the favorite paper got published on the squared online blog so if you're a digital marketing student that's huge motivation, that's really going to engage you and we have been publishing papers every cohort it's really a really really encouraging sign and I invite you all to go check out the blog and see some of the content but I think the number one thing if I only have one minute left or two minutes three minutes left that I really want to talk about that I think led to the success of the first iteration of peer marking was expert presence and intervention we didn't hire a marker, we hired a tutor to participate on the forums and to intervene whenever they saw that feedback was getting a bit thin, a bit light not appropriate, so the students then could trust that the feedback they were giving was appropriate, the feedback they were receiving was also appropriate trust, motivation incentives we thought we nailed it so we ran that for a couple of we ran that for a couple of cohorts so I guess about 500 students went through and then we noticed some themes coming through in the feedback these areas that are greyed out are areas that we just kept the same but in the second and last iteration we decided to edit the rubric because we were getting feedback from the students that that simple rubric that we had created was too simple it was still too simple I'm just going to show you take you through and this is the last thing I get to talk about I just want to show you the before and after so you can see the before rubric it's ugly, isn't it like it's white, it's got like three columns there's five criteria, it's all qualitative criteria so that's good so I'm quite happy with the design but it didn't look like the rubrics that students were used to working using as an actual student that the tutors actually used to mark the papers so we redesigned it we redesigned the rubric to look more like what they were used to we increased the columns to a six column rubric we added four more criteria we made the whole paper out of 60 marks instead of out of ten it looks like the stuff they've already been using throughout the course so we found that that has been really really well received so students weren't they could trust that we actually trusted their knowledge and their ability to synthesize all this information so without further ado I think I've got maybe one minute so I'm going to just speak ahead we're not going to get a chance to talk it's not really enough I'm just going to speak ahead I'm going to skip about that skip that, skip that you can take a look at the exemplar paper the very first ones that got published on the blog you can go take a look at that, it'll be on the site but if I have, I'm going to leave you with any talk tips and tricks because that's what I promised with this talk is that really really focus on reducing the cognitive load by implementing simple intuitive tech and marking instruments that students are already familiar with try to foster trust by providing to the guidance we found that worked really well use incentives to motivate and engage and experiment, analyze and iterate so with all that, there's the emergent model what I couldn't find in the research and why I think there might be an emergent model is I couldn't find anything explicit on the value of experimentation in any of the literature if anybody here in the room has I would love to know but that is why I think there might be a potential for an emergent model because for us without experimentation we wouldn't have had the success that we did have okay those are the references, if you're curious I did it, 15, right? yes, okay thanks everybody, thank you so much okay I guess I can stick around here then I've got five more minutes up here okay oh I have to wait for the questions oh we have to wait for the mic because we're recording this hi so I've got two questions my first one is really interested to hear how you involve students in the iterations getting their feedback and how you involve them in the process of iterating and then the other question was I'd really like to hear a little bit more about why you think do you think it was a presentation thing around the reason they wanted it to be a more complex rubric rubric was a familiarity or do they genuinely want more guidance on that okay the first question okay so I'm going to answer the second question first because that's the one I remember I think that we didn't it's hard to know whether it was the actual production value you're right I'm guessing I'm just making an interpretation and that's my reading of the situation because we didn't actually test a black and white rubric with six criteria in changing just the content we didn't test that so now that we've gone two steps ahead it's kind of hard to go back and give students something but I don't you know what you're right I don't know it could just because there's lots of sessions talking about that when we're creating and constructing new environments for students is it just how much is it the environment and how much is it the actual design and so your first question first one was about how you involved students with iteration okay so after every module I think it's about a 30 question survey where we ask students like what is their opinion what was valuable most valuable there's a lot of likert scale stuff there's opportunities for quantitative and qualitative feedback so we gather all this feedback at the end of every module and we ask a specific question about the peer marking activity so we do that and then we just we look at the actual activity ourselves and for some of our bespoke some of our corporate clients that we do special size versions of in different ways of measuring and that's sort of sort of a bit quant and qual is that answer okay yeah oh sorry I have to wait who's excited to try more peer marking now anyone did that help while we're waiting okay so you post assessment gagnais 9th step or no 8th you said something about building trust in the forums by somebody monitoring the discussions and then I didn't quite catch who was doing that oh sorry I probably was starting to talk really fast because I saw the minutes kind of increasing we hired an expert speaker so we have tutors that mark the papers so instead of marking the papers we hired someone to participate on the forums and just to like live on the forums breathe on the forums and intervene and read all of the posts and to guide feedback and to prod along so the discussion forum was part of the peer marking because you got the criteria grid which is separate to that isn't it yeah so we set up the discussion forum so that within the discussion prompt we had the rubric and the marking instrument and we had the ratings we had an option on Moodle to select ratings so students would and we had lots of signposting I don't know if I have time to show you what that looks like but we had lots of signposting instructions to guide the markers on on how to complete the activity but it was all done within a discussion forum it wasn't done, we could have used workshop, we tested it but it just was too much too much for and we just didn't find that it had the functionality that we wanted and I'm not sure if I answered your question no you did but I think it's quite interesting because when I was someone who participated at master's level in this type of activity I did find it very frustrating for the feedback that was coming back because it wasn't true basically and you had no one intervening because it wasn't a discussion based activity it was just the use of a criteria grid and written feedback I'm not sure if Eropa is here but Eropa has an amazing tool but I found that is so you can be able to have some forum for discussion so that students can ask questions and that you can actually intervene and ask those questions otherwise how do you know that the correct thing is going on really but then that might contradict why some people might want to use peer marking as well as a way of not having to do so much because it's too much to mark but there's still then that involvement isn't that of you're still overseeing something in quite a detailed way and for us because one of our methodology is social and collaborative learning it just tied in we had to have that element of social collaboration and we had to have open a forum for conversation and one step that I didn't actually share with you was that at the very end after all the papers were marked we opened up all of the forums so that every student can go in and see how others marked was completely transparent as well as being 100% social so people could kind of go oh who got a nine oh wow you got a nine wow and you could comment and we kept the conversation going yeah I like that idea because I know I was like you I was so frustrated who is this young thing marking my paper what does she know she has no life experience I remember being really angry when I was a student now of course I'm in love with it and look at how tides have turned but I think I can empathize I'm afraid I'm going to have to cut the question short if you can catch will you be around later? I'll be around yeah so thanks so much everyone for your time and attention, super appreciated thank you very much and I'd like to invite Sarah to come up and give our second presentation and Sarah you're from Warwick I have come just a few metres along let me find my slide I'll just show you one thing first if I could you know it's a short session and it's hard to kind of look at a marking rubric and other things and watch and listen all at the same time so I've created a Mahara page and it's open to you via a short bitly link so that's on the first slide so my materials are all on here including the marking rubric there on the right hand side so that gives you a bit more detail about the work that I've done and how I've done it in the rationale and the kind of critique of it and also things that I've read and referenced so you can access that and I'll leave the first slide on just long enough for you to get that bitly link so there we go so very clever of Alt put Marnie and I together because actually we're talking about some similar things slightly different, Marnie's very small scale actually so Marnie's about work with a very small group of students, eight students and you were also talking about I was very interested in the fact that your rubric became more complicated had more bits added we were at the point where we thought ours was very complicated and needed to be reduced so we're kind of coming at it from the opposite point there so this is a sort of work in progress I'm not calling this a finished product but it is a product that the students produced with me together to aid them to peer mark and self-assess their own work for a short postgraduate award this is kind of Alt C part two for me so I went to Manchester last year and last year I talked about the course which through which this was a part so I ran a course called Transforming Technologies last year which was a 20 credit postgraduate award here at Warwick and the award was a 10 week short course all about the use of technology in teaching and learning and as part of that course I wanted the people coming to the course to kind of really get under the skin of technology and curriculum to really understand it but also to have an agency in the curriculum we had so it's not about my curriculum imposed on you I want you to have a say in this curriculum we're all experts tying into notions of students as producer and students having agency in curriculum and their resources these are all teachers know what teachers know and I wanted them to be able to contribute as well and one of the ways in which I wanted them to contribute which I thought was novel was that we could do a novel piece of assessment which was a digital poster which counted for 20% of their final mark and that the students with me would kind of co-author their mark scheme for that digital poster so I'll tell you more about that in a little while so it's sort of looking at the sort of meta-curriculum if you like as well I just got a question to pose to you actually I'm calling it warm up question but it's very hard in a lecture theatre like this to be interactive and with time limited so I just wondered how thinking about your experience at ALT you may have been here for three days or maybe just one but how would you assess your own engagement at ALT C over the last three days and I've come up with a little rubric here a system of assessment so you might be a passive observer you might be an active observer you might be a passive participant or an active participant but you might be very crudely flung together four ways of measuring your successor ALT and you can see that from the top is that the person who's kind of turned up, made no notes, might have had a bit of lunch not made a contribution right through to the person who has been tweeting like mad probably consumed the most wine at the conference dinner has engaged with all the play stuff they've used social media, they've been engaged in sessions they've asked questions, they've contributed they've made a presentation they've had the full fat ALT experience so I don't know in that rubric I'm not going to ask for a show of hands but you might want to consider where you think you sit in my rather crude assessment rubric but my second question to you was if we were assessing ourselves at ALT and our participation and our engagement at ALT would you want to be responsible for writing that assessment criteria how many of you would be interested in putting that together and bringing it together and one of my questions is how far are students interested in this and that business of trusting yourself as an expert as well as trusting your teacher I think the things that Marnie said really rang true here as well so have a think about whether we are enhancing the student experience by getting them engaged in this way I love e-portfolios I'm a real e-portfoliosado I use Mahara e-portfolio here we call it my portfolio and I've had great success with it here not personal success but my students have used it really well so currently I work with postgraduate students who teach here at the university but I've worked with a whole range of teachers from the HE and FE sector and I've seen not just that an e-portfolio is a brilliant place for putting your stuff but it's also a great place for seeing the transformation and the information of learning that's the thing that really gets me excited I've been able to see how students through reflective practice in an e-portfolio have changed, have moved on in their identity they've shifted that that's what learning is about to me and an e-portfolio I think has got the potential to capture some of that what we tend to do in higher education we spend a lot of our time assessing and thinking about the cognitive domain so you'll recognise this, this is Bloom and we spend a lot of time considering academic norms knowledge, analysis, synthesis etc that's very HE I don't think very much of what we do in assessment in HE works in these other domains actually so we don't think about the emotional side the affective side of learning and that's what I mean by kind of transformation of thinking, change, identity, attitude I'm not sure how much of that we capture it might be a byproduct of studying at a university but how much do we capture and potentially try and measure and then the other side is this physical side I'm calling it physical but it's about the kind of making something curation, craft, digital literacies publication management publication tends to be in the domain of people who have already graduated and are doing a PhD or a lecturer's here but do we give our undergrants the opportunity or are our master's students the opportunity to publish, to be publishers so these are all themes that I'm interested in and it's that physical domain that I want to explore today thinking about my project so the project was as part of the transforming technologies postgraduate award to co-create with my students and test and carry out an assessment matrix or rubric for marking an e-portfolio and actually more specifically the students' task, the students assessment task was to create a digital poster on a technology and the idea was that they would create this digital poster in Mahara they would anonymously peer mark that, I would give feedback as well and then we'd compare how we'd use the assessment rubric first of all of course we have to write it so that was my objective and it was part, like I say, of giving the students more agency in course design a sense that they belonged and to see if that actually improved grades and participation so some of these themes are things I'm interested in alternative assessment methods this idea of student as producer curriculum as a process as well not just an end product again HE is quite obsessed with the end exam, the certificate the two one, the whatever else but what about how you do along the way what about understanding the process other people are talking about this sort of thing so I've just picked out here JISC and the HEA has two people who have growing bodies of work and interest in this area I like this notion in the HEA quote there of assessment literacy my background 20 years ago I began as an adult literacy teacher so the term literacy is very important to me and we heard this morning about copyright literacy and we talked about digital literacy so multiple literacies is something I'm interested in and assessment literacy and integrating assessment literacy into course design I think he's very powerful this notion that I think students sometimes don't always appreciate what they're being assessed on you know have come across students over the years you've gone oh is that what the assessment criteria is it's never been made explicit to them so one of my questions was if it's made explicit not just explicit but actually if you help to write it does that improve your performance or help your understanding even further so we'll see so students created a digital poster in my portfolio I'm going to show you a couple of examples of those and it was going to be part of a repository we have here at Warwick called TAP Technologies for Academic Practice and it's actually a staff repository currently staff have created pages about technologies which they've used and found interesting so these will be the first student contributions to TAP as part of the assessment the students design tested and evaluated their experience of writing this created their poster and then marked somebody else's so thinking about how I integrated that into course because of course this was a chunk of work that needed doing and we were time constrained it was only a 10 week course so if you have a little look at the diagram there on the left hand side are the themes for each of the weeks of the course so we bear on with the digital self curriculum design the things that are in green represent distant study work and the rest of it was face to face sessions 3 hours a week for 10 weeks more or less and it was at these points sessions 1, 5, 7 and 10 that we worked on this in fits and starts so we introduced the concept to begin with and then at session 5 the students looked at TAP pages that already existed and decided what made a good digital poster and from then we started to come up with what we needed for our assessment matrix so where do digital skills sit well it's an interesting one because HEs I said before traditionally assesses knowledge and understanding analysis critique and academic information kind of conventions where do digital skills go are they a subsection of that last area or do they sit on their own in the end we decided that digital skills needed their own column so we had a four column matrix digital skills was the extra one if you like but in addition to that we threaded digital skills throughout the other three so digital skills appear in knowledge and understanding they appear in analysis and critique and I'm going to give you just a couple of examples from the matrix because again looking at it it's quite quite time consuming here but you'll be able to see the full thing later so in terms of knowledge you can see there that this is what I've picked out as the difference in terms of distinction, merit, pass and refer which are the warwick postgraduate scale postgraduate level it'll be familiar to you and you can see the differences there from a limited command of a tool to solid understanding of a tool including the software or platform in terms of presentation skills these were some of the things that we might expect so that for refer digital content might be limited to just text and hyperlinks but at distinction level it might be about all kinds of things multimedia, RSS feeds infographics, embedded polls you know that kind of thing so you can see the difference in complexity or skill in there and then if you think about that last column these are the things, some of the things that we assessed in terms of digital skills authentic versus appropriated materials so did students make their own screencasts and include them in that page for example did they use, connect their artefacts well did they think about the reader in terms of accessibility scrolling linking things, how current were their ideas have they thought about copyright have they thought about interoperability so there's an example of a student poster a student looked at the tool, I know it and another example of a student poster they looked at social media they looked at Facebook in particular because these are assessed pieces of work you can't actually get to these beyond today the screenshot gives you an idea so let's think about the peer marking exercise then so the students were given an anonymous piece of marking, they marked anonymously they used the matrix the rubric and so did I so the students marked first and sent me their feedback and I added my feedback to it so there was my feedback and the students feedback together and this little strange looking colourful blob on the screen is all about how our marking tallied basically so wherever you see a blue squiggly line and a yellow blob in the middle that's where me and the student agreed on the marking and for this one that was about 75% agreement so I asked students to highlight on the actual grid itself on the marking criteria and it was quite detailed as you can see I asked them to highlight where they felt that particular digital poster had met those particular things so it was an interesting exercise and I can say that one was at about 75% agreement so we included this but we included that qualitative feedback as well feedback from the students on this exercise then was actually generally very positive so students A valued the digital skills but they also quite liked the process somebody says here marking someone else's poster makes you really look properly at the marking criteria it brings it to life and other people valued the fact that as a piece of work this wasn't just going to be an assignment that kind of got shredded or put in the cupboard it was actually on a website that could be shared with others as well in a sort of repository of expertise so just concluding really these I think are the successes and issues of this small piece of work and that is that students digital skills were enhanced by this exercise they learned new things they'd never used my portfolio before they'd not embedded a YouTube clip into a page and that sort of thing very practical things that they learned students generally their awareness was raised of the assessment process but on the other side it's extremely time consuming I think Marnie gave some examples of a bigger project and how long it takes the iterations that it takes to get this right and for my students I was with them for 10 weeks so within a 10 week period we had to do what we could do and it could be developed I think further in the future this issue of ownership of assessment criteria is interesting I'm not convinced the students fully owned it even though they had helped to write it so that's something to consider as well and then my final query was about the digital skills and grades related to that and that students generally did better I think because we'd included the digital skills column I think if I took that away those students that had got married would probably have been left with a pass instead so I've put that in both success and issues do we assess digital skills alongside others in higher education can they equate can we find a way of making them match so these are my critical questions not for answering necessarily now but how far do students want to be producers and how do we prepare them to be equal partners in curriculum design does writing your own assessment criteria make you use it more effectively I had one example of a student who said I really liked using that assessment criteria to mark my peers work wish I'd used it when I was designing my own poster I thought yes, I wish you'd used it so there's a bit of a mismatch there is a disconnect there and then finally can we equate an account for digital literacy skills alongside traditional academic criteria in higher education then my details you'll be able to get to that page via the Bitly link and learn a bit more and there are 20 copies of the rubric down here for the first 20 lucky people who'd like to take them away today otherwise you can find it online thank you very much and if you'll stay up to see if there are any questions while the microphone runs around where's our microphone there is a question down here I do feel a bit starstruck up here actually I've seen my stuff on a massive screen the last time I was here Eddie Isard was on the stage hi I really enjoyed your presentation I particularly like this idea of including the recognition for the digital element and my question is your colleagues in the team, the course team what did they feel about that about recognising this things that is not the cognitive element because I tried to do something that didn't go down very well for me no, well I've done it in a sort of small cave where nobody will find me I'm being flippant but my immediate colleagues we're academic development and we're technology development so actually generally it's been quite well received within that circle in terms of broadening it out into the wider academic community at Warwick I'm not sure it's actually the rubric so it's building on that it's not actually stripping it away completely I mean you can't kind of for all kinds of reasons course regulation reasons just say I'll just do my own assessment criteria but I suspect if I were to try and introduce this rolling it out further it would be a bit of a slow burner but it doesn't mean it isn't worth trying I would like I would love for the people to test it actually it would be great so I mean feel free anybody who sees it would like to use it that's absolutely fine it's an open tool oh great, thank you let's be in touch then, thank you just about see you up there in the stratosphere hi, just today we're at University of Derby the digital literacy thing has sort of raised its head around a new program that I just validated one of the things we struggled with is that you've got your indicative content, your learning outcomes for the module and then your assessments and if you put digital literacy into the assessment you've now got a problem sometimes justifying it because your learning outcomes have to link to the overall assessment framework and learning outcomes that the professional bodies will be accrediting you for so how have you found doing that for this program is that a problem in this program for example, are you marking digital literacy because you feel like they're important but they're not in the module spec they are, in terms of the weight of the course it's very small it's not really as part of the module it's a 20 credit postgraduate award so it's very very small and not part of the bigger wider module so from that point of view we've had a bit more freedom and at the point of like all universities we have a kind of course approval process here and so at the point of doing that but actually to me it was integral to the course itself the course was about using technology for teaching and learning and that to me includes assessment so it was sort of integral to the the meta-curriculum if you like so it was small scale and I think that's what made it manageable and something I could do now is implement it more widely and more broadly I think would meet with questions later on that's what I'm interested in getting it into engineering subjects and it's very difficult to get some of these things in because you've got to then fit it in around everything else then what I found is when you work with students and you say okay we're going to do something on digital literacies and they're like well it's not in the module spec so why are we doing that? I mean we have to then question don't we module specification I think there's quite an appetite here at Warwick looking at digital literacies we have quite strong digital humanities group for example and there's quite an appetite for it I think it might take a few of us coming together to kind of propose a very sound rationale for this and think about at module design level and module approval level how we kind of pitch that to people who make decisions about what can be allowed and what can't so you can do that at university level but then you have the problem with the professional bodies so if you want accreditation to get them to agree and if you don't have accreditation and engineering degree you might as well not do the degree I think in my case small course and it didn't apply but you're right I think that raises a really important point I think we'll have to end the questions there but thank you very much and we'll move on to our next presentation which is Amber Thomas and Robert is somewhere around Robert O'Toole from Warwick that's particularly useful following on from Sarah there because we're going to talk a bit about some of the thinking that we've been doing about designing the platforms behind the learning which is happening in parallel with the thinking about how we design the learning so it's a bit about our story there I'm just going to move some props into position while you're at it he is, I will ignore him you can ignore him too so this is our story about implementing a more joined up approach and to just head back in time for a minute about four years ago we were just introducing Mahara and believe it or not we were just introducing a VLE for the first time so it was a point at Warwick where people were a bit unsure about what are the core tools which ones of these are pilots is this definitely here to stay can I rely on this and there was a need for greater clarity so we were discussing that we had these needs amongst the academic body and our fellow professional services as well to be clearer about what it was that was available of what was core and supported tools so we'd had various discussions and we knew that we needed to describe things better in terms of what the borders between different tools were about what they did and about what the benefits of them were but we also knew that in order to do that it couldn't be just my team we sit inside IT services we knew it couldn't just be that we had to work with colleagues from learning development centre colleagues from the library colleagues from various different parts of the university and most importantly with the academic departments and the learning technologist in academic departments so we knew that we needed a sort of framework for thinking about how we lay out those tools and how we deliver those tools and we knew that what we'd got then wasn't static so we needed to think about how these things would change and progress and think about innovation and at that point I'm going to go back to there so you can press that one so one of the things that Rob and I had been discussing and we've been talking about within the team is that essentially Warwick is very much mostly blended learning and our approach was to try to go from classroom based teaching to more of a blended aspect so we knew that it was something to do with thinking through how people used tools both face to face in the classroom and online and over to you Rob Over to me, okay, so next slide so at the same time for a few years running up to that I've been doing a PhD looking at design thinking design methodologies organisational designally approaches and how they might work or not work how they might be synergies with how things actually happen in higher education specifically the University of Warwick which is when I started doing that PhD the first thing I realised was just how immensely complex this environment is and how complex the design innovation of the spread of innovation ideas and enhancement ideas is in this environment so I've got a slide here that just gives a summary of what we think design Robert out of my PhD design thinking means and how it fits into education and one of the big messages which really fits really nicely with the experience we had in academic technology is don't just address isolated problems so we have lots of isolated problems lots of reactive stuff all over the place in this very rich complex world and us being academic technology in IT services we're running around for many years addressing these lots of isolated problems in a lot particularly joined up way so find solutions and build enduring design capability together rather than just focusing upon solving problems we actually wanted to build and we are building more robust more university wide design capability so people collaborations across the university can get together and prioritise work out what the challenges are what the ambitions are and join together to create solutions together so and that design thinking that design capability that we trying to encourage across the whole university is a continual process it's not a one-off it's not something that just starts up for a specific project to meet a specific challenge and then disappears we want it to be there all the time and design thinking if you come across that term so there's plenty of literature on it it's you looking at some of the literature you might think design thinking is a method for doing design projects it's not it's much bigger than that it's a continual organisation wide process of finding and creating designs that and this is straight from my PhD practices behaviours designs using technology or not that fit, stick, spread and grow so they fit with people's needs, their capabilities where they want to be their ideas, their culture their value, that's really important they stick for a reasonable length of time to justify the effort that goes into creating them they spread and the wider and the greater the spread we can achieve for the effort that we put in in an organisation like this a complex organisation the better and they feed back into the design capability so the practices that are adopted help people to go on further reflecting, finding design ideas refining design challenges finding solutions and working together and help to create the community of design capability that we're trying to create so and we're using a wide variety of design innovation methods we've got one of them curled up there happily we'll see if we can incur it in a minute and hopefully it won't spring back and rolled now I'm going to whiz through the design innovation enhancement management model that underpins this so this is typically what we see are happening in the university in a very unconnected way so we have people let's get this point in the right way and people maybe they have a practice that they're continuing that they've adopted often just because that's what has always happened of the strategies used in achieving the learning outcomes during the improving stage the learner considers the experience and decides how they will alter the way they learn in the future so on to the focus of the study the study focus is on how students develop skills and confidence to learn autonomously it tries to bring together existing practices and approaches with the intention of providing students with opportunities to develop skills the study uses learning design principles as I said before my mapping technology and the apprenticeship model to teaching so looking at traditional learning design it looks at how teachers create learning activities for students and how teachers share those activities among their community the change with what I'm doing is that I'm looking at what students how students design their learning so in the traditional approach to learning design the power of learning design sits with the teacher what that means is that the students are rarely involved in planning how they learn they're told what their learning goals are what to read, what to discuss what to reflect on and how their learning is assessed so as I said learning design is defined in this study as an approach used by the learner to plan their learning this means the power of learning design moves away from the teacher to the learner and I think the shift of power is vital to help students become more autonomous allowing students to design their own learning in modules does not have to mean that traditional teaching methods are redundant the student can build in lectures seminars, workshops into their learning designs as long as they create their designs before the formal teaching takes place this means that they have to think through the learning design process before the teacher introduces the topic as they learn they can use the learning design map to record them and monitor and evaluate the process of learning if teachers continue to create learning designs of how they would learn a topic these can be used by students to check their designs again so what this intervention is hoping to do is to train students to ask the questions they need to ask about their learning and to have conversations with themselves about their learning some questions students could use are on the screen and these are questions such as what are my learning goals what resources will I use what do I already know about the topic do I have any knowledge gaps how will I fill these knowledge gaps how will I know that I have covered important points how can I improve my learning process so this is a view of the design map just a small section of it and it's in a way how we hope to help students externalise these internal conversations so externalisation is necessary so that learning designs can be compared with others and improved most students are familiar with spider diagrams these are usually used when we rise in exams in schools spider diagrams created using parent paper to map out a subject the same principles can be used to help students map out their learning designs in the context of this study the metacognitive learning design map attempts to externalise the metacognitive processes taking place in the mind using the mind map Shaman Bruce 2011 says that metacognitive processes are involved in regulating the learners thought processes that control the learning strategies they use on to the technology so as I said before mind maps mapping technologies is to help students create their learning designs a template is produced using mind mapping software and this will aid learners to externalise their learning processes there are many free mind mapping software available to learners some examples are free mind mind map and free plane there are also various mind mapping apps I am planning to use free plane which you may know about it allows students to analyse the information contained in the maps and this is going to be useful with my research study will provide participants with the free plane software and instructions on how to use it the mind map file will be shared with the researcher and images of the learning design map will be shared with the class in this approach mind mapping technology is helping learners to share and communicate their learning designs by sharing they are working as a collective pooling together resources and knowledge of a topic this approach may also help students recognise the value of sharing and talking about how they learn so how do we enable students to create their own learning designs the template is created by the researcher using existing knowledge on the learning process it covers areas of planning, monitoring, evaluating and improving learning it has prompts to make the learners think about the different aspects of their learning process the questions of the map are shown on this slide I will read out some of the prompts because the screen is probably not big enough why do I need to learn this topic how does this topic link with other topics in my module how is this topic relevant to the program I'm studying how could this topic be useful to me in my future career new knowledge links what do I know about the topic that I didn't know before how do new things I have learned link to what I already know what did I find interesting about the topic what aspects of the learning process worked what aspects of the learning process didn't work what would I change next time so initially I designed the template and then it was reviewed using a panel of experts the panel was invited to review the template using an instrument to reduce fullness of each section and prompt the comments from the experts were analysed and used to edit the template experts on the panel included module and program leaders and professors professional support staff graduate teaching assistants and undergraduate students all together there were 14 members in the panel so moving on to introducing the learning maps to students the idea behind the intervention is to get teachers who are the expert learners to share how they would learn the topic with student novices the sharing is enabled by using learning design the teacher externalises the way they would learn the topic using a mind map of their learning design the learning design is shared with students teachers demonstrating the processes they use to create their learning design then students are asked to adjust the maps created by by their teachers to highlight their own individual knowledge gaps and as they learn the topic they use the learning design to record how they would monitor, evaluate and improve their learning module leaders provide support during this step and in the final step the teacher starts to fade into the background and the student creates their learning design independently so just a overview of the research design I'm using a mixed methods approach the diagram shows the exploratory short design used to collect the data first student experiences explored using focus groups and then the data from the focus groups are used to create questionnaires which are administered to the module population so the study is using three different topics within a module the first topic is used to collect baseline data before the intervention the second is where the metacognitive learning design map is introduced to students the third is where students are asked to create their learning designs independently and the module leader data is collected using exploratory sequential design so all that means is first of all question is sent out to module leaders and then both are analysed and participants are chosen for interviews so something that I'm planning to do is once the data is analysed from the interviews a module will be chosen to implement the intervention so I'm just going to talk to you just give you a brief glimpse at some of the initial data coming in from module leaders so in the pilot study 25 module leaders were sent questionnaires 15 responses were collected overall the module leaders seem to think that students see them as the key to achieving the required grades a source of knowledge that they can tap into and select and organise of their learning materials and as far as staff expectations go there is general consensus that the main expectations teachers have of students is for students to take responsibility for their learning so hopefully this intervention will be welcomed module leaders clearly believe that the responsibility that the students learn evaluate and improve the way they learn lies with the students but module leaders also seem to believe that their role is to encourage, motivate and inspire students to take responsibility for their own learning so the next stage is introducing the map to students so thank you you have to do that thing of waiting for the microphone to run around the room the microphone is attached to our help any questions or comments can I ask you about the mind map tools that you use am I right in thinking students have yet to try this tool or some students mind mapping is used at a university currently they are familiar with mind mapping but not on this scale and not that they have been collected from those maps I was just interested in how students get on with mind mapping if it's something that's been introduced institutionally because I've got students who've used it really well and successfully and I know that I came to mind mapping a bit late really and I quite like it now as a tool but intuitively I didn't find it a natural thing to do to do that have students been sort of coached in how to use that tool I think first year students because of the spider biogram aspect tend to like mind mapping it's quite easy to take notes using mind mapping mind maps as well so I don't think the technology really kind of forces problems for students I think more of the academic staff who will struggle with mind maps thank you thank you again it was a really interesting session so thank you