 Fy enw'r cwestiynau ar y cyfnod, Ben yn ychydig i'n gwybod ei twyddedd yn ymgyrch chi'n ddweud. Rwy'n gweithio gweithio 5 gwestiynau a wnaeth yw'n gweithio 15 ymgyrch, sy'n gweithio gweithio 5 gwestiynau a'r gweithio'n gweithio'n gweithio. Felly, rydw i'n cael ei gwestiynau. Roedd yn cymryddsodd y cwestiynau gan fy ngwrs. Felly, mae'n rhaid datblygu dda chi i chi'n gweithio'n gweithio. Felly, o bobl gallwn amddangos, yn ymlaen, ac mae Dominic i'r Su, yn cofnod o'r Llywodraeth Hwychol Gdyddolol Cysylltiddol Llywodraeth. Ddod yw'r Llywodraeth i'n ddod yn unedig. Ddod yn ymlaen i'r Llywodraeth. Mae'n gydag ymlaen o'r gweithio'n gwybod. gyda'r ysgolwyddon ni'n gweithio'n gweithio gyda'i hefyd i wneud eich bod hyn hynny'n bwysig i'r hyffredigau. Rwy'n ddim yn meddwl i ni'n gweithio i gweithio eu modd yma'r hyn. Oherwydd, y dyweddeg yn cael ei chyf hypnottyf ar gyfnog, rydym ni'n unrhyw meddwl eich rhaen i'r litiau ei ei ddifigodol, sy'n gwybodaeth wedi bod wedi gweld foodsiwch arna Ieddaf y fath wedi amser. Cwm yw'r Llyfrgellol Cysylltwyr, a all yr raiolau a llwyddiadol yn eu cyrddysgu'r unig. Mae'r raiolau, mae'r cyfrwyr yn mynd i'r Llyfrgellol Llyfrgellol Cysylltwyr yn y hoffi, ac mae'r cwmhwml yn amlwgol oherwydd mae'n rhoi ddwylliant y dweud i'r unig yna y byddai'n ei wneud. Mae'n dweud i dweud i gael y byddai'n unig. Mae'r ddweud i ddwyng, mae'r ddwyng yn ddwyng yn y hoffi, o'r lleidio'r unig yw'n ddiwedd. A'n fyddi, dwi'n fyddi, mae'n ddweud y ddweud am y cwm, Rydym yn wych yn iawn i'r ddaf yn drosio trafwydig. Roedd yna'n ddweud i'r ddweud o gyda'r llwyffydd o'r gweithio. Yn ysgwmwr yma, yna dwi'n ddigonwyd yn ei ddaf, ac mae'n ddweud amdano, ac mae'n ddwy'r lleidio, mae'r llwyffydd yn ddweud i'r llwyffydd a'n ddweud i'r llwyffydd. Mae hyn yn dweud hynny wedi gydag y tro am fylinellion Gweithio i wych yn gweithio'r cod Gweithio'r cod yn gweithio'r cod Gweithio'r cod yn gweithio'r gweithio'r gwydech Onl mae'r gwelio sy'n gweld bod iawn i'r ysgrifennu Dym ni'n gweithio'r i'n gweithio Mae'r ysgrifennu hefyd slidef yn gweithio'r padd y dyma Mae'n gweld cyfrydypa iawn i gweithio. Felly mae'n cyfath o'r rhaid yrhaith Mae yw meddwl ni dda 여기서 ac wedi'i rhainwys r fadesol. Those are blood, some are gas, The weapon of choice was a revolver with a full metal jacket bullet for testicular ablation. What about this is CTC, CTC is a really nice way of using x-rays to get lots of nice images in any format you imagine, so you might wonder what this is, well again this is the gentleman who was a little ill advised with his comments, he got home fed, I don't think much about tea, now his wife was really miff, she'd worked really hard that day, a ddyna'r ysgol sefyddi'r cynllun. Dyna, mae'r ysgol wedi'i gweld i'u cyfrifio i ymddangos, ac mae'r ysgol yn gwneud i amser i'r hynny yn hynny. Nid yw'r cyfrifio'r yr ysgol yn gallu i'r cyfrifio'r lyfr yn gyfnodol, alewch angen i'r cyfrifio arall, a'r cyfrifio'r cyfrifio ar y cyfrifio ar y cynllun, roedd hynny yn ei gydig i wneud amser, mae'n amser yn ystod o'r cyfrifio ar y cynllun. We're not satisfied with just pretty pictures. We're quite good at putting needles into places. Certainly one of my favourite activities. This is an abscess is deep inside. You wouldn't get this easily. And there's the needle from which I think we've got about half a pint of pus out. So this is the kind of thing I do for my job. And you can see why the med students need to understand this. Sadly, teaching of undergraduates is apatrocious. Certainly pre 2012 all we had in Leeds was a thing called the Radiology Breakfast Club. That was a short series of lectures for final years. Did quite a bit of anatomy teaching for the first years. But the rest was a random selection of lectures set up primarily for the other departments in the medical school. No structure, no recognition, complete waste of time. This was recognised by the Royal College of Radiologists in 2012 who came up with their own undergraduate curriculum. Now this has got excellent points in it. It lists all the things we think a junior doctor should know. Should recognise and should act upon quickly. There's nothing too clever in this and it's just basics. So in Leeds we decided it was time for an undergraduate lead and I didn't step aside fast enough so that was my job. And the first thing I did was cancel all of those lectures. Now that, if you can imagine, created absolute mayhem. And these have been going on for years. Let's ruffle the loss of feathers. What it did give me is a lot of leverage afterwards to do what I wanted. I decided it was time to completely overhaul everything. And one of the nice things is I got to meet Sue. So everything from now on is courtesy of Sue's hard work and my irritating comments and emails on a regular basis. She'd already started work on this. This is on the time website and this is really nice website which gives quite a lot of information for our undergraduates. It shows where they are, it shows something about careers. And we've got a nice little 3D interactive tour of the Radiology Academy. This is where we work and report. And it's usually a lot busier than that, I have to say. And there's usually a lot of coffee cups, debris, you name it. This was cleaned up especially for you. There's the online seminar. So the breakfast club I mentioned, we're putting all of the lectures on there so that the students can go away and look at these at leisure. There's five there at the moment. There's another five that Sue knows about and there's another five she doesn't know about yet. And that keeps her very busy. But the problem with these, these are old-fashioned ways of teaching. It's very tried and tested but it's not very imaginative. It doesn't really let us cover the length and breadth of the topics we need or the curriculum. So what we really wanted was some case studies. Nothing hysteric, dead simple. And we've got the fast food students now. They like things that are quick, easy to digest. They've got to go on smartphones and tablets. So we decided it needs to fit in with that. I did get the university by into this and this is now the sole source of exam cases. So if the students can be bothered to go through the case library, these are the only cases they will find in their exams. In other words, they look, they will pass. It's as simple as that. For those of you worrying about anonymity, all patients who come through LTHT, there's signs everywhere stating that images are used for teaching purposes. This falls within national guidelines. I'm allowed to use these as long as you can't identify the patient. So if it's something unusual, I have to get their written permission. If it's a standard thing, that's okay. So although we've got the time website, this links into the VLE. And this is how we've presented it. Now, this is exactly the same as the radiology curriculum. It's in the same subsets, but you'll see there's no obvious answers here. So they have to actually trawl through the cases. We don't want them to kind of pick out what they think they would like to see. We'd like them to go through the lot. And this is the kind of image you'll get. So here we go. Simple image. Interesting case. I've lightly got hit by a cricket ball. He's unconscious in the ED. And one more slide that gives the answer. It is that simple. The idea is they can do these within a minute. This, by the way, is a life-threatening emergency. He will be dead in three hours if he does not get operated on. So this is really important. Now, if you go on to the internet, you will find countless websites telling you they've got the best selection of medical imaging known to man. You should use their website. Theirs is the best. Don't go anywhere else. And possibly pay a large sum of money to use it along the way. None of them actually have any proof of use or validity. So we thought it would be quite nice to actually see how it's used, when it's used, see if we could actually correlate between exam use and success, and some feedback for the year two so that if you've got a candidate who's failed, we can say they've not accessed this. So this is maybe something for them to look at. And the first big hurdle we got was ethical approval. Despite a very simple ethics application that went through without any trouble eventually, the fact that I am not a full-time university employee created mayhem. There was lots of arguing, bickering. There was revocation of access rights. You name it. We had it just because I don't work full-time there. So this is how we access our data. And we've had to use the VLE. We have to use the 360 tracking. This is not ideal. And it would be better if there's another system that gives us the student ID. The data I get is this. And this is a bit of a pain because it comes up with Christian name surname in one thing. So you copy and paste that, then goes into an Excel spreadsheet. We have to laboriously add all Christian names individually to everyone. It doesn't allow for marriages. It is a pain. I absolutely hate it, so you would not believe how long I've spent doing this. The great thing with the 360 is it does give us the whole year. So what we've got in the case library, well, this for the year 344, then they're allowed to access some more in year four. That gives them 134. The fifth years get the lot. The cases were excluded of those we've added in after the exams because this has been our pilot year and we are still just about finished it now. So it's not fair to look at cases they couldn't see. UCs now, we didn't promote this at all. This was by word of mouth only, a little bit by the back door. So the year three is a little bit of use. Year four is quite a lot. Year five, we're absolutely delighted because we've now had an overall 18,000 hits with no effort to promote this. We are looking to increase this by tenfold for next year. So this is great starting for the data. What about a little bit more drilling down? Well, the year fives on average, 82 views per case. That's about a third of the year. That's all right. That's a good start. You can see the lower years use it less. Case of the week has been a bit of a disaster at the moment, but we've not promoted it. So we think that there's some work to be had with that. These are important cases that fit in with their curriculum at the time in the year. So we're going to push that a little bit harder next year and see whether it's worth carrying on with. This is interesting. So education is to tell you students should study uniformly throughout the year and to gradually gain their knowledge. I'm glad to see they still do the same as I did when I was at medical school. Half of all the views were in the year of the exams. Sorry, the months of the exams even. So they're cramming, as they always did. The last one won't mean anything to you. It was the maximum views of the NG2. This is the number one serious untoward incident in LTHT. This is a big deal to our trust. Misplacing to these tubes can be fatal. So the fact they've looked this up is an absolute bonus as far as we're concerned. Basically, is that tube safe? There's the answer. The tube was safe. But what the clever students picked eventually is the endotracheal tube is in the right lung. That's left there. The left lung will collapse. So it's quite a cute case in that they get the obvious, but there's an extra learning point. Resource fatigue. Interesting to see. You only use the first half, the second half, half as much as the first half. In other words, they start at the top of the cases and they get bored. They are targeting certain areas. I'm a musculoskeletal radiologist, so I'm quite smug that we're quite high up. I have spent a lot of time, sorry, winding up my pediatric colleagues. There we go. Who've had rather little views. Numbers buys. This is really interesting. They paid one of the 50 years to print off every single case within the library. He spent a whole day and that's all he did. What a waste of time. They've all supposed to be IT savvy with the phones. The thing with this, of course, is bits of paper you can shuffle through really quickly and it's a kind of a nice feeling when you finish them all. So we need to think about that next year, but it is quite funny. Exam correlation with lights now, how they do. So there's an obvious reason for this. And this is a real pain. This is about learning. We still can't get permission from university for me to look at the exam results. They will not let me look yet. I will get there. I'm not going to give up, but it's amazing how much mistrust there is somebody who's not a year tutor. And this goes down to this. What they're quite like is me to put all these nice pretty pictures. They've got a great resource, but when I start to get a bit dangerous and get some useful information that might challenge them, then it gets a little bit techy. It's a bit like this poor dog who's got really confused with the data put in front of him. So learning points for us. Well, we did get university buying, so that was great. The VLA, I'm afraid, and I know they're here, is not my favorite. Now, that might be we've got an old-fashioned no bells and whistles one, but I still don't like it. The cases, data protection issues, I had to go to national bodies to check this. Student Anonimity is a big deal, particularly the way we collect the data. The biggest thing is me being a non-university member of staff. I've got an honorary appointment, but I'm not there all the time. Advertisd into students we're going to do next year, and the stats have absolutely killed me. I've got three minutes to go up, I'm told. This is what I felt like for a large amount of this process. However, the great thing is radiology is now recognized within the medical property. Our cases are in their curriculum, which is great. They're starting to use it, and we've got massive opportunity to increase this. The data comparison next year is going to be brilliant because we've got smallish, but nice uses this year, and we're looking to really go through the roof next year. And the unofficial feedback for students is they absolutely love this. Interestingly, we can't even get ethics approval for a questionnaire. Just a bog standard one, of course, trouble. And the bottom line is we've still got lots to do. Thank you very much. Sue, you want to join me? Any questions for Dominic and Sue? Thank you for saving me from having to do that. Sharon Slade from the university. I'm wondering whether you've got plans possibly difficult given the issues that you have with getting access to the data. But if you've identified which of the cases are the crucial ones for students to engage with and if your plan later might be to track and to intervene with those students who don't pick up the key issues. Isn't it? It's a really interesting question that I actually put to the head of the dean of the medical and actually they're quite keen that we don't pick on people. It's an interesting idea and I absolutely agree. One of the things we'd like to do is then to say, we've got somebody's failed. Have they accessed it? So that would be good. And you're quite right. There are also some cases that we consider critical. Now, I am going to address that because we've just this year got three of the trainees who helped me with this. I've now got permission for them to be on the exam setting boards. So in other words, I've indirectly got myself the in on the actual exam set so I can actually suggest what my own might like to see in them. I guess that kind of answers your question. Any more questions? Can I just ask in terms of the broadest of the medical school? Are there any plans of using the sort of the methods that you've used here? To extend that sort of library of cases and the way that students are using visual images in formative assessments? Not as well, not that I know of. I think the answer to that is the only one I know of is the renal medicine team are starting to use the kind of interactive or the online resources. No, we're pretty well the second one in there and we're far better than them. I'm going to say that obviously. And I guess they've been waiting to see how this works. So the data I've now got with engagement is going to be really powerful to go back and say, look, this is what you've got. This is the use. Now we can go here and actually, why is nobody else doing the same? But yeah, it's a really attractive thing to take on. But there's a lot of work. And what I've not pointed out there is I have no time allowed this at all. I don't get paid at all. So soon as I do this all in my spare time and that's one of the stops to actually getting this going forward. I would love to have time just to do this. It's a lot more fun. Any other questions? You can shout. One of the things that it could do is to give it a look at its type of ability. Yeah, look better than but more than that. Fantastic. Thank you very much indeed. So the next presenter is Gronja Hamilton. Gronja's presenting alone but the co-author is Vicky. And the presentation is Exploring Learner and then Peploria Perceptions of Open Badges in supporting young and employed people into education, employment and training. Thank you. OK, hi everyone. So I'm going to be looking at employer perceptions of open badges and also young, unemployed people's perceptions of open badges as well and seeing how they how we can support them in that transition whether it's into employment, education or training. So we'll take a look at the skills gap that exists at the moment. Have a look at open badges and take a look at a pilot that City and Guilds run with Youth Focus Northeast a community group in Newcastle supporting young people into employment and have a look at the findings of those. So there is a skills gap in the UK apparently. We're going to have 14 million skills shortages by 2020. The UK Commission of Employment and Skills found that skills shortages are becoming an increasing challenge. Six percent of employers surveyed had at least one skills shortage which is a 2% increase since 2013. And in a report by CBI in Pearson last year 55% of employers that were surveyed said that they had skills shortages and didn't feel that there were going to be the necessary skills for highly skilled jobs in the future. So we seem to have a challenge in terms of skills but another thing that has been highlighted again in a report by Pearson is that young people can also find it difficult to articulate their skills. So when they're coming to going into applying for jobs they're not able to present their skills adequately so that employers are then able to feel confident in recruiting the right person. That same report find that the most important factors that employers look for are aptitudes, attitudes and they are actually well ahead of what they're looking at in terms of formal qualifications. So we know that employers are looking for those sort of character attributes or those soft skills that will help them to be job-ready as well as their formal qualifications. So we need a way to bring together that understanding of a person's wider achievements, wider attributes as well as their What are open back? Sorry, I'll just stop that for a second as well as their ability to express and articulate their skills. Can I just check if everyone in the room knows what open badges are? Is there anyone who doesn't who would like to who would like to get a quick overview? Yeah, okay. I'll just play a very quick very quick video. I think I can start this again. This is about a minute. What are open badges? Open badges are image files with metadata hard-coded into them that can be shared across the web. They provide a flexible way to recognise and reward learning. Organisations create badges on the Open Badge Academy for their learners to discover. Learners complete tasks within a badge and submit evidence at their work. Evidence is viewed, verified and badges are awarded. Awarded badges are displayed on an Open Badge Academy profile page. Learners can share a link to their Open Badge Academy profile or individual badges to gain endorsements for their work. The evidence contained within a badge helps bring learning to life because when badges are shared all the metadata contained within it can be viewed. The verified evidence and endorsements demonstrate the skills required to earn the badge. Organisations contract impact and progress of their programmes through Open Badges. Learners can use their badges to build their confidence and unlock opportunities in education and employment. Here are some of the organisations around the world who currently use Open Badges. OK, so that's a quick introduction to Open Badges. It's still a relatively new concept. It was initiated during an international competition in 2011. And in 2013 this reducing the skills gap pilot took place. So it was actually quite early on still within the whole understanding and awareness of Open Badges. So this was a young person-led training programme that was to look at developing employability skills via badges and work placements. 150 employers and 237 young people contributed to the focus groups and the research that informed the pilot. And 26 young people participated in the pilot. There have been a further 80 since then. The information you've just seen in Open Badges there was in the context of the Open Badge Academy which was one of the badge platforms used. And young people were asked about their confidence in their skills and ability to talk to employers about their skills before the pilot took place and then after the programme to see if they felt more confident. They also undertook a two week work placement. So some of the employers that were involved were Marks and Spencer's the NHS, Marriott Hotels, University of Newcastle Business School. And there were various focus groups and online questionnaire and also a presentation to employers because none of the employers bar one knew about Open Badges prior to undertaking this programme. So the research questions that City and Guild were interested in were how do employers value Open Badges compared to traditional credentials? What are the opportunities and challenges of using Open Badges to recognise skills around employability? What kind of evidence makes a difference to employers and to the young people in getting into employment? And City and Guild was obviously interested in what their role would be in this process. So the pilot the feedback from employers was that there were five key soft skills they were interested in which were communication teamwork, time management, professional attitudes in the workplace and confidence. So badges were created for those. There were skilled modules around those soft skills and they gained a badge at the end of those. And there were three additional badges which were interview experienced, work experienced and young trainer. So the key themes that came through from the young people were that they found that going through this process gaining badges helped them with the articulation of skills. They wanted to use them on their CVs but were very unclear as to how they could share them during the recruitment process. They were also concerned that employers wouldn't know what badges were. They were unsure how to display. One of the participants didn't know what LinkedIn was. So there was a wider issue there with having professional profile and understanding how to present themselves. There were mixed opinions over whether they thought badges would be popular. Over half believed they would. The rest didn't know. They liked the idea of having evidence to back up statements. So particularly sort of competency based interviews. They thought that would be useful. Again, even without presenting a badge, they liked the idea of going through a badge helped them to understand they had skills and to articulate those skills. They did have concerns over levels and effort so differentiating. They wanted to understand that if they'd been given a badge and someone else had been given a badge that they'd put the same amount of effort in but also were interested in the level of the work they had put in or their level of understanding being represented as well. They thought badges could be useful because of the portability. You can have your badge profile on a tablet on a phone. So they thought they could take that into interviews and some of the employers were interested in that. Although others commented that there wouldn't be enough time in an interview to view badges. But overall, they found that the badges gave them a boost to their confidence and they were motivated by the thought of having evidence of their learning. So the badges didn't motivate them to learn but they were motivated by the thought of having evidence of their learning. And in terms of city and guilds, they felt that that sort of known brand would give an element of trust to a badge. So if an employer knew the badge had been endorsed by or validated by city and guilds that that would build trust. In terms of the employers, they were concerned with and interested in the calibration of badges. So you get a effective communicator badge from one issuer. How do you know that that means the same thing or what are the comparisons? What's the parity between that and that same type of badge issued from another issuer? Quality, they want to know who's assessed it? What's the rigor behind the assessment? What is the assessor's ability to make those evaluative judgments? However, they did like the idea of having endorsement in a badge. So with the OpenBadge Academy badge consumers or anyone viewing the badge evidence can leave an endorsement on that evidence. And they thought that was useful because it was like a reference upfront. And so they could not only see the evidence. They could see other people's endorsement off that too. There was consensus that they wanted evidence of applied skills and they could get the theory elsewhere. They wanted to see things like videos or photos or those kinds of things of applied skills. They felt they feel soft skills are vital to be successful in the workplace. So we do need something to capture those. Again, like the learners were interested in differentiation. They do want to use badges for recruitment where every one of them was keen on that but are concerned about how it will fit into the recruitment process and if that would add to the time needed for that. And also like the young people they thought that endorsement could be could be really useful for adding credibility. And so that could be endorsement off the badge itself. And again that's something you'll be able to do so you can have a validated badge so it could be a badge could be validated by city and guilds for example on that includes a marker and for on the badge visuals so that someone can see that straight away. So the opportunities and challenges for the participants just to sum these up and we can see here opportunity skills articulation boosted confidence and employers don't know what badges are those still an issue and wish they want to have levels and I'll just pick out this point about known issuers that is something that the employers also picked out that if it's a if it's a well-recognised or well-established organisation they would trust the badge more and this trains with some work that digital me are doing with the open badge network which is a pan European network influencing policy promoting the uptake of of badges in Europe and I've recently been working on a quality paper one of the outputs from that project is is looking at quality and open badges and so a survey asking what earners would value in terms of a badge again this came through we want a badge from a well-established brand well that's not always possible so one of the one of the other things that we've been developing with the open badge network is is work on badges and territories so how to promote uptake of badges within a geographical area and and we know from case studies that that networks can really support some of these connections and people understanding so earners understanding who the issuer is valuing them the consumers so potential employers having those relationships as well with the with the issuer understanding what's gone into their badge that gives some more more sense of security and trust in where the badges come from and how it has been awarded so there are there are ways that that that can be addressed and just to sum up the challenges for the employers they want and opportunities of opportunities enhance bland series that was terminology used by one of the employers evidence of soft skills evidence of applied skills of what they want they're looking for but the challenges are they don't know what badges are earner authentication trust they may assure that at the moment they're not easily built into the recruitment process and some of the wishes you can see there are similar to what the the learners were also looking for but perhaps just pull out quality assurance there that that was a key concern as well that they felt that badges have gone through this and so just in terms of the recruitment process these are some of the challenges that were really highlighted that they want badge evidence but don't know if they'll have time to view it online job application systems go through automatic screening processes and at the moment those HR systems are unable to mine badge metadata but there is work underway at the moment with the IMS open badges extensions for education group for example that are do extensions to the open badge standard that to provide machine readable data around levels and quality markers things like that they're also as I've mentioned that opportunities of having sort of verified issuer status which some systems badge systems allow and you can have a quality assured marker which is certainly available in the OVA and you know maybe we need to think about sort of flipped interviews we flip the classroom can we flip the interview so that we're actually looking at evidence we're looking at endorsements references upfront before we invite candidates in and then and then just ask them to present themselves just have that conversation within the interview itself so just to finish off and the recommendations that came through from the pilot were that we need guidance for employers and we also need guidance for badge owners about how to display and share their badges that they're looking for practical evidence we perhaps also need to be providing support more widely about how to assess evidence there's perhaps a need for badge standards and levels of badges should certainly be explored further and there are tensions there between you know having everything badges then becoming a bureaucratic process which the the sort of flexibility of badges was supposed to guide against so there's there's probably a balance to be reached there and we also need to have further consideration I think of the recruitment process and how we build that's that's me and so happy to take any questions of anyone has them that's when I put the the back there Thank you My name is Mark Glyn from Dublin City University Thank you for your presentation I totally enjoyed it just with regards to the recommendation on the practical evidence you may be aware about a gentleman called Alex Ambrose in Notre Dame University has interface where not only can you see the criteria that the person's got the badge for but it also links directly to the student's e portfolio system which shows them their evidence for actually getting that badge so he may be worth the contact Alex Ambrose in Notre Dame University Yeah thanks very much for that it's actually something as well that is a feature of the systems that we used for this pilot that students can actually upload the evidence directly to that system and actually the findings from the quality work that was that was quite a clear a request that evidence is very easily visible within the context of the badge so the criteria the tasks whatever they are and actually from that they were they were interested in evidence being hosted in the same system to guard against broken links so if we are using external links we need to think about just the longevity of that which you know which is an on going discussion around portfolios anyway thanks Westam Australia um one thing I've been thinking about though with the portfolio and and with badges though that if we're saying that it's going to be used for employability it would have to be driven by the employers then like you're saying it doesn't get into the recruitment process so do you have any thoughts to do see that there would be some openness to that that changing Yeah I think employers are um we we certainly worked with a lot of employers and and that's increasing they can see badges being useful for that but not only that also for things like their corporate social responsibility programmes and because they see badges as a way of having communication with em with their potential customers as well so em and you know some of the big em some of the big organisations for O2 for example we've worked with to look at talent pipeline but also recruitment and retainment processes as well so they can see the opportunities for not only using badges to to gain employment but also they can earner controlled em they've seen that opportunity for the continual professional development as well and that that's going to travel with the earner because they have control and manage their their badge portfolio hi I'm Dilecia Finando Euniceio'r partner chef just wanted to see em if if you can tell em what sort of assessments did you use to assess the skills for of the students and did it help with the articulation of their skills when they've been the jobs afterwards? Yeah so I actually wasn't involved in the pilot so I don't know the ins and outs of how it was assessed em I know that it was young trainers em who were had also gone through the program or being supported in a similar em similar context who did the assessing and I think that's why some of the the considerations around someone's ability to assess came up so while that could be useful in what was trying to be achieved and it was obviously a great learning experience for those young trainers em what city and guilds find when they were overseeing that is that there needed to be a bit more of a perhaps bit more of a systematic process in place to just ensure that that's you know that there was the parity et cetera em so em but what certainly came out from the employer's perspective was that they wanted and they actually stated we want to see videos, photos et cetera so they want to see those applied skills em and often with badges that we're still talking or writing about our experiences but if we can also in a in a authentic context em show the application of the skills then that's what the employers were saying would be really interesting to them okay thank you very much thank you for you the next presentation is from Neil Witch University of Plymouth on learning analytics what do stakeholders really think? much em so learning analytics most entertaining subject for the morning I'll try and give a slightly different twist em my institution like many institutions em is contacted by software suppliers em and our vice-chancellor at the time David Coslett em as I was saying people are getting these phone calls who from companies offering of the solution they're not going to cost me what thus you know 200, 300,000 pounds alright you know is it worth it and you know I've been involved in in this sort of work for a number of years and if we're going to go if you want an institutional approach to learning analytics we really need to find out what it is an institution we want to do and that means talking to people so we were really fortunate that em with David Coslett our interim vice-chancellor and Paulie Neil our pre-vice-chancellor teaching and learning we were able to get some funding last year from higher education academy under their strategic excellence initiative and what that means is you actually get a vice-chancellor running a project and that I will tell you opens doors right across the institution the first speaker was talking about ethical approval if you want ethical approval put the vice-chancellor's name on it it goes straight through very, very quickly I'm not going to spend a lot of time talking about learning analytics there's some very, very brief definitions here about the sort of the footprint our students leave behind it's getting data from a number of different systems what it is it's about using that information for a number of key purposes so the examples there about tracking student engagement identifying students at risk these are all you know worthy ambitions to do with learning analytics and of course we look at ways of actually how we display this information how we get information to staff and students so there's a number of ways of doing that but I'm not really going to focus on the technology and what we're going to do with it in terms of data I'm really, really interested on the edge there information about the individual and of course we can take that information from an individual group it into a module group that into program, schools, faculties and end up with a whole load of academic analytics big, big challenge ahead for us but at the moment I'm focusing just on the individual and I started looking around and talking to students does anybody here drop into Ychhyac when their students are back? Try it's a really good way of finding out what's happening to your institution this doesn't come out very well I'm quite glad that it doesn't because there's lots of swear words which I hadn't realized so I was going to be live streamed so I apologise for anybody with sensitive nature but Ychhyac is really good now we found that right in a whole load of yaks about students issues to do with halls of residences the upcoming EU vote and general angst and day-to-day traumas of being a student there was this little innocuous comment there about how do we work out our overall grades? That's interesting so I started following this I've got about 16 or 20 of these screenshots as it progresses and what we've got is probably about 15 different students all giving their own opinions on how an overall final grade is calculated and they're all wrong this is information that is available through student handbooks it's on university websites it's buried and I'm thinking if we really want to get this right we need to expose this to our students this is obviously causing students some issue so we're thinking ok, what else do we actually need to give to our students? and the way we can actually find that out is actually talking to them so we put the work together from the Funded Through the Higher Education Academy to look at learning analytics through lots of different lenses so we met with lots and lots of students lots of staff staff, professional support staff and academic staff but I also thought it would be really interesting to talk to some senior leaders we've got deans with heads of school Pro Vice-Chancellars Deputy Vice-Chancellars and I also thought what I'd really like to know is learning analytics is those issues which I'll come on to in a moment but if it all goes horribly wrong who gets the blame? who could go to prison? Board of Governors so we're able to actually speak to our Board of Governors and it's been a really interesting journey actually talking to all these different people bringing some of them together having those conversations to see, okay if we're really going to go down and take learning analytics seriously what do we need to think about what are the issues and how do we actually make it sustainable across the institution so the challenges identified these are quite well documented across the Board there are a whole load of ethical, legal data, technical, policy, process challenges hey, they're only, they're challenges alright, they're problems we can solve them so I'm going to be quite flippant we know those challenges there I'm not too worried about them at the moment the biggest problem I see though is that if we do find a way of using learning analytics to systematically identify students at risk well that's brilliant but then we need to also sort of systematically deal with those issues that means we've got to have an institutional wide personal tutoring system that gives exactly the same level of service and I'm going to use the service word here alright, to all our students alright, that's quite tricky I know of one institution that's got nine different personal tutoring systems alright, it's a complete lottery on where you are so if you're going to provide this overarching service looking at data to actually see students at risk identify students, identify issues give more information to students we also think about okay, what's the end point well the end point is providing support for those students so that means if we're going to do this we've got to take it really, really seriously with our institution this is not a technical problem this is not an IT department finding something out going there you go that will help this is really thinking okay we've got an opportunity here to really benefit our students how can we make best use of it so we've spent a lot of time talking to students and students are brilliant students actually thought we've done all this already we explained some about learning analytics and lots of screenshots of yeah you can have this on the mobile app and you can have this dashboard and you can do this what are your concerns, what are your issues I don't have time to go through everything but I'm quite happy to share this with other information as well they're going yeah it's quite obvious we're going to be transparent but we trust you already and that's really great our students trust us with our data that's a good starting point yes you're never going to get a one size fits all solution we had a majority of students going I'd really like to compare my performance with other students' performance within my cohort obviously anonymised I'd like to see where I am but there are some students actually go well I don't want to do that so we're thinking okay well we may need to actually provide an opt-out solution for that if we continue down this route but the big big issue there and it's flagged in red but if we use analytics to actually identify an issue that's actually got to get some people involved that's got to start a conversation I know I can actually go to software suppliers by a solution and it will actually automatically generate emails if a student doesn't do some things and it will send them you haven't attended this, you haven't done that or you're in breach of this particular code of conduct that's not very helpful so the students are saying if you're going to do this link it to personal tutoring get some people involved, get the human factor there staff, the staff perspectives so I've sort of lumped in academic technical support staff again institutionally going yeah we're actually going to support this this would be good there are issues staff are more considered about the legal issues, the ethical issues but again we can overcome them getting the point about being open and transparent using this to actually benefit the individual, benefit the student learning so this is from the staff perspective this isn't about using analytics for performance management of staff, this is about using analytics to actually benefit students benefit the student experience help with the student journey but we also know from talking to staff that actually if we're going to go down or start an analytics journey we don't have to start right at the beginning we have lots of data, it's just in the wrong places it's just in the wrong format but we actually know where the data is one of the jobs I've been spending over the summer is actually trying to find all our data and I had some didn't have the grey hair that I have at the moment when I started that journey but we're all in the same situation there senior leaders it's always quite interesting getting deans together to find out what they want and we did, we had some great conversations with our deans and it did show to me that if we are going to go down institutional-wide learning analytics approach we need to generate that shared vision to make sure that everybody sees the benefits and we're using it for the same purpose right across the institution and there are some concerns about things like scope and quality of our current data but hey, there are challenges we can overcome them but when you start digging with our senior leaders they can actually see actually we could use this as a measure of learning game we could use this to inform curriculum design in the future so we're actually seeing some other benefits there that if you can improve your overarching teaching, your course quality it will again have an impact on the student experience this quote in here in red for me was one of the most important things that came out of this piece of work actually to have one of our Board of Governors actually sit down and say well after explaining what learning analytics is what we could achieve with it, what we could do in the future actually saying now really data is everyone's responsibility it's really easy when I've worked in higher education for over 25 years now to say well there's a problem here it's with the data, we blame the data well actually we all own that data we all provide that data we input into it so we will need to make sure that we have the right support training and the embedded use of data and data does become everyone's responsibility so as a project we went around, spoke to a whole lot of people and we came up with a lot of recommendations and the recommendations fit into three areas sort of process, policy and technology and I'm just going to go through these really really briefly so in terms of policy this is actually having a plan making sure that we actually have learning analytics owned at a high level that we have goals, we actually have an initial set of measures and we know that what we're doing and we actually want to make sure that we are future proofing ourselves and we make our use of learning analytics sustainable across the institution in terms of process very much about embedding into practice so we're actually looking at some cultural change here about, I mean some of the work we're doing with staff is identifying staff going well if you want me to do this we need to have better training better staff development, we may need to look at how that inputs into individual roles we also need to be aware of that this is not a general panacea implementing learning analytics will not solve all your problems it will actually identify lots of problems we haven't done this yet and I've already identified the fact that we've got policies missing or two policies when there should be one that the right people own the policies all these things will come to the surface but we need to be aware of that we can actually solve those issues but staff development is probably one of the key things that's come out of this piece of work and I also get into a lot of trouble with my IT department when I say that actually to do this that the technology is relatively easy it's a plumbing problem the problem is that the plumbing we have at the moment is really really bad and we don't have single versions of the truth for everything people like to collect data they like to think it's my data I'll keep it myself and I might let you have a copy of it the things you have a copy of the piece of data it's no longer original it can be manipulated, it can be changed so we must have this cultural change about single version of the truth for all data and all the policies that are associated with it as well and the other things that we came around about the technology is that we don't throw things away we've been working as an institution we have a really good mobile app for our students to love hey that's an interface we can use that we have an internal developed system called S3 that sort of does some attendance monitoring some other bits and pieces really well liked, we use piece of software that can become part of our analytic solution this isn't a question of just throwing everything away and buying another product so I was asked in this session to sort of develop a checklist and this is for us is quite the interesting part of it is trying to pull some things together this has got to be really really student focused so this whole idea if we generate a warning that we actually get some people involved we get some real process in there that links into personal tutoring but aware that that's actually going to cost us in time an effort single version of the truth really really important we need to look at our policies as well institutional policies, I spent half of my life looking at institutional policies because I have no friends and I can guarantee more or less go to any institution now and I can find at least two copies of any particular policy, I'll dig around and find them for you but we need to actually make sure that our policies are there and there's a single version of it training and support really really important and it's really become about ownership as well it's one of the things that we're pushing really hard is to actually go with the data is really important the policy is really important they need to be focused so bringing in people such as your academic registrar to actually take ownership of analytics so from our perspective is that we can't do this without the support of our senior managers and now as our board of governors it's our university executive group it's the deans and so on and it's coming up with that consistent vision of what we want to achieve institutionally it's not a question of bringing in as I said a piece of IT to solve it where do you want to be now three years, five years, ten years down the line single version of the truth really really important ownership and responsibility for data and for policies it's reusing stuff that you already have that's really good it's about making sure you provide that linkage into personal tutoring and to the support and you also have the benefit that you can use the information gathered for institutional decision making and I suppose the final point that I'm trying to make is that I should go to a lot of events of learning analytics and see lots of graphs and lots of data and it's not about the technology it's about cultural change and we really really need to start is talking to our staff and students finding out what they want to do what they want to do with the information and then we work out where we're going to go in the future I think once in my life I'm exactly on time so obviously if you have any questions very happy to take them stunned you all into silence must be some questions somewhere, anywhere? great just wait for my quote please you said that students were less concerned about privacy and kind of all those issues than staff were in your survey do you have any reflections on why that might have been might be that they assume it's happening already I think it's because students are actually used to sharing data and we hear lots of sessions on digital skills and competencies and what we should try and get our students to do and you're actually going to talk to them and see their behaviours they're reasonably savvy and it may be that they just expect if I go on Amazon I'll get recommendations someone is going to share my data and there's an acceptance to it I mean one of the things that I've actually found quite scary when doing this work and I've really kept on saying something called it here is going and actually looking at what our students are asked to sign up for online registration and if you've got to spare five minutes go and have a look at that in your own institution because I've found that we've actually built on over the years more things are added students don't read it no one reads it but maybe that's a cultural thing that we do every time you do an iTunes update do you bother reading the terms and conditions so students are quite happy to do that because we get some more benefit from it or there may be some urgency in that but time and time again they just trust the institution so we've got an obligation to make sure that we frame it right and we look after their data I think the students are actually saying we trust you to look after our data Question over there Hello Rodney Tamblyn from Tellus here I was just wondering in one of your slides you talked about data must become everyone's responsibility I was wondering if you could just expand a little bit on that about what you're thinking of I'll give quite a good example a few years ago when I was an active academic I would do marking coursework end of the term or semester I'd have to collate those marks and put them into another spreadsheet because I was not allowed to access our student record system direct so that work then gets carried out again by another person and I get called in to actually check that the data that they've typed in and provided is right what we're saying is that actually with the right training and support and we make sure that it's flagged as being important enough that as a practicing academic I can just update my student records my marks immediately because I'm trusted to be able to do that the university trusts me as a professional to do that and I will take responsibility for it that means that I can't then complain that no no that data is wrong so what we stopped doing is farming the data input out to what can be low paid admin staff and we just embed that into everybody's practice and workflow we make decisions on this data and we're responsible for it we should be able to input it but isn't that a symptom of the disease that universities are dying in administration and devolving the responsibility back to the staff doesn't make it go away it depends what you mean as part of an academic role if I set an assessment transferring that information into a student records system is relatively straightforward unless it's 11 o'clock at night and you've got 200 students unless it's 11 o'clock at night and you've got 200 students to get through it happens because of pressures there's no difference there if you're just writing all that down on a piece of paper you're right to then expect to go into typing it directly into a browser or into a web form there's no real difference there there's no responsibility so a very quick question please Sharon Slade at the University I agree with pretty much everything you've said we've been through this ourselves at the OU we haven't fully solved it but I think you're asking all the right questions and I think fundamentally you're right it's not technically the problem the problem is ethical it's not about gaining trust and getting transparency but the big difference that we had I think to your experiences that when we questioned students about how they felt about our using their data they were not very pleased they wanted a tailored individual service that recognised who they were and what they were doing and what they were not doing but they didn't really expect us to be looking in any real detail at what they were doing and in tracking their behaviours I've addressed this by adding yet more detail to the registration process I can't think of another way to manage it at the moment so we haven't fully bottomed it out but it is about transparency and trust so what we've done is to create a policy another policy make it as public as we can create case studies for staff promote the policy and in plain English promote it to students so that they know more clearly what we're doing with their data and I think it's about that promotion that embedding side of it the example I gave there about students not being able to calculate their overall degree profile we don't make life easy because it's 30% of this year and 70% of that and you can take the top two marks it's not really transparent and what I'd like to be able to see is that coming through a student's academic career and this isn't just about identifying students at risk this is also identifying opportunity so it's about going actually that conversation with a personal tutor so look you're averaging a good solid high 2-1 but with a bit more effort over there pushing for students to the right sources we can actually take them over another boundary I also agree with your comment about the process shouldn't be fully automated that's absolutely key but our problem at scale at quite large scale is how do we manage that personal conversation component it's a huge challenge I would rather go through do this piece of work go through the conversations and having back my institution with other institutions so that we are aware that we are going to have to put more effort as a load of other institutions into personal tutoring and we need to streamline that as well but we need to be more explicit about what students actually get from personal tutoring and that's also a staff development issue as well in actually showing persuading my academic colleagues actually as a personal tutor it's not just being there when things go horribly wrong it's actually there to mentor and encourage a student throughout their academic life again that's a cultural change so analytics going in for that idea has triggered off a whole load of other areas of work as well Thanks that was really interesting Thank you Okay so we're just moving on to the fourth of the last five presentations this morning it's Paul and Sharon talking about learning analytics again and this time at the intersections between student support, privacy, agency and institutional survival If I can just get the presentation going that will help It's somewhere in there Great I want to acknowledge Neil and these colleagues for their presentation She learned them following us With Sharon we would like to present some thoughts on learning analytics at the intersections where we can take on learning analytics and we would like to explore the intersections between student support privacy, agency and institutional survival We want to acknowledge that we published the presentation this morning on slide share on a creative comment license so you can access it there and at the end of the presentation you have a list of references So when we started to engage with what is happening at all these concerns the challenges and the opportunities we realised that yes it's complicated there's not a single solution So I would like to just map the intersections as playing survivor as a high education institution the end game the issue of the murky middle how do we get students to move quicker through the system the notion of educational trihars and then student data So if we play the game survivor the high education series new rules, new contestants better than ever I think we have to address the the issue of increasing competition changing context internationalisation it's no longer and maybe it never was business as usual there's the issue of rankings regimes if we really want to move the murky middle let's just lower standard let's just lower quality and the murky middle will move but we cannot because we are being watched there's the issue of increasing funding constraints and austerity measures there's funding follows performance rather than preceding it the need for evidence and whether you like it or not evidence based management has become the new mantra in which we all dance or not and actually that quote of funding follows performance rather than preceding it was hardly in 1995 then there's a persistent consensus about student retention failure and drop out and we haven't solved the drop out puzzle yet and maybe never and more data will not necessarily help us I suspect there's the history of well intention bang bang approaches to increasing student retention and success it's like walking into a dark room and we hear noise and we have a VC that supports us and we should there's a mandate to provide effective teaching and learning and we have a fiduciary duty to really do our best and to do no harm and there's our intention to really optimise the student experience ensuring student success so that's the one the survivor the high education series then there's the issue of moving the MOKI model if we identify students that with some additional support and additional interventions we can actually move them to pass more without loading our standards and marking the papers differently that's an easy way without or actually moderating the marks upwards if we really care about moving students through the system then we sit with the issue of how do we determine which students to move and how do we target our interventions and then we get to the notion of educational through harsh a concept that originated in treating patients and the wounded on the battle fields where medics would walk through the wounded and all of them screamed for help and all of them are wounded in one way or the other and they're all screaming for help and you as a medic must make decisions on who to look after first and there's those that can actually go back into battle sorry for the metaphor I hate war but go back into battle with the minimum attention you can wrap a bandage and kiss them better and there they go or they need more attention or actually and that's choice because you lack time and you lack resources you just pull the blanket and then the question is are struggling college students like cuddly bunnies that should be drowned should we just walk away from those on the margins so the issue of how do we determine criteria and which characteristics what are the costs of care the intervention the return on investment increasingly because we don't have resources our VCs and our management want to know that if we spend more time and more care on that group of students will we get a return on investment how do we move the murky middle and then the implementation is the implementation of educational triage and then into the evaluation that often doesn't happen that this intervention really make a difference and who cares because by that time there's another noise in another dark room and we should let's just also finally engage with some of the assumptions and practices regarding the need for more data somehow we have this data fetish that we think if we can trace more and calculate more and gather more data I resuspect that often having more data will not necessarily increase our understanding on the factors that impact on student success success and failure and retention and then can we assume that knowing more per se will not necessarily understanding and care and that more data will necessarily contribute to better teaching and learning some else our VCs believe that there's a danger of context collapse and the need to ensure context integrity when data collected from the spread sources and you mentioned that and for the right purposes are combined what happens when we collate and merge data from the library and the student management system and the learning management system where we collected that data for different purposes and yet we merge and we think we know there's inherent biases, dangers and potential algorithmic decision making working at scale we cannot have humans interact with 380,000 students it's just not possible in my institution so we must make use of intelligent tutoring we must make use of machine learning we must make use of all these systems but what are the biases what are the dangers what is the potential and then the issue that Sharon as well as Neil mentioned the scope of students' right to privacy do they have a right not to be tracked can they opt out and say I prefer you not looking at how I engage with the resources over to you Sharon okay so I was going to talk about what triage means in practice for many of us and Paul has already referred to what we mean by educational triage which is about allocating a resource to the most effective place and that's a very difficult concept to get your head around and what you may or may may not already recognise is that we do this all the time anyway we do it unthinkingly and that's one of the naughty issues that we're starting to explore is trying to get some more formal sort of learning or a framework or some sort of explicit recognition of how we're making those decisions and my own experience as a parent is seeing my kids who are who work hard to get their grades and understanding how the school directs its resources and the focus at school level on achieving the lead table results and getting those crucial A to C grades so quite often the focus is on those pupils who are just bubbling under the boundary In traditional universities Paul and I both work for Distance Learning University so we have a very different sort of experience but in traditional universities a lecturer standing in front of a class of students may react to what is visible to them and it may be an individually led decision on how I see that student appears not to be understanding it may be at that kind of level at our level we rely as Paul says more on the data to tell us what we think is going on so that's the problem in a way at the Open University we've implemented learning analytics in quite a big way and we're doing some really very clever things with it we think, we're using predictive modelling to identify students at risk in real time so as they appear to be heading towards dropping out so potentially vulnerable students we're using analytics to better understand our learning design so redesigning our assessment strategies our teaching and learning strategy but one fairly simple model that we're using is a framework which delivers consistent support to our students so we've set in place a series of module level and qualification level interventions which track students so basic binary yes no do they meet these criteria interventions which in theory allow all of our students to get a common support experience and that relies on the data that we're collecting about them some of these are very basic interventions hello welcome to the OU, you've started your exam is coming up that kind of thing and some rely on study behaviours or demographic characteristics we notice you haven't submitted your assignment the problem that we have is that we have such a large number of students that we're having to make decisions about which interventions are key which are the most important and the decision making process is at a local level if you like how do we make those decisions are we relying on high population modules which ones are the most important to university in terms of income streams or are we looking at particular groups of students at the OU we've got to focus on widening participation for example so do we focus on those students ahead of others and who makes those decisions so for us in practice this is where the concept of triage applies we don't have limitless resources we have to prioritise and how do we make decisions on those priorities at the moment we're moving towards moving away from a system of what was intended to be standardised support and consistent support for students towards one which is driven by constrained resource and us making decisions on where to invest that resource so I think Paul's mentioned a lot of points really but we're in this reality where funding affects what we can deliver the complexity around we would all like to move those students who are bubbling under who are potentially just under the sort of past boundary we'd like to move all of them up but we can't we can't afford to do everything Admissions is a big issue perhaps more so for us we're an open distance learning institution so we have no admissions criteria is that a moral stance to have should we be applying some sort of admissions criteria and balancing that with the cost of success for students so those who we may think have very little chance of success should we be admitting those students so lots of decisions there the issue of triage again transparency about how we're making those decisions to our students how transparent should we be about why we're supporting you and not the person next to you should we be telling our students that should we be transparent with each other about why we're prioritising in a particular way and then the issue of consent which is a big issue very tricky issue to get right should we allow students to have the ability to opt out of receiving support or constrained support based on what we think we know about them or should we be making decisions for them on their behalf as Paul said it's really complicated and we've done quite a lot of work in the area now in our research but also at the OU and we don't have all of the answers it's something that we're still groping towards and I'm hoping that some of you will have some ideas that will help us in some of this so thank you very much there's a lot of references on the end of the paper which will be on the slide share if there may be thank you Any questions? Thanks that was really interesting you were talking about using algorithms and machine learning to identify students and I've been fairly involved in a similar project at my institution and I wondered I suspect that some of the time the criteria that we're using to identify the students that you know we call it at risk of disengaging are instinctive like we think is there I don't know how do you approach that do you have any comments on how you might improve that because I suppose it's all based on you know that's the starting point if that isn't done right then the whole rest of it for my context and Paul from his we've done a lot of work on students who succeeded or not succeeded at particular points in their studies and so we've understood that there are types of behaviours if you like that lead up to withdrawal so that's part of it we've looked at data and that's one of our concerns I guess is that all predictive modelling is based on students like us not necessarily us so going back to the human intervention we try not to rely on predictive techniques alone so we're trying to filter through a human if you like so our model tells us that we think these students might be in danger of disengaging we'll get their tutor our associate lecturers in our case to act on that data model but I think I personally would like to explore what's proposed by John Danaher at Galway where he actually has a matrix where where you have humans acting on their own which is not scalable in our context humans working together with machine learning or artificial intelligence whatever your particular fancy is and then humans overseeing algorithms and then algorithms acting on their own and it happens on four levels where we sense something is happening and we make the decision or an algorithm makes the decision and acts on it so in our case back in South Africa if we sense that a student hasn't logged on and there's some assumption that students have logged on to the learning management system has a greater probability of passing if we redline those students immediately it can be not because they don't want to but because access to online is not something that is applicable to everyone and the cost of access so if we give them access they may actually survive so if we make one criterion online access then we may exclude a huge number of students and in our context we have a social mandate you may have the privilege to exclude students and feel good about it but in an open business education institution we have a mandate to social justice and we want to make it the present that's what makes redlining students and killing the bunnies a non-option sorry a conscious of time so thank you very much indeed thank you so we have our final presentation now and I have just noticed in the programme it's just say that we finish at one o'clock but actually it's one ten so we have 20 minutes left for the final presentation which is from Jim and Sheila learning analytics you ready okay just while we're getting set up I just want to thank all the presenters in the session before us I always feel slightly like I'm standing on the shoulders of giants when I come to all but I really feel like this today so I wouldn't say we're going to lower the tone but we'll try but we're going to perhaps give you if you're maybe struggling with learning analytics and you're thinking gosh how are we going to get started hopefully our experiences will help maybe reassure you that yeah we can't all be as good as some people all right there so um sorry and this is Jim Henry sorry this has been live streamed so I'm Sheila McNeill senior lecturer at TCU in Glasgow and this is my colleague Jim Emery we've been working on learning analytics along with our colleague Linda Carina in the audience for the past couple of years so we thought we would talk about joining the dots just in terms of the conference theme so this is just kind of an overview of where we are with learning analytics a nice kind of timeline about it so kind of back in 2014 so when we sort of started thinking about learning analytics and we did all the first sets we've been to the conferences, we've done things we were trying to find our data getting the right people, getting the institutional buying and we got quite far in 2014 we got a bit more started we got institutional buying we got even a draft pilot written and approved and then a few changes happened so we had a temporary CIO at that point, that change with a change in senior management so it kind of stopped so then we had to start again and it took us to 2015 to start that process again so we got a new CIO we had to then get institutional buying and get the right people and that took a while and then we had the just discovery phase which we'll talk about a bit more which has really helped to kickstart things again so we kind of started and stopped again so we had another change of CIO at the beginning of the year so we had a bit of institutional buying we lost a bit of institutional buying trying to get a bit more institutional buying again but we have got some more institutional support we have got some work approved by our academic policy committee and we have got a big dump of data from Blackboard which is our VLE and we are developing a code of practice and we're instituting a very small technical pilot so it's all quite a nice kind of set of dots but actually as we all know all the dots are very complicated and the handouts that you've got we just thought just because it was play we get you all to do a bit of dot to dot as well so there are no prizes for guessing what this is but the best coloured in and first person to actually join all the dots in the right order may get a prize so I'll hand over to James All we're actually doing with the conference theme very very simple and straightforward is have a look at the forms of handed out all we're actually doing is join the dots connect if there are not enough copies would you please collaborate with one another and if you don't want to start at number one be creative start anywhere but I'm not going to tell you what the highest number is remember the basic rule is join the dots you only do through a number once so we'll have a goal so you can come out with it and please nobody create a troll based on this morning's we're okay what we're looking at in terms of joining the dots is we're looking at our systems we have within GCU and we're based a lot of our activity on our VLE in our particular case it's blackboard but we've got lots of systems which feed into or sorry more accurately to our VLE and what we're trying to find out and we're only at exploratory stage just now is what are all the systems how is there any way of impacting on the VLE do they relate to anything in the VLE and what happens outside the VLE this is important for us because at the end of the day we can just generate simple reports but we want to go into a bit more in terms of analytics not in terms of predictive analytics as such because it's too early in our development but we're only trying to understand what trends are out there if any that's the point if any and a lot of previous speakers have pointed out what they've managed to achieve sadly we're not at that stage yet but we're starting this journey and over the next few years as it's going to happen we'll be blogging and tweeting about it so these are the main systems we're looking at primarily within the learning teaching environment and we're hoping to learn how we can analyse or find the data and think about analysing it so we're kind of at the discovery exploratory stage yeah so some of you may have seen this when we were doing our exploratory work marched up from MMM you heated a fantastic version of the London tube map with all the university systems very very impressive this was kind of what our systems were a bit more like I don't know if you're familiar with the Glasgow Underground it's basically two concentric circles going the opposite direction a bit like that with our data in our university and we never met so that was kind of a way of us starting to visualise our data again it's a very complicated picture so just starting from number one we thought we would start with our VLE that would make sense so back in 2014 that's what we started to have a look at but what we've soon found out and I think what everyone finds out is that actually getting from one to do takes a lot of time and that's what we've found a real real challenge getting that money and resource unfortunately unlike me we haven't got as high senior management buy in yet for that though I think that's I think there's more interest in analytics just now but that was a real challenge for us and getting resource particularly from our information services department is still a real challenge for us so we're working on that that's when GIST came along and actually the GIST learning analytics programme was very very timely for us because they had what they called the discovery phase which was basically free consultancy great so they had consultants who would come in and assess your institution's readiness for analytics and it was great and it actually did many of the things that Neil was talking about that he did in his project with the HEA so we talked to, we got set up meetings with same kind of stakeholders with senior management with staff, professional services and with students we didn't have as much success with students as it sounds like you had we found it quite difficult to get students to come and this discovery phase the consultants were only on site for four days so we had quite a short time to organise everything and although our contacts and student association were very very keen and we did actually have a one to one interview with our then student president we had quite a small turnout for the actual face-to-face session with students so that was slightly disappointing but we did get something which was good and we got the report at the end of it this is kind of the summary of our readiness we were assessed on four pillars of readiness through culture, process, people and technology and we were kind of ready with recommendations in most areas not ready with recommendations in technology which didn't surprise us because as Neil alluded to as well and we were kind of sitting there smiling when Neil was talking about the plumbing we have quite serious plumbing issues as well but actually we were quite happy with that report and if you want to see in more detail the findings of that that's a link to our open educational repository so that's there anyone can have a look at it it's a really useful exercise to go through I don't think it probably told us, our team much more than we actually knew already I think having that external validation was actually really useful and it also generated some a groundswell of interest and started some conversations about learning analytics which we've been able to kind of utilise as well and the key recommendation from the project from the discovery phase was this to implement a pilot project using the VLA's KDF data so you know, not exactly rocket science but it was great again to have that external validation of what we should be doing so we've gone back to our DOS and that's what we're starting to do so the Red Dot GCU Learn is our badged version of Blackboard so one of the things we wanted to do was to try and get access to our Blackboard data I don't know if any of you have Blackboards as an institutional VLE and if you've managed to, have you ever asked Blackboard to get any data yeah, it's quite a challenge so we've had a very interesting discussion with Blackboard but we did and we have now got basically all our Blackboard data there were some challenges around that including not being able to find a plug for about a month and a half perhaps so we're looking at that just now and also at the same time and this is based on a lot of work that Sharon's done and the GIST project as well we're looking at developing a code of practice and the ethical implications of learning analytics cos that's one thing that's come through so we certainly don't want to start anything without actually having that in place so we're spending quite a lot of time so again going back to this culture element as well which is really important and we want to, you know, we're trying to learn what other people have done so we're trying to get that in place and we're working on that just now and it's been very timely with the changes in data regulations in Europe despite Brexit so it won't happen, I'm sure that it's still going to have an impact on data regulations so we're working on that and we've got a small technical pilot but yeah, we just thought I'll let Jim there, we kind of go into the data a bit more This is a first shot we've ever had of the data from Blackboard here, of the 18 tables too many to look at so the decision we have to make what we've got to do here so essentially, and this is the sad part about it is actually delve into each one to find out what content was there it's not coming up there we go here's one element winter and you can see here's a whole series of values a whole series of elements attributes which may or may not relate to the learning teaching practice but that's the kind of thing we're looking at on an individual basis just now the reason being is we're trying to look at something we can strip out of that we can then relate to other learning teaching systems and make connections so it's not just a case of accepting what's there in a broad term but as ourselves investigating what's in there and that's the key point we are starting out in this journey we've learned a lot from people elsewhere we've learned a lot today and any further advice you give us would be greatly appreciated what we are concerned about is that some of our data sources may not be able to access but these are the kind of questions people are asking of us giving some kind of feedback on stats and reports and all that kind of stuff not within modules but across university to represent or give an idea of university practice so it's a challenge for us it's exciting times, it's going to take us a bit of time but the critical thing is initiative to contradiction as Sheila alluded to earlier the university has its own digital strategy whereby saying that learning analytics will influence what we're going to try and do but has never put in place the process to develop those learning analytics as an outcome so we can then use our learning analytics so what we're currently doing this is why we're taking our time on this we've not set up a specific project as such but we're digging into the data having a look at it, trying to build relationships and then finding out if we're not we can then develop a project with a pit and we'll have to probably have to do it in Prince II principles but we're not at that stage so we don't even know what we don't know so we're almost fans of Donald probably fans of Donald that we've helped my whole audience here just now in so far as he's known known to he's unknown unknowns so this is who we're at we feel a bit, not quite embarrassed but we feel that we're on the same wavelength as everyone else appears to do it's not a fashion concern but it's something we're committed to as a university as a student I think that says we just really wanted to share that we are trying to do something around learning analytics but it has taken us a lot longer than we thought an awful lot longer and it's actually going to take us an awful lot longer again but maybe in some ways that's quite a good thing because it needs to be as Neil said about data becoming everyone's responsibility and that is a huge culture change and that is going to take time so we're just chipping away to have more to show you and we might have been able to make sense of the 800 odd tables that we got from the Blackboard data dump but we are sharing this as openly as possible so we'll be blogging and sharing our findings as well but that's where we are so thanks very much and have anyone managed to do the dot dot? Yeah, well done well done pictures and tweets please that would be great I know this is a rush for everyone to finish that Any questions? I know we are the only thing that stands between you and lunch so we're not offended at all as there are no questions Thanks very much Hi Sheila I was just wondering what do you think are the challenges in getting that kind of high level buy-in for something like this that really makes, as Neil was saying things start moving and doors start opening is it just competing priorities or is it perceived as a technology issue and not necessarily needs consideration at that highest level? To be honest I think it's probably a bit of both, Sarah I think that there are a huge amount of competing priorities and at different times those priorities can really act in our favour in terms of getting things done other times they won't I think there's a bit of misunderstanding of what I have noticed that people are using the word analytics and I think that's possibly because every system had an analytics component to it just now so it's becoming more of a commonly used term but there still isn't that understanding really at the senior level and I think for this because it has such huge implications for things and particularly because you're touching on data management and people are very protective of their data and they won't give it up they've got a very good motivation to do it or they've got a a very strong reason and I directed from on top to do it so I think you kind of need that but I think it's a bit of both I think one of the big issues is that we have to make a move on within the competitive environment and if you don't do anything what will probably happen at the level at some point some will turn out why don't we have it, what have you been doing so that is a kind of institutional risk element we've got to ameliorate somehow or prevent so starting at this now in a slow but hopefully effective manner we can start generating something that we can get our teeth into create a project that will have value to the university community Hi there Joe Wilson independent education consultant it's always interesting to see the back end you know you hear people talking about analytics and the data that the blackboard generates or indeed Moodle generates and again that bit where Joe kind of dropped you said you've got this data and you're now beginning to interrogate it but I'd have thought usefully in the system somewhere there would be some road maps and things that would show you you know because the back ends the back end and what other people are doing with and naively I thought that would exist already at least to some to some degree I think some of it does and there are schemas that are there but you still have to interpret that and I think for our colleague Ken Fraser who we're working with what he's trying to get his head round is understanding how how the relationships in that data actually work because with this dump that we've got from blackboard we've got absolutely everything we've got tables about invoises about nothing to do with learning and teaching we've just got everything so going through that data is going to take some time and some of it yeah we can just strike off we can do that but that does take a bit of time and that is kind of human time just now I think once we get into some data that has got some meaning for us hopefully and we're at the stage now we're identifying some of that data then hopefully the other tools that are available we will very quickly be able to get to that stage but yeah it was basically kind of there you are I got almost that type of thing so yeah there is help but it's like everything it's like the IKEA instructions are only really useful to some people adding more questions time for one more can I just quickly ask one it's really relating to that coming from a software supplier how close are we to define some level of schema to start to demand from these software companies some data back because at the moment from our perspective you could provide it on an individual basis much like Blackboard have done but it would be easy if there was something that we could actually start to sort of define and then everybody would conform to a similar standard I think again it goes back to what you actually want to do because just now I think the thing is that everyone wants to every solution has the solution that they will give you the answer because they will sell you their data warehouse and they'll sell you their way of doing it so I don't know how close we are but I think you'll want to kind of come in there as you mentioned suppliers I currently have a list of various systems and I've got nine different ways of defining a unique identifier we could actually and we're going to have to do a look up table a translation of unique identifiers to create a unique identifier it's very frustrating that's one of the biggest challenges that we have is that the word unique doesn't work so thank you for that it's now 10 past one so I think it's lunchtime thank you very much indeed Sheila to Jim and to all of our speakers thank you very much indeed hello I'm Ian Griffin and I'm a learning technologist at the University of Thampton I think Ian has taken very much a practitioner of view to actually working with technology so he always puts the user first and then the technology second for me what makes Ian's approach to the use of technology so outstanding is the fact that he has this very analytical and methodical approach yet he manages to temper that with a real human element Ian works in a very stricted way he always puts the user first and whether that's staff or students and it's very important for us as learning technologists to do that because we have a very enabling and supportive role I'm really impressed with Ian's attention to the requirements of the end user and that that end user has a really good experience of using the software I've worked with Ian across a number of projects and I'm you know I'm really always impressed by his can-do attitude nothing is ever a no Ian is always like we'll find a way to do it and he always does he's a really good leader and he's a really good motivator and helping people to embrace new technologies and think of new ways of doing things that's what he's done for me Ian's been brilliant really he's got leadership skills he's very motivational we're always changing things within the portfolio and he never gets cross and he's just like yeah let's go with this Ian's been really really useful to us in developing Pebble Pad almost from start to finish and enabling the students to feel as though this is something this is an application that they own yeah he's also been able to give us ideas about how we might use Pebble Pad differently how we might be more creative using it with our students which has been really helpful Ian has a very positive can-do attitude and the focus on enabling and supporting others not just taking over and carrying out the work himself but actually supporting you in a very non-threatening very positive way to develop yourself as well as a colleague Ian's very positive he's always easy to get on with and always comes out with very good ideas we started at the same time and we've kind of gone through the process of learning technology and becoming a learning technologist together so we've kind of developed as people but because we started together we kind of started with the same kind of view, focus and we want the team to be the best that it can be so we've worked quite hard and we've talked about a lot of different things that we could do to make our team the best team obviously he's better than me I'd say it's a pleasure working with Ian and I think his collegiate approach to working with me has also enhanced my development as well so we're able to I think all benefit from the working relationship that we have I think for me it makes me feel as though there's no stupid questions that I can ask he's always approachable he's always available he always seems to understand what my role is as a lecturer and he kind of gets it from that point of view he's very responsive and very clear in how he responds as well so if he can help you there and there and he will if you need to give him a bit more time he'll say it's no very responsive I think Ian's passion shines through the most but as I've already said he's really motivational he loves what he does he's out of exudes out of him and sort of influences anyone who's working with him really and we all then become excited I think the particular thing that Ian has done is he's made us think differently about interaction with some of the technology we need to constantly challenge ourselves as learning technologist and within the learning technology team and Ian has very much embodied that whole ethos of an excellent learning technologist working with him on a professional basis the most extraordinary thing is that you can give him an idea or a problem or a project and you know that he's going to go away and analyse that and disassemble it and then he's going to come up with a proposal for a solution but he's going to go away and then surprise you by not just coming up with that solution but then coming up with an entire workflow which he will then implement and then produce a response to that or a solution to that problem that's going to be way beyond anything that you had even imagined was possible He's just providing real innovation and creativity and for me that's what makes Ian stand out as a very special learning technologist and a great person to work with that's made me do things differently I'm Fiona McNeill and I am a Learning Technologies Advisor as part of the e-learning team at the University of Brighton I believe that lived experience is fundamental to our retention of knowledge which is why I am always ready to don any hat necessary to create the desired effect My stock and trade is getting under the skin of experience both understanding how other people experience using technology and thinking about ways to package technology skills into meaningful accessible experiences I have approached this in a number of different ways over my career Drawing from my background in the creative arts and curation I consider wayfinding reflection and discovery in both virtual and actual worlds From early projects helping students to build virtual art galleries in Trimble Sketchup in the USA to mobile device speed dating is a means to learn about different apps and device potential to app swapping over a continental breakfast or afternoon coffee to my gamification projects for your size Linking experience to learning supports retention and stimulates delight leading to more positive feelings about technology and yes, more learning All of the events I previously mentioned have some core elements in common They consider space and changing spaces to enable reflection and collaboration lifting spaces out of their day-to-day doldrums and transforming them temporarily for play and exploration They consider time and how the scarcity of time can help to sharpen resolve when learning about technology They consider sharing and how these events, their relics and products can be shared with the wider university and educational communities They are playful yet remain grounded in learning theory with a focus on mobile technologies and are informed by the needs of our community Taking app swaps which I founded at the university in 2013 as an example If you encounter an experience like an app swap or even one of my workshops you will arrive with an idea and by the time you leave you will have a solid plan and you might possibly even have created something for an experience to be successful It needs to be open to everyone so inclusive practice is very important to me Whenever I use a technology I consider how others might use it and ask questions to find out where adjustments may be needed I support these goals for staff through my involvement in the university's Disabilities Equalities Group and students through training for academic and support staff and groups of students through voluntary participation in student support events I am not afraid to rock the boat and frequently take part in user studies and undertake my own research to try and inform and improve the design of technology and software To wrap this all up I am passionate about my work and I consider my role as that of a tech translator providing vital layers of empathy and interpretation between the experience of my clients and the technology they use Thanks for watching The module is part of a blended and online learning and teaching course for which I also attain certification aimed at academic and professional staff across Hong Kong's six tertiary institutions The module is designed as self-paced to enable participants to accommodate their portfolio development around their busy workloads whilst the mentor views offer audiovisual mentor advising guidance on demand and on the move Currently with 24-an-roll participants we have adopted the module raised awareness of ALT, ASALITE and C-ALT across Hong Kong whilst also reducing the number of referrals increasing statement quality and critical reflection and encouraging participants to use alternative platforms for their portfolio I am an active member of the ALT community contributing a blog post to the ALT C blog Limig in a mixed reality focusing on the re-emergence of augmented and virtual reality and its potential for teaching and learning and also co-facilitating a session at the 2015 ALT online winter conference showcasing different approaches to building a C-Malt portfolio Expanding on my previous practice and experience in the UK I have led the adoption of augmented reality for teaching and learning at PolyU Utilising all four walls I use augmented reality materials to transform the conventional classroom into a 360-degree interactive learning tool encouraging students to explore, engage and interact with different multimedia learning content during free-roaming This self-paced free-roaming that key intervals allow students to absorb digital content which is overlaid onto the real world using their smart devices before coming together again as a group to discuss constructive knowledge in peer-to-peer interaction This work has led to my invitation and delivery of a guest seminar to staff at the Hong Kong University of Science and Technology and to deliver a professional seminar on a digital corporate communication module aimed at teaching master's students on marketing how they can successfully apply augmented reality to corporate marketing strategies and campaigns I am also leading a work in collaboration with the Centre for Innovation in Teaching and Learning at the University of Illinois where we are focusing on the research and development of emerging technologies through knowledge transfer and sharing of good practice and I have proposed an internal community of practice here at PolyU to ensure effective dissemination of knowledge and skills My work with augmented reality has culminated in contributing a book chapter entitled Enhancing the Physical World with Augmented Reality which is to be published as part of the book Innovative Teaching and Learning Practices in Higher Education by Labour Publishing in collaboration with the Institute for Learning in Higher Education Working closely with my colleagues at the Educational Development Centre I have enhanced the existing traditional phase-to-face method of staff development into a more accessible and productive suite of training that capitalises on the online platforms and a new innovative and collaborative learning space using mobile computers on wheels A new learning environment within our institution called the zone contains flexible furniture that allows educators to manipulate the physical environment to meet their teaching style Access to this innovation has allowed us to redesign our traditional instruct-the-led staff development and invigorate standalone 90-minute sessions This is achieved by creating four tool or technology-specific stations within the environment in which participants spend 20 minutes before rotating to the next station This bite-size approach allows us to introduce multiple tools to participants in a digestible and concise way In parallel to phase-to-face delivery I am currently co-designing a self-paced small private online course for all new and existing staff focused on the continued development of knowledge and skills in the effective and appropriate use of the institutional verli This spark will progress through beginning intermediate and advanced levels using Blackboard Open Education Likewise, I have co-developed with my colleagues an e-learning support website for staff and students as an accessible central hub for video tutorials pedagogy guidance policy and procedure, FAQs and much more to promote a community approach to the adoption of Blackboard, MOOCs and online initiatives Cawer to my role supporting the institutional verli I facilitated the introduction training and widespread adoption of iSpring, CUT and Padlet a toolkit to increase active and collaborative online and phase-to-face learning developing achievable skills that staff can quickly and effectively adopt and integrate into their teaching practice capitalising on the mobile devices which they and their students already possess I focus on instructional design for a massive open online course in collaboration with the School of Hotel and Tourism Management here at PolyU providing pedagogical guidance and insight into curriculum design for online delivery I actively support the course team through the project management and construction Hosted on the edX platform current total pre-course enrolment stands at over 2,000 with participants from 122 countries As I have particular experience and skills in the development of clinical decision making formative assessments I'm currently working with putting an application in myself to maybe do a PhD one day so without challenging things within the portfolio and he never gets cross and he's just like, yeah let's go with this Ian's been really really useful to us in developing PEMOPAT clinical editors and to improve the coverage and steam of Wikipedia articles about women Another initiative supported by the open education team is Open Scotland which aims to raise awareness, open education Quite pleased, she's about to get a Pikachu I think that is important Anybody confirm? Okay a Pikachu is important We've got about another two or three minutes There's no clock in here is there You certainly lose it Are you streaming yet? Okay that's good, so I just wave to the internet Wave to the semi-buffered 90 second delayed internet No that is actually just the way streaming buffers so I teach network engineering so if I swear it it's a bad habit really Okay are we on the principle it's now two o'clock are we in agreement it's now two o'clock because I've suddenly lost my ability to see time I've got to give it another two minutes Okay I'm the warm-up act Okay the important question is before we start before we start recording is I am a geek I am a through and through geek proper geek trained as a great geek did geek in 1977 when sort of geeked them really took off He's already given himself a good preamble around periscope and teaching one of the network engineers Okay thank you very much Hi obviously I'm Andrew Smith from the Open University as you can tell by my slide there I'm iddling around with my phone so I'm actually live streaming now a periscope broadcast from my phone so quickly cast across the audience and Lisa's there me and point it at the screen sorry I've just disappeared from the camera on YouTube so Lisa at the front is not playing Pokemon she's not trying to catch a Pikachu and she's not running a gym or anything here so some of you are trying to get your Pokemon scores up this is the wrong place to do it working at the Open University I teach a large number of network engineers some of you may have heard of Cisco systems hands up many of you have heard of Cisco systems you know what a Cisco router is Cisco technology Cisco have their own academy program they have their own certification and we're very fortunate we teach a large number of Cisco students each year and because of that sheer scale we are the largest Cisco academy in the UK and one of the largest Cisco academies in Europe because of that we've actually got the benefit of being part of a global community we're all part of a global community all the subjects we teach are part of a global community but there's a very clearly defined social media community that I have access to that's preeminent language is English is a dialogue about network engineering in English sometimes in American English sometimes it doesn't spell it the way we do we have to get over it but the practicality is I discovered a few years ago that I could actually use with this community social media to enhance the learning so I've actually done other presentations in the past or at sea in other conferences around the concept of leaky teaching and actually use in social media as an automated pedagogical tool so I do a set of daily outputs on OU Cisco and other platforms based entirely around my module my course content my technical content it's stuff about routing technologies switching technologies IP addressing how the internet works and the kind of things that these geeks find very interesting because they need it for their careers and what I've found out of great interest I mean we've only been doing this for a couple of years we applied this to multiple platforms because at the time we didn't actually know which one would be the most interesting but between them we have around 2,000 followers and we actually have a couple of other groups that give us another 2,000 followers as well so we're reaching a community of 4,000 students which is even more interesting when I know that out of my 500 students on average only 20% of them engage with the social media so I've got these people from somewhere that are very fascinated with what we are talking around around Cisco so they are other Cisco students some of them are Cisco engineers some of them are alumni and some of them are just interested in sort of being part of the community that we've generated around this learning so Twitter about a year and a half ago they launched Periscope Periscope is broadcasting me at the moment as well as Google broadcasting me on YouTube any of you use Periscope one hand two, three, four, okay I would say about 5% of the population have used Periscope how many of you don't know what Periscope is yet and again about 5-10% of the audience quick crash course it's live video streaming here and now it's got a very brief buffer what's being broadcast from my phone with Lisa is being sent out to the world 30 seconds later so you can literally pick your phone up and broadcast an event and there's world events that are now being broadcast via Twitter this way it goes out as a tweet there's a link it goes straight to the video so I've just broadcast it as an alt C hashtag we've linked it to our OU Cisco account and we've actually then used my smartphone or other tablets to actually broadcast it and the beauty of it is it doesn't matter what platform the other individuals got so you could be watching me now on your smartphone or your laptop using a web browser I don't need to know what you're using and actually I don't need to care either so we're doing so much now in technology where we're always very platform specific and this is a very platform non-specific technology while you need the device to actually broadcast I'm probably doing it via my 3G connection at the moment because the wifi in here is sorry Warwick unreliable so what we decided to do was create a set of mini networking sessions 10 minute micro teachers around network engineering topics that a beginner a wannabe network engineering student would find useful my open university students at the start of their module need this to get them going fair enough other individuals out there prospective students or interested parties want this as well we kept it entirely technology based but we limited it to 10 minutes why 10 minutes because we found out from talking to our colleagues in the open media unit that most youtube videos drop off between 4 to 8 minutes you stick an hour youtube video up there on your really exciting lecture most people give up after 4 minutes and might stick it out to 8 minutes so it was about keeping it precise and concise and because of that we limited it entirely to a single topic a single skill one thing that they needed to know one thing to learn I use something called a network simulator so this made life a little bit more straight forward in my world there is nothing exciting about networking I like to think that it's a really wonderful cool thing but sitting there with a rack of Cisco routers just humming away and little lights flashing is deadly boring ok so we set it up with a simulator so they could see a graphical view of the network structure and could see us doing the configuration and the command line a little bit like a show and tell a little bit like blue pizza for those who don't know classic British television program when they always made one earlier then showed you how they did it so there's a little bit here is the working network now this is how we're going to set it up using this one skill that you need to learn today so my phone and a cheap 10 pound clamp from Amazon clamped to my desk I pointed at the screen and then just did the videos of demonstration nothing clever in it at all you could do this on any topic as long as you're demonstrating something or teaching something it's quite straight forward and it's not costing you anything for us these are to me are obvious subjects in the pedagogy of network engineering these are the kind of skill you need to know before you start I'm not going to bore you with what they all mean but basically they will teach a beginner networker the principles of routed networking technology and give them a taste of what they need to know when next and the shape of things to come but we went for a very straight forward formulaic delivery style one because we knew that these videos were going to be used used reused and abused to those that were to becoming familiar with it or comfortable with it would know what to expect in that short time slot it's very much introduction set in the objectives then doing a step by step demonstration and then telling them what that session was about and saying goodbye so I really only had about seven minutes to do something clever and something precise slightly less time than this presentation but it was about getting it out there because most people will disappear if I don't grab their attention within those first three to four minutes up on there that's a pretty standard network that's meaning to network engineers like myself but the principle is we could go into that PC do some PC configuration IP config and do the ping command we could go on to the routers do some routed network configuration and they could actually see the network behave and they could actually see the command line appear and develop and for want of a better phrase coding take place which is pedagogically quite useful for these individuals irrespective of where they come from well how did it go I think it actually went quite well but what we did find quite importantly is we had to pre announce it's no good just doing a broadcast on periscope unless your lady gaga no point if your lady gaga or George and you do a broadcast immediately you're going to get 10,000 followers we are not that interesting so we decided to do a set of pre tweets and pre updates saying on Monday night there is going to be this session we actually did two types of sessions are many Monday networking sessions because it was a good alliteration and it was a convenient time of the week for me and then we did another session where it was a Monday Tuesday Wednesday Thursday Friday night where we did one session each night in a series that built up so by the end there was a pedagogical path of development but again we pre announced and we announced each one as we were going along saying this is what we're doing this time of the evening this is going to be available and of course there is going to be recordings available for everybody as well we found that was quite interesting the scope of the sessions varied we tried some different times early on and we had some sessions picked up between 20 to 40 people but we found that the recordings afterwards easily picked up another 80 to 120 people when you think on my face sorry on my twitter following those around 850 participants at any time I would say that is quite a good attraction rate I don't know the values yet because we are actually starting to look at the impressions and impact but I would say that is a high hit rate for a community following based on that technology but we are finding that time of day makes a different obviously I'm going for a British evening time and I'm finding I've got an audience that are quite willing to watch things between 7 and 8 in the evening obviously it doesn't conflict with that important episode of Emmerdale or Coronation Street or whatever I really don't know but I think people are willing to participate at that time I have experimented with lunch time I have experimented with early morning and again I get a reasonable hit rate but I find that early evening seems to be the best for the audience I'm engaging with but you may find that your audiences are different didn't have time for Facebook Live any of you use Facebook Live Facebook Live no hands no hands whatsoever and if you know what Facebook Live is couple of hands Facebook Live is Periscope's competitor and going on almost I think this is the last but one slide so already tried Facebook Live and the fascinating thing with Facebook Live is we are already discovering that we are getting a higher population hit rate on a much more popular platform compared to Periscope because of the nature of Facebook in the community that participates in it and we are actually going to run a parallel contract we are actually going to do the same thing on both platforms we have to do them separately but in October I am going to run a series of the same lectures but one on Periscope at 7 o'clock one on Facebook probably at 8 o'clock and try that out and compare the two so we are still learning we are still finding out what works well and what doesn't work well screen affordances aren't always perfect sound quality isn't always perfect as well and sometimes when you rotate the screen it doesn't catch up and you suddenly find people turning their laptops to try and watch it but I think for free technology sometimes you have got to run with what is available but what we are seeing is a rise in participation on Twitter and we are seeing a rise in participation on our Facebook group as well we just haven't done it often enough to do a good critical analysis yet so one minute after the one minute any questions or giving me an opportunity to backpedal on what I said thank you bye mum thanks Andrew so any questions Mike if you have got one hello just a quick question was there any feedback on the quality of the video from the students compared to professional resources the quality is not the same I did a few experiments where I was watching on one screen and it was getting the right distance from the screen and I even stuck tiny little bits of tape on the screen to know where I could do all my command line and techie stuff so they could see what is going on and you are pinching in and out and stuff like that so the quality is rough but people are willing to accept it and if I had to move shake and I sometimes knocked the tripod I just apologised and told them what I did and they said you have got little smiles and hearts and stuff like that so if you are looking for a quality production nap but if you are looking for something useful that just gets you immediately so if you are out doing a filled trip somewhere and you want to do have an audience see it immediately I think Periscope and Facebook Live offer something really interesting and I just think that Facebook Live is actually lower quality because it's got that square box rather than the landscape box but it's got a higher participation so it's not quality it's probably more people just willing to accept it any other questions? no they they all want the more exciting presentation so we've also looked at our video use and found that it's about four minutes before people start to drop off so you mentioned that you went for ten minutes did you find a drop off before? I would say eight minutes is your optimum I just want ten minutes was an easy figure to explain to the audience when I was pre-tweeting saying this is going to be no more than ten minutes long probably engaged more people eight minutes is a silly number to tell people and because some of it is saying goodbye and this is what you covered in the session I sort of got the drift that most people were disappearing just before that anyway and then people dropped off and that's my data and I haven't done enough to see enough but I know because this is going out on YouTube I've got a friend and I are doing some YouTube videos and we're clearly seeing the magic eight minutes is the killer point and I would say four minutes is the average four points so something worth doing because I'm also looking I don't know if I'm Shirley's here but I'm doing some work with the Royal National College of the Blind and we've changed our view on podcasts we're doing micro podcasts the podcast of half an hour nobody listens to people will listen to a podcast of five minutes and of course you can stitch them together okay I will walk away Useful thoughts at the end though, wonderful Andrew okay so we're welcoming Vicky Dale now who's going to talk to us about university teachers experiences and impact on academic practice of a course on technology enhanced learning thank you okay so thanks for coming to this session this is work that I've done with Angela in the learning teaching centre at the University of Glasgow and obviously academic practice and aiding academic development in the latest technology enhanced learning priority so we just want to share some of the outcomes of a little study that we did with our participants to give you an overview of what we'll cover obviously we'll talk a little bit about the literature and the kind of background for this we'll talk about a rationale for developing because we've got some course which is a ten character course as part of a pg cat we'll give you an overview of the methodology unfortunately the results and where we go from there how we move this forward so in terms of in terms of what are the benefits of technology enhanced learning to staff well obviously technology enhanced learning is creating various affordances for training learning with lots of opportunities a lot of articles to the fact that technology enhanced learning is providing a catalyst for change for institutions and individuals it provides increased flexibility which we're all aware of and it also helps with the development of new skills and staff as well as in students but it's not always easy there's a lot of frustrations and challenges some of these challenges are around the clock and sustainability of technology enhanced learning I think we're all familiar with projects that have been funded for a short period of time only to fade once the funding is withdrawn for example the infrastructure is sometimes still prepared we've got campus-based learners working at a traditional institution when you try and move that to new online spaces sometimes the infrastructure isn't a slick they're confident and confident and they're self-efficacy all of this is time consuming it takes time to do this and there's a range of approaches to academic staff development in this area and I think for a really good review it's worth having a look at Mira Vogel's work she highlights things like design partnership working that Dillie Sammon does in relation to the Carpe Diem project she cites work around academic champions and she also notes that they're not related to technology enhanced learning other approaches to academic development such as serendipitous everyday conversations and I'm sure there'll be a lot of them happening during this event as well but more recently we've seen and I guess it relates really nicely to what was being said in the last session there about the 4 minute video lately the emphasis seems to be on chunking some foodie into much smaller components you know more accessible bite-sized learning and we see that in relation to Twitter chats for example so I'm sure that most of you will engage in something like LTAT it's really a really good way for academic staff development in technology enhanced learning but what did we do? We developed a PDCAP course and this was a recognition of the fact that the landscape is changing our teachers are no longer operating in the same traditional environment that they have done for a long time and we wanted to enable staff to engage with teleporches to engage them with scholarships but fundamentally this was all about creating a better learning experience so more self directed active learning more enhanced learning experience much more enjoyable learning experience more flexibility and what we sought to do in designing this course was to address this challenge that neurovogal identified which is to balance the advantages of a centralized provision and support for innovation with a respectful view of academic teachers as necessarily autonomous individuals who's learning needs are situated in their own factors because I think we're all aware of the fact that we can do something sensually and that's fine we can act on participants but that's not terribly successful on its own we need to be cognisant of their particular context and what they do in their day to day job there are course ILOs we're as outlined with four course ILOs and we map them to the UK professional standards framework the PDCAP results in the award of philosophy for the higher education academy so it's absolutely critical that all our courses are related very closely to the UK professional standards framework in terms of the evaluation evaluation methodology evaluating the success of this new course which we just implemented this year we had a pre-course survey, a post-course survey to capture immediate impressions, perceptions and then an interview six months later because I think the danger with a lot of evaluation studies is if you do it immediately afterwards you're only capturing those very immediate immediate responses you're not getting to see whether it's been successful over time that's what we wanted to try and capture so we can't present all the results of these to you because we'd be here for a couple of hours and I'm sure nobody wants that in the post-lunch slot we're going to focus here on the motivation prior experience and implementation of technology enhanced learning and the expectations of what the PDCAP would do for them so I'll let you read these quotes in a second but just to give you an overview why do people choose the PDCAP course we're really happy that most people were introduced to the PDCAP course we're really happy that most people were introduced to the PDCAP course we're really happy that most people were intrinsically motivated so this was about enhancing student learning primarily or recognising the fact that the landscape was changing and as practitioners as reflective practitioners of staff we really needed to engage in this space there was some evidence of extrinsic motivation around getting a certificate and there was also sadly some evidence of avoidance and I guess if we read these quotes from the bottom up if you like because I didn't want to engage in the scholarship of teaching and learning that's a bit sad because the course is embedded with the scholarship of learning and teaching but I'll let you read some of the other quotes really good that most of them were intrinsically motivated these were the technologies that they'd used in the past coming into this course so some of them you could say were more traditional teacher centred or very much embedded in the institution things like Moodle, Nahara, PowerPoint and video we also had interactive classroom tools maybe electronic voting systems or a big blue button that was a virtual classroom tool and we also had student centres our party tools as well some evidence of web 2.0 use a Ropar Helen's already given a talk earlier on about the use of a Ropar in the rest of Glasgow so some of our participants had used these tools before coming into the session what were the expectations again I'll give you a second to read these quotes just to summarise it was about becoming familiar with the range of new learning technologies engaging with the appropriate literature because all these academics have come from very different backgrounds and evaluating when and how to use the technology appropriately we do pick them up in relation to terms like learning styles because obviously that's quite contentious but the sort of overall sort of sentiment there really was about judging when to use technologies and that's appropriate that's what we tried to sort of encourage people to think about so the course course survey I'm going to talk a little bit about levels of attainment of ILOs, the perceived level of attainment the actions people have implemented as a result and what people think about technology enhanced learning as a result and those of you who can do maths recognise that there are six learning outcomes there and not four and that's because when we first designed the course we had the course specification in place before we designed the course backward way of doing things but most of you know that these things happen so we wanted to incorporate in the evaluation some of the aspects that weren't in the ILOs but you see that most people agree that the medium is agreed for all of these statements so that's good nobody disagreed that they couldn't meet areas of the ILOs and what did people do as a result they used a range of different technologies we had social media we had more use of Moodle quizzes content a roll pass of people who'd heard about a roll pass in the classroom through talking to peers decided that they wanted to experiment with that with that as meant of reality blogs etc there's quite a range of technologies that people felt they were able to explore in relation to their own students learning and it did change the perceptions it made them more critical I think in ways that were positive that opened the horizons in ways that were more negative so that last one it's about learning technology not technology and then what can this do how long have we been having this conversation in this particular terrain I think quite a long time but the fact that we've encouraged people to become more critical in questioning about technology enhanced learning is a good thing I think it's quite funny people thought when Angela and I went in to do this course we'd be like saying technology enhanced learning is the way to go you must do this but actually we've made people more critical and say you know use it where it's appropriate and I think that's what we're all trying to do so in terms of the interviews looking at the practice later this focuses on the impact and practice they enable us to successful implementation in some of the barriers because obviously these academics are going back out to the context that we were keeping in mind when we were running these classes so in terms of the impact and practice what does this reflect we were thinking differently really fundamentally differently the focus on deep learning reflects not using technology for technology's sake understanding the opportunities afforded by TEL but not being kind of slept away by them and understanding how students use and value TEL and maybe we don't value them in the ways that we thought they might question the idea that I'm digital literacy so I think that represents some evidence of transformation which is good and there were a number of enabling factors that were seen to help support this so these were course related and significant others and local infrastructure so obviously the way that we ran the course it seemed to be useful and also the text that we used up so the horizon report which is a horizon scanning document was to be particularly useful in terms of opening up a menu that was current topic but this is interesting and significant others which also includes students we've heard some reference today to things like student co-production and obviously students who enable us as well as the TEL contact that's an academic champion if you like and also the college learning technology and this is a college learning technology that was nominated for an art award before that was named there and it was nice to see that of course we've worked with them we're more welcome to use the technology so I think that's just a really nice reflection of how much value you bring as a learning technologist to learning and teaching also the local infrastructure but what stops this academic development being successful so we identified a room of factors some of these were academic some were student-related some were logistical and some were subject specific so if you're teaching in a course and it's taught by several other academics and you're not using the technology in a consistent way then obviously it's going to have a limited impact students may not be able to see the value of what you're trying to do so that particular educator was trying to open up the world stage if you like for using social media to encourage students to engage with each other but they wanted to engage in a very parochial sense and then there's logistics things like scaling up interventions or if people are not teaching immediately how can they actually implement the practice what you're encouraging them to think about and then there's subject specific issues like for example comparative lack of evidence and some disciplines and enhanced learning is a suitable approach so where does that bring us to? we do feel that as a result of this course that the participants are more focused in student learning ballin technology for technology's sake they're implementing innovations and engaging in scholarship and we're really keen to support them in continuing to do this but the success of this is the impact of very much those textile factors and we need to maintain longer term contact with our academics to provide the support that they need and to help the community of that to bring them in to share their expertise with others and some references there any questions? first of all can we just thank Vicky first of all is there any questions and I'll leap up with the microphone to you thanks Vicky that's very interesting analysis of your new course are you going to make any changes is there anything that's come out of the discussion and the evaluation is a course that suggests that you need to make any changes to it? definitely I mean one thing obviously the course ILOs we need to make sure that they are bang on that they say what it says and what it does this kind of thing but the other thing is that by encouraging people to engage in evaluations we ran into other factors like for example the cohorts of different learners in the course have different times of the year some of them are students they can work with and others don't so broadening the evaluation mode to a scope and exercise is going to actually raise the whole of academics and get information from them and sort of almost build the kind of findings we are doing in a study at a later stage and so I think it was perhaps up to me to expect that everybody would be doing the evaluation thank you any other questions? thanks Vicky that was very interesting I'd just like to ask given that you are taking quite an innovative approach to the teaching this course have you also adopted some innovative assessment approaches and that's it that's a very good question and no we haven't the basically the assessment is the evaluation and they have to write it up so I guess it is innovative in the sense that it is reflecting on their practice and something they've implemented as a result of the course but it is a written account and so maybe we should be thinking about being more innovative in the sense that maybe students could do digital artefacts to help them mirror what they could actually implement their own practice so yes we're quite standardised at the moment but I think that's a really good point any of this? sorry Vicky this is just a clarification question how many people were there on the course you may have mentioned that and did you interview them all or was it just a subset? no I've lost over that and that's a good question we had basically we had three cohort awareness so we had eight and eight and then 16 and we haven't actually served the last cohort yet so we had we had four and four responses we had about 50% response rate to the person in the survey in terms of the 50% and the interviews I think we had five or six out of eight so it wasn't bad that's not bad thank you that was how we did the batch you're just making me run aren't you? hi I just wondered where the course sat in terms of the rest of the PG cap so how many weeks it was how many credits? at what point would I under-tight the module was it optional? that's a very good question as well it was an optional module there are ten credit courses and there was about five or six of these I'm not quite sure because I know they're introducing more at the moment but they're in the region of five or six a couple of core modules and then the rest were optional really but it built on the first year of a PG cap that also had a unit on technology enhancer learning so it was about just being aware of the possibilities for technology enhanced learning and broadening horizons whereas this was trying to encourage them to be much more critical so in relation to the literature and also in practice any final ones? I just have one at the end actually just a comment I guess for you Vicky Bielser for the room some of your results revealed that actually if staff aren't doing it in context they really struggle to embed it and it's a problem we find as well so I wondered if you or others in the room had thought about ways to try and counteract that to see if you can get learning to the right place where they're actually in the context they need to use it this is just one of a number of different approaches so obviously there's the just in time approach to working with people we've got a number of funded projects where we work in partnership with people and support them and I kind of add space and we've tailored that over the time to focus on their needs unless about what we want to tell them so I think it is really important that you do need to to work with them at that point so it's one of a sweet different approaches so it's a sweet approach great okay okay so thank you Vicky just put our hands together again and the final session for this block we've got I guess Lisa who's going to open up as well to talk about computing chemistry and business oh my thanks very much for bearing with us myself and Mark are delighted to be here from Dublin City University to talk a little bit about the what works and why project that ran recently until June of this year as you know there's many national and international reports highlighting the challenges facing higher education but also the potential of learning technology and we are faced with that challenging situation of rising student numbers and increasing requirements on our time as teachers so staff development has equally been highlighted as key in all of these reports and continuing professional development for teachers from the basis of our what works and why project what we wanted to enable was the students that teachers were able to work together work smarter and work more collaboratively so as is evidenced by us I think in this room today and the sessions that we've already seen at the conference today there is tremendous work going on with technology enhanced learning I think the challenge with using technology enhanced learning to support the student learning experience is that it seems to be in the hands of very individualised practitioners and it does intend to funnel down to the greater teaching populace and the teaching cohorts so that was really what we aimed to do with this project was to broaden the digital literacies of all of our teachers and we did so by asking the question and exploring the question what works and why and when we are talking about what works and why we are specifically looking at different disciplines and different contexts so let me ask you something everybody here works in a different discipline different context maybe with remote learners maybe in a classroom setting so I'm going to ask you what have you found has worked for you and for your students in your particular context and I'm going to have a look and see what we found with ours so I'm going to ask you to go to menti.com and this is a mobile polling tool so if you want to pull out any device that you have be it a laptop or phone, tablets, anything at all and if you can go to menti.com and use the code 386674 what you should find on your screen is an option to input up to three learning technologies or tools that you found particularly useful as I say everybody here is probably an expert and is engaged with using technology enhanced learning what we were trying to do was to spread that out so be interested to see what exactly you are using has everyone got the code and the URL and I'll switch to and all of your writing appears to have gone on top of one another okay I will take anyone who tries to shout out one or two tools in the old fashioned way anything else that has worked particularly well any web 2.0 technology or any mobile polling tool that has worked better than this one near pod near pod the reason I use Mentimeter and trust me on this one worked perfectly earlier on it enables the entry of free text so answer garden would be another one that would enable you to support the free text entry as well okay let me just take you back for a moment thank you for your old fashioned input I'm loving this laptop so back to where we started so there are a lot of different technologies that could be used what we wanted to explore in our community was what was being used so we actually went to the students and we asked our students what did they find, what technologies did they find in their classroom and what particularly worked for them so let me just cue this up for you sit back, relax set, I use on business so we use a lot of translating software and crazy illustrations fair for full screen here set, I use on business so we use a lot of translating software and crazy illustrations fair point I'm just kind of YouTube as well for videos. By lectures, it's mainly PowerPoint slides that I see them using, and some of them use a drawn pad kind of thing for drawing. And that's it, actually. In the classrooms, mainly PowerPoint. To be honest, that's about all I can remember, and some places don't even have that. PowerPoints, and at the odd time, there's smart technology. So smart boards for using in primary schools, so we get freeped on that quite a lot. How to use the smart board? Generally, it's just PowerPoint presentations. Since the general team, so while we do have pockets of expertise as regards digital literacy amongst our lecturing staff, what the students are actually seeing is mainly just plain old PowerPoints. Thanks, Lisa. And that was the challenge we were faced with, and we all have our expertise within our institution. That was actually filmed in DCU, but this was a partnership project done with that loan, Institute of Technology, Dundalk, Institute of Technology, Ammanut. And the same thing was seen. We all have our champions, but the majority, the vast majority, were still using the basic PowerPoint and basic of no learning technologies. We have that. The more it's adoption cycle, as you will be familiar with. So how do we get around that? Let's just come back and hope this smart board behaves itself as fast as we go. So what did we do? Just to put the project into perspective, we ran, it was between the project that Lisa alluded to between January 4th and May 31st. We ran 34 different workshops. That's over one a week, right? On average, I'm just making sure I'm actually facing this the right way to get it. We intensified around our tail week technology enhanced learning week, but on average, as I say, it was one a week. But that wasn't, even though we were acquired as part of the funding for this project to do a tail week, the most beneficial aspect of the project, orientated around these tail projects, and then also the teaching groups. So just to explain those to you, we had 26 innovative projects which we funded, and I'll explain that in a little second, but we had 12 plus three just to explain those numbers there. 15 different teaching groups. And essentially what they are, are peer mentoring circles. All right? Where we had lecturers talk to one another about their teaching. We've no problem getting academics to talk to one another about their research, and to set up collaborative networks around the research, but I asked them to start talking about their teaching. It's a different game, different ball game altogether. And the reason why I've separated out 12 plus three, 12 of them are individual teaching groups, and by that I mean they were located and kept within the institution, either at Lone, Manute, Dundalk or ourselves. Three of them were inter-institutional. All right? With the whole idea being we had 15 different teaching groups that came out of it. Starting off with the workshops, there's a selection of them there. I don't expect you to be able to even see the screen from at the back, but we ran stuff on Google Drive. We ran stuff on e-portfolios. We ran stuff on Skype. And these were what I would call supply-driven workshops. We had an idea for a workshop and we delivered them. And they had limited traction in their experience because what happens is we're getting the same people turning up to them by or anything else. But they were supply-driven. We weren't making a long-term programmatic change and that's what we wanted to. We only had the funding for a short amount of time, but we did want a long-term change to be seen. And this is where the projects and the teaching groups came into effect for us. So just to give you an idea of what sort of projects we're talking about, and as I say, we funded 26 of them. When I say funded, it's not big money. It was 2,000 euros. And this was pre-Brexit time, so that's about £1,200. Probably about £1,700 now, but we won't even mention that, right? But the fact was it was chicken change, right? It was absolutely, it was minor money for us to plant a seed. And that's exactly what we did. We facilitated the School of Computing created online quizzes for formative and summative assessments. The School of Engineering did something similar. They used open source platforms which were web works which you may be familiar with, which is a maths database, free maths database of questions, and they tailored them more towards engineering. Within that loan, they looked at using PIC controlling boards and actually using a mixture of screencast and peer assessment and self-assessment to integrate these into the classroom. All the time, each one of these projects, and the library one I'll get to in a second, all led with good pedagogy. Technology was just a facilitator, and that was the key, that was the key thing when we wanted to fund these projects. That's what stuck out. The library, there was a number of projects that came out of the various different libraries, but the one there was our staff wanted to know how to record a decent video and they wanted to integrate video into their online resources. We hooked up multimedia students with the library staff and the multimedia students trained the staff on how to capture a good video. In terms of sustainability and overall benefit, everybody was a winner. Other projects, and there are loads up there, and again, just like Vicky, I'll let you read the slides there, but we had an augmented reality project. We had an electronic lab book project. We had headcams for lecturers using them in the barteries, all sorts of different things, and they varied, as I say, from, I think, the cheapest project we funded was 60 euro and the most expensive was, well, 2,000. But what they all did, without exception, they all planted a seed, and they planted a seed for people to get comfortable with the technology and use of it. Now, the full list of projects for anybody interested, and I put this out as a shameless plug, is on whatworksandwhy.ie and if you're looking for ideas to generate, I know there's a funding call in the UK for something, I think, with the half-case stuff, cut and paste your heart's desire. We take that as a complement where plagiarism would take place, in this case, and we're open to collaborate as well, if you wish, or share any of the findings. But the whatworksandwhy website is one of the sustainable outputs for the project. So the project is now grinding to a halt, but that will be our legacy, essentially. This is the beauty for us and it's the teaching groups. This is what made us sustainable, right? And I have to keep coming back to this because when we are doing these projects, we only have a short limited amount of time and a short limited amount of resources and we want it to go beyond the digital champions. And there's some great examples here within. The edtech teaching group, and the particular lady that we have, and I know you all have an equivalent in your own institution. We had a monarchy, right? Monarchy is absolutely brilliant. She's teaching a learning project. She loves her students, she loves her job. She's so enthusiastic about doing everything. But if I'm perfectly honest, we were bleeding Monica Droy in terms of wheeling her out of all these presentations and getting her involved in all these projects. So what we did was we got in contact with Monica. She set up a group with her colleagues where they spoke about teaching and again, I bring it back to the point, no problem talking about research, but we wanted them to start talking about teaching. And a real example of how that was sustainable then, thereafter, was we had another funding call opportunity landed on our laps and I put out a call and Monica didn't turn up to the table, but four of our colleagues turned up around the table. Four of our colleagues who were participating on this. So that was a real live evidence of this has worked. And if it worked nowhere else, that's fine, but as it happened, it did work somewhere else. We're looking at rolling out e-portfolios. We've set up a teaching group within the education sphere looking at e-portfolios. We looked at teaching online and teaching at scale and Mentimeter, which normally does work, I promise. Right there, this is what it should have looked like. We looked at that and rolling out audience participation systems and student engagement, all of which it trickled out. One person did it and then they started showing their mates. And that's essentially what it's about. So the teaching group was certainly the biggest win for us. And based on literature as well, we weren't just making this up. I think Michael Henrich from Australia works on if you want to embed good practice and you want to make it sustainable, you go where you get colleagues talking to one another about it. There are, again, other projects there. We set up a teaching group on Moodle across the institutions. We were even talking last night, Richard and myself, about how we can collaborate across institutions. Projects like that make this sustainable. So how did we evaluate it? And again, similar challenges to Vicky. We thought we were doing great work, but we weren't quite sure what's the long-term impact of it. And again, very similar, and my compliments to whoever did the timetable for this similar to Vicky's presentation. We had a short-term timeframe to actually do this evaluation and measure the impact. But we had over 160 people attend the various different events that we had made. And the evaluation framework that we used was from Coolbear and Hinton, and that orientates around four different factors. And you'll see there from the screen it's on reach, right? It's on impact on practice, impact on learners, and then of course impact on project teams. So immediately you can see project teams without the teaching groups standing out straight away. The impact on practice and impact on learners are that little bit more straightforward with it. Let's just look at our impact on practice and various different comments that are now coming up on the screen. Again, I'll leave you time to read those comments. You will see there, and please feel free to finish reading the slides. But yes, I like using Feed Forward in it. It made me more aware, it gave me confidence to try new things. All of these things were all very, very positive comments that came out of it. One question that was in our survey that really stuck out for me is, do you intend to investigate further? Now what I'd like to do is ask them the same question 12 months time and say, did you actually do what you said you were going to do? But over 90% of our respondents said yes. So we were thrilled with that. So that to the powers that be, and they were thrilled as well. If I can just, I'm going to wrap up here now just with this slide talking about sustainability. But the teaching groups, the monarchs of this world, they are now embedded within the faculties, embedded in the library, embedded in engineering, embedded in computing, and so on. And the evangelists now are not alone. That's the way we would see this. In terms of sustainability, we produce a series of briefing guides and even a very catchy title, which I have to give full credit to Dundalk Institute, Telltales. That's all up online, up on whatworksandwhy.ie. Please cut, paste, plagiarise, harvest, whatever term you want. We've just been delighted to actually share this with you. I'll wrap up with this particular slide. We know from the literature that integration of technology-enhanced learning for it to have a long-term impact has to be led by pedagogy. You have to embed it in course design. The whole idea of just coming along and having a pop-up workshop is not going to happen. We actually managed to touch all faculties across the partner institutions at Dundalk, Manuth and ourselves. All faculties at all different levels. What we did was we planted that seed by giving them that small pot of money or by pulling them together to do this workshop or pulling them together to do this teaching group. We gave them the opportunity to have that critical reflection. The stimuli was the cash. Then we've seen significant impact on practice. We set out, and this is my contact details and Lisa's, and again our colleagues that can't join us today from at Llan and Dundalk are listed in the brochure, but we set out to change a mindset, not a skill set. That's what we wanted to change with the mindset, and that's what we feel we achieved. Thank you very much. Thank you, Lisa and Mark. Any quick questions? We've got time for two before the end of the session. I think your idea of the teaching groups is particularly interesting. If that doesn't come up, we'll discuss that in a bit more detail in a minute. Let me first ask you the question. You said you kept 160 academics in total. What fraction of the total academic is it? Having tried seeing these things, is it 10% or 20%? It's varied because we were across partner institutions. I couldn't come up with a figure up to top of a head, but you're talking in excess of 30%, 20% maybe. It is a substantial figure. It's a substantial figure. It's a substantial figure. I think it's quite good. That was a relatively easy one to answer, so that's okay. Can you manage that? Any other questions? Don't ask me another maths question now. Mark and Lisa. I've seen you present on this before, and I always love seeing you present on it because it's such a success story. My question will be around how did you sell the idea to the uninitiated and the great unwashed. Do you have any advice maybe in terms of, in your experience, what was the best way of doing that, or what worked, what didn't work? What worked for us was getting their colleagues to do the selling. Those were always going in with a pitch, and the pitch is what's in it for me, trying to solve a problem for them, because I can preach, and you know me at this stage, I'll talk the hind legs off a horse. But they're sick and tired of hearing me. Whereas when they hear them, and again I mentioned the colleague Monica, when they hear the Monarchies, when they hear the Jerrys, when they hear their colleagues talking about it, that works. But what we've done in the past is we've taken quite a lot of the, let's just say less competent individuals when it comes to the use of technology. And we've actually sat down with one-to-ones with them and says, okay, Shane, you say you don't know anything about Moodle, so I'm going to sit down and tell you how you can integrate Moodle with it. And we held our hands, virtually speaking, through the process. And then when everybody else in the department actually think, well, God, even Shane can do it. Shane can do it. I can certainly do it. But all the time, every single case study, every single example that we did, we made case studies out of them. We promoted the hell out of them. Monica gets as much glory as we can give her. That particular example, we let everybody know that Shane is doing mentor, student polls, or is doing online quizzes in Moodle. And that sort of stuff makes it a little bit easier to adapt. But the pitch really comes from their colleagues. And if we can get that, it's easy to sell the story. We had a number of just drop-in workshops, so we went to them. We made it easy for them to talk to us. And as part of bringing it to them, and also as a result of that video when we saw that PowerPoint was our primary presentation tool or teaching tool, we developed a number of tabletop card menus which featured different tools that could be used by students and equally different tools that could be used by the teachers. And they will be ready now to go out into cafeterias and public spaces. So even as people are informally having a cup of tea, we're bringing learning technology to them. It's our version of Google Ads. We want to place it where they see it. OK. Well, thank you very much to all four of our presenters today. Just a final round of applause. And then you've got a 15-minute break to get your next one. So thank you all. Thank you. Thank you for your programmes. So I'm very minded to let people speak for up to 10 minutes so they get a little bit extra in order to share their views with us. This session is being live-streamed to YouTube. And so therefore, with the questions, we have a roving mic. A gentleman in the auditorium is going to handle that for us. So I'm going to start off by welcoming my partner in crime from Oxford, Zavlor, who's going to talk to us about neuroscience learning and VLEs. Talk is being given by Victor Nilsson, who is standing in for Grace Brown to talk about the potential of a mobile learning app for transferring learning from the classroom into the workplace. It's been a long day now, so I'm going to try to disconnect a bit from all the connectedness and not use a PowerPoint. But my name is Victor, and I come from Ashrid's Executive Education. I work together with Grace Brown, Karina Shofield and Vicky Hopham. We try to develop a test pilot of a learning app. To get a better understanding of the app, I want to explain some research we've done at Ashridge that is very relevant to the app. And it's more specifically studying the transfer of learning. How can we get the learning from the classroom into the workplace for executives? One of the key findings we have found is very much up to the programme to find and the practical sessions. If we use feedback processes continuously, it's a bigger chance to enhance the learning experience and get a better learning transfer. A key barrier is more related to the executives' work environment. If the executives don't have time to embed their learning, they will be less likely to get a good learning transfer. So with mobile learning, and the rapidly changes that is within the technology today, we wanted to see if we can use mobile learning to get a better learning transfer. The business environment today is very fast-developing, and this is an increase of demand for constant and continuous learning. The executives are requesting learning material at any time, any place and anywhere. With today's technology, we have a solution to this request. We can even design the programmes individually for each request. So with the research we've done and insights in mobile technology, we aim to do this test pilot to follow up the learning, analyse how executives enhance the learning in the workplace. We worked together with an external app developer called Retenda and developed an app that we called Into Practice. We had three research aims to capture the obstacles and opportunities of the learning app, investigate just-in-time element of mobile learning to see how to get on in their busy working life, and the most important is to collect the data in the moment where the executives are applying the learning in the workplace. The app was based on gamification, so they got questions, questions were sent out to them, and if they completed these questions, they got a reward. These reward were learning material, developed pedagogically and engaging. For example, it could be Olympic training to improve brain performance. In this, we aim to create new learning with an element of game and play, and in that way, enhance the learning experience. The just-in-time element is more related to that we all use the mobile phones, even executives. This and the fact that executives don't often have time to get on to learning, and that would be a learning transfer issue. We aim to add exercise of reflection and self-assessment when executives engage with this app. In the moment, it is, for me, the most important, and that is that most programmes today are based on retrospective assessments. People are giving programmes after they've learned this, and it could be months, it could be weeks, it could be anything, but we wanted to get in at the moment when it happened. With capturing data, we could get instant feedback from the workplace on how executives enhance learning. This could be a very good assessment opportunity for teachers, but also for client stakeholders to measure their return of investment. We had 47 participants from six different leadership programmes at Ashridge. They participated, so we started with a pre-survey. We aimed to catch their engagement with mobile phones, what apps are they using, what phones are they using, but also obstacles and opportunities that they could identify with this app. The app captured enhanced learning in the moment. Notifications and questions were sent out at six different time points. It was week one, week two, three, four, eight and twelve. These questions were where did they apply the learning, where did they use the app, did they feel confident based on the programme and any situation that was based on the programme. We also had a post-survey afterwards to assist the app, review the role of the app and if it was added significant value to the Ashridge programme. The findings we found was that the executives were overall positive. They could identify the need for assistance in the area. It was a suitable tool and they really appreciated the discipline that the app gave. They had time to reflect, stop and pause. The app also captured confidence. People wrote down that they did get better confidence. They overcame difficult situations and they had better results directly at work. The participants applied learning quite stable over the twelve weeks, except for one week which sadly was Christmas. According to the post-survey, they found a really good convenience of the app and this was directly related to learning transfer by the app. They found it useful and engaging and they managed to recall, reflect and apply and they really find a provided value with this app. We had some very big difficulties that we need to take on for the next step. We had technological issues. We didn't work with companies, IT teams or technical teams so we were giving them support ourselves. If you had a Windows phone, for example, you couldn't use the app. We also had the problem that notifications and questions were sent outside of work sometimes. Some people could get notifications on Sunday morning so in that way we failed to capture the data in the moment which was one of the major important things that we tried to solve with this app. But the conclusions are that it would be valuable to work with the whole client company so we could get everyone on board. This could overcome some technical issues but also encourage more engagement. We like to produce a checklist and missions that the users can add so today I've done this, tomorrow I'll do that. We also add something social so that the participants can engage with each other via this app. We didn't have that. This was a pilot and overall we are happy. We are looking into how to develop this to the next stage and I would love some feedback to that. Mark towards how we can get people engaged with the app because it did drop. We had 100% from first week and after 12 weeks it was 45% at the beginning. Could that be anything to the rewards? People said they liked the rewards so any feedback that you have to make this more engaging dies all from me. Thank you so much. I'm really curious. It's a really interesting audience that you're targeting at executive level. I'm curious if you can share a little bit about the needs analysis and the pre-work that you did to try to determine the app and try to cater that course to that level and that target. The main target is executive. We are an executive education so we are doing leadership training for executives so that was the easy answer to what target to use. But it's also because they are the leaders in the companies and if we want to get into a company. I guess I'm sorry. I probably didn't articulate my question very well. How did you? It's just because you are already an organization that is an expert in teaching experts and executive level. You felt like you could tap into their needs and you knew already how to approach them and what would work for them. Is that what I said? I'm just interested because I came back from Asia and did my own regional needs analysis on that audience so I'm just curious to see what needs analysis. That's all I need to know. Thanks. What were they doing with the app? How are they doing with the app? It was mainly learning material. So we had different types of articles. We had how do you present good? How do you inform your being performer? It was also different types of questions. How can you use this today? To reflect, to make them reflect. How can you incorporate the learning this week? It is also that with the assessment how they got on with it, you are asking about confidence levels. It is a tricky question perhaps. Would you expect executives always to say that they are confident they wouldn't admit to being unconfident at some point? How reliable could you feel that there is self-reports might have been? I feel like since it was in the moment that it is more reliable than if you would have asked a few weeks later. So it was a situation if you have overcome any difficulties. How confident were you overcoming those difficulties? If we could access that at the moment and happen, it happened, I think it is more reliable than if you would have done it later on when you think back to it. I can't see too well. I have a question now. I am sorry. Is there a mic around? From the staff perspective, I am assuming that there was some monitoring that was taking place, looking at what the students were uploading using the app or doing on the app. Did the staff use the app as well? If they did, what were some of their feedback? We did use it on our own staff to test it as well. They were overall very positive and find this a very good solution for our clients and the students we have. It was overall positive there. You said it was adjusting time so immediate responses immediately they can upload things onto the app whenever, wherever. But staff usually works Monday to Friday and 9 to 5. Are they getting feedbacks this Sunday or out of office hours as well? That was the problem we had. They got the feedback and they engaged with that outside of work where we didn't want them. We wanted them to do it at work. But it came outside working hours. We don't know if that's a technical issue or if we just didn't think it through from the beginning. That was the problem. That's something we're going to look into in the next step. Thank you very much. Is Derbyn Murphy? Is Alice going to come to you? Alice is going to speak to us about using a learning object to improve the understanding of the daily integration. Quite an awful. Hey, I'm Alice La Roy. I'm from Brunel University. I'm part of the learning technology team. We're a small team and Brunel has a reasonably mature level of usage of our VLE, Blackboard. What the team primarily does is support staff and develop staff to use more of the VLE. We're doing things like teaching people the basics of Blackboard and then working with people that want to extend the uses of Blackboard and sometimes going into project level side-by-side working with academic staff. What we're finding is that now with a mature level of usage of the VLE we're needing to provide support to other users of the VLE which is more at an administrative level. We've delegated responsibility and done targeted training so school, office administrators can manage the grade book area, monitor submissions out for EMA, that type of thing. As part of that work we're realizing that there's another level of support staff around the university that people that don't really use the VLE but they have lots of contact with students at the first line support level and the students have questions about the basic things about the VLE. The first step here is talking about the VLE integration which is primarily how students get access to the VLE via the link between the student information system in this case. I just want to present a little scenario to you that's probably familiar to some of you around this time of year. The student says they don't have access to the VLE. They've done everything that they're supposed to do to have access to the VLE but they don't have access to the VLE. They might go to see their school office probably where they're spending a little bit of time at this point and people always want to help students that don't have access to the VLE. So they might say when you go see the student centre they might know why you don't have access to the VLE. The student gets to the student centre and in this case the student centre for us is kind of like the hub or support enquiries or whatever you want to call it it's kind of one stop shop and the student centre is like they don't really know why you don't have access to the VLE. It certainly looks like you should have access to the VLE so I'm going to refer you to the IT help desk and the student gets to the IT help desk and they want to help the student get access to the VLE but they also can't see why they don't have access to the VLE so they won't do something like reset their password and hope that that fixes everything. It's a bit of an extreme example but these things at this time of year do happen there can be hiccups with the integration or maybe just a misunderstanding of how it works and what delays it might not be instant in our case there's a 15 minute an up to 15 minute delay before the student can access all of their modules after registering them so we do see these cases where students are passed from department to department and it's not a very good service to be offering them so we thought well what could we do to improve staff understanding of how the integration works for the VLE so we're working on developing a learning object so we're using learning technology to teach the staff or up skill the staff that don't really use the VLE a little bit about it and what we've done is when we looked at the integration it's actually really dry materials like a 20 page document that sort of outlines what status a student has to have and the student integration system to access they would get in the VLE so it wasn't really something that we could just pass over and say here read this then you'll understand how the integration works we even sort of as a learning technology team went what does that mean so we wanted to create something that made it really easy to understand at a service level and was also hopefully quite engaging so we came up with a comic book style learning object where it's interactive and the support staff can sort of hover over the different student scenarios and learn a little bit more about their case so we looked at five student scenarios where different access to the VLE might be required starting with their first enrolment so they would get access to the VLE but maybe not have any modules or courses that they could look at they would then have kind of a registration period a reenrolment period and then they would get access to their modules all going well then they might also find a little bit later weeks in that they don't have access to the right things that they're supposed to have so needing access to materials submission of work and then finishing study where there could be different requirements for students when they're completing so each of these five scenarios in the learning object has an interaction with a member of support staff and then for further information the support staff can look at kind of a swim lane flowchart diagram to see what is triggering the access to the VLE We haven't actually put this out to staff just yet so we're hoping to put it out just in time. We've got registration week, freshers week so they're happening next week so we're looking to put this to staff very soon and I hope they'll be fresh in their mind so we're going to track the usage of the learning object another way of bringing awareness of the VLE to support staff is to put this learning object within the VLE and then they'll get an opportunity to use it so we'll be able to track their usage or viewing of this item in the VLE and we'll ask them to complete a short survey just using google forms what did they think of this it's a different way of providing information to support staff in our context so we're interested to know if they like it and if they do then we might look at creating more of these learning objects for other for other skills If it's been successful then hopefully we'll have less students being referred to different places around the university I've got a few more minutes left so I'll attempt to do a live demonstration of where I'm at with building this I don't know why I'm always a bit of a gloss on the punishment I always go for a live demo when it's probably not looking right So this is the landing page for the learning object it's quite simple so there does need to be just a little bit of explanation of what the user is expected to do which is basically roll over the different five scenarios click on them and find out more information there is a bit of audio but I've just got it switched off at the moment it's more for accessibility So to make it a bit more lively the idea is when we roll over the scenarios they light up or they colour up and then you can see that that's a hot link that you can click on and the text that they're holding on their flash card changes as well it says I'm provisionally enrolled and when you hover over it click here to find out a bit more about me and when I click on that all going well what should happen is that we can learn here we go So with a little bit of audio over here for accessibility hopefully we have a bit of an authentic conversation happening between the provisional student and a member of the centre the students are asking what kind of access they can expect to what sort of systems they can expect to get access to and if I've got access to Blackboard does that mean that I can start studying straight away which in this case it wouldn't because they haven't registered for their individual modules yet they're just provisionally enrolled and then for a bit more information the flow chart a bit of a delay here will come up which just kind of says what's triggering this information and go back to the homepage like I said live demo is always a little bit of a hiccup so it's a bit brave of me to attempt that so I'll just pop back to my traditional powerpoint presentation and at this point I'll welcome any questions Yes if the student still doesn't have any access well then our team would be there to support them face to face well I'm going to assume I did a really good job of explaining that then Thank you Thank you again We still don't seem to have Geraldine who's going to do our fourth presentation so that means we're finishing 20 minutes early so I guess we can all go and be first in the TQ which is the T I think because she starts on the 415 but sometimes they're ready a bit earlier the next thing I think is the Q note 445 back here