 just about to do now so we're now recording and so yeah so the DESI project arises from that strategic goal and what we've been doing is working directly with institutions to kind of support well-planned well-structured institutional adoption of effective data usage just to give a quick plug as well for those of you who may not be aware we've developed ORLA our online resource for learning analytics which is an online suite of resources guides case studies dealing with kind of most of the aspects that we would expect that people are kind of dealing with at the moment GDPR accessing data, data quality, effective interventions, communications, case studies for how existing lecturers are currently using their data to to drive their teaching or to support their teaching so that's the first URL there is the link directly to ORLA and the second one in a manner of getting your feet wet before you jump in the pool you may want to go to that's an insight that we published recently which outlines the main steps for developing effective strategy and links directly to some of the resources in ORLA that are there to kind of support each step and so again so again i'd like to kind of particularly thank our speakers for making the time today and for for providing content of this so interesting and engaging at this point it would have been as you can already tell it would have been a very long and so i'll go through kind of each of the speakers kind of more more detailed in turn before the presentations but i'd really like to thank Linda Korwin leadership today and for their presentations so without further adieu to kind of give you all the chance to contribute sorry let's see we're having kind of audio problems here hopefully we'll kind of get them ironed out and we'll one of the the as i mentioned earlier one of the kind of drivers is really to give people the chance to to collaborate and to talk about what so i would particularly with regard to this question what challenges are you currently facing i'd ask people maybe to use the the chat room sorry i know we're having audio problems there we'll see what we can do again as i say to iron them out um but okay yeah i'll try turning off the video see if that helps see you later and so i don't know is that any better for people um we just wait and see are people having more more luck with that can you okay excuse me for a second i'm gonna try that and see if that works better okay so is that in terms of the sound is that any better now uh we've we've swapped over mics okay great okay fantastic excellent okay thank you very much sorry about that um so for this section we were just hoping we're just going to take a few minutes and we'd ask you to use the chat section just to say in a few words kind of what's what's happening for you at the moment with regard to data usage either you personally let's say in a teaching capacity or within your institution what are the sorts of challenges that you're facing at the moment just to see kind of what's happening and where where everyone where the crossovers are sorry you're having difficulties accessing data or or figuring out what to do with the data that you have um and for those of you who haven't enabled it yet you see there's a chat box down in the bottom oh sorry okay so currently you're not using data lots of data who analyzes what to do with it yeah integrating multiple data sources so is it fair to say that at this point of course yeah find time and resources getting access right space resources yeah staff unaware of what to do with the data um so there seems to be yeah a lot of of there seems to be a recognition generally for the amount of data that there is but really being able to either get the institutional culture or the ability or the time and resources to really know um kind of how to maximize the value of it is that a fair a fair statement that's kind of what's happening for people oh thank you very much excellent okay so i mean it's very interesting to see we've we've a real range of institutions and of institution types and of individuals here today so i think it's really interesting how similar the kind of challenges we're facing are and i personally i would take that as a very good sign as well if we're all having the same it means that the we can all do together to get through it so at this point i will thank you very very much it's fantastic to have that um and thank you all for your contributions so at this point i'll introduce our first speaker who is Linda Hannah for the University of Essex i first heard Linda presenting at DigiFest in Birmingham in March i think it was and i was really struck under under Linda's kind of guidance Essex have taken and i mean i know Linda would probably correct me on any of this but Essex have taken a very structured and carefully thought out approach and one that has involved kind of culture development and very much the kind of thing that we have endorsed as part of the DESI project so i'll pass over control now to Linda so Linda you're in charge now is your sound on there Linda sorry Linda we can't hear or certainly i can't hear you there that should work can everyone hear me now hello hello can you hear me oh yeah okay thank you it just seems to have taken a couple of seconds to do what it was supposed to do um fantastic thank you very much hear me now okay so i'm just going to do a very quick run through um talk a bit about our project which as Lee said it's been probably ended up being a bit bigger than we thought it was going to be even though we did plan for it to be a four phase project over four academic years so we knew it was going to be big but i think it's turned out to be more than we thought um the first slide you're seeing here has got the name of our dashboard system that we created which is called the learner engagement activity portal um and one of the things to point out on this is we talk a lot about learning analytics here at Essex we talk about learner analytics and that's because we wanted it to be very student centered so at this point it was much less about the bigger daytime if you like in terms of management information it was much more about how do we support our students so it was deliberate that we call it learner we call it learner analytics so if i try and move on yeah there we go um so what have we actually done we've introduced some student dashboards so now staff and students see academic activity data that's things like usage of our vle we use wiggle um how often they use our online coursework submission platform how often they use our lecture capture system um we've also got attendance monitoring so how often people have tapped into events um all of that's put together and weighted and given an engagement support so these student dashboards are out there now and they're being used across the whole university so that's around 14 and a half thousand students uh using it um we've created an ethical use of data policy so some of the comments that i'm saying about GDPR we spent a long time thinking about talking about GDPR and they eventually culminated in this ethical use of data policy that makes sure that as far as we are concerned we're compliant with GDPR and we're being very open and transparent about the detail that we use how we use it we've created new ways of working so this is very much a cultural change project with a technical element to it rather than the other way around and certainly over here um the universities that i've talked to in mostly scotland england summer wells um have come at it from a technical point of view some of them so it's the IT people who are leasing on it but actually it's much more about the cultural change so the new ways of working for staff very much so but also we're trying to help students understand a lot of data that we didn't have access to before um and one of the most important things we did was to listen and respond to our staff and to our students although it's a lot harder to get students to to talk to you um but we listened to the staff so the timeline on the project as i said this is a four phase project we had the first phase in 2016-17 that setup phase was recruiting a project team we already decided we would have a dedicated project team so there's me as a senior project manager and we also have a business analyst and we had a project support officer at that point and since then we've also gained a data analyst and a graduate intern as well and as we've needed staff we've had a budget for the four years and as we've needed staff we've got staff the setup phase also included going out to the market and looking at what was out there in terms of software deciding whether to build something in house whether to use disks um learning analytics solution or whether to use one of the commercial supplies which is where our business analyst came in very handy and in the end we decided to go with solution path and their stream software and the reason that it ended up being solution path although they didn't have everything that we wanted they had a good students interface and we had already decided right at the start of our project that students must see everything that staff see that transparency was really important it was pretty much the only solution that's allowed that for us so we signed up with them for a year initially and we just signed up last year for another two years um so that was quite a lot of work to get to that to the point where we had a working dashboard which then phase two and then actually we could pilot we piloted with three of our departments about 900 students um I think around 70 or 80 members of staff we did that for a whole year we had business school students we had English literature students and we had biological science students and we had different year groups so we thought that would be quite a good test pilot went well um and so we decided for the next phase we would roll it out to the whole university but it wasn't a business as usual situation we realised there was still a lot to do with the system um so we called it roll out and development we made a voluntary we said we told the 18 departments they could choose to use it or not um all the team did choose to use it so we actually had the whole university using the system this year and then next year it's transitioned to business as usual so we've started work this year on identifying the new business owners of the system and the new technical owners of the system and starting to transfer some of the responsibilities to them so some of the some things around governance and future vision and whatever we're talking to them so it's a four year timeline that so let's do there was a lot of thinking went into it and I think that's been really useful and being able to develop it in this iterative sort of way so that we've learned from one phase and we've used that in the next phase that's been very useful this is a new approach for um certainly most of the staff of the university we've put a lot of new data in front of them this is just some examples and some of these are from the stream software and the dashboards that we show um the one on the bottom left hand side is a tableau report that we've created so along with the software it gives us we've also created a suite of tableau reports that give our staff a bit more management information but also individual student information at a level of detail but not that software just can provide this relief so this is the kind of thing that we're now asking people to look at that they didn't ever have to look at before not only to look at it but to interpret it understand what it means and then take some action so this cultural change idea um that's where this comes in we are now asking staff to do things they didn't do before and do things in a different way and we have to help them understand what they're seeing and what to do with it so we've had to build some policy we've had to build some guidelines procedures good practice around okay when you see a certain pattern of behaviour with the student do you do anything and if you do something what is that thing that you do so you've got a whole lot of our interventions our personal tutoring policy our attendance policies all of these policies have been influenced and changed by what we've done here but one of the crucial things that we did was to start with some big decisions so we started right at the very beginning to see this is about supporting students we came from a point which I think a lot of people come from which is we have to do something about continuation and degree outcomes so we started there we thought about predictive analytics and we thought about and how we could try and find the people who needed more help but we thought it was quite cool to stay up front this is about supporting students because we did get concern from staff that it would be used to monitor them and their performance so we had to say quite quickly no it's not about that it is a cultural change project it's much more it's much wider than just implementing a bit of technology we decided that needed a dedicated project team and for our point of view that's a very good decision because not only can we deal with the technical side of things but this wider policy information sorry my screen just changed and I don't know why. Not only deal with policy and we could deal with things like GDPR and we could talk to them all the way and so there are a lot of questions that came up from staff and from mostly from staff actually students didn't turn up here about getting all of this information the back and all of this stuff looked at. Staff wanted to know how well if at the time was this extra work it wasn't extra work it was actually making their life easier because they had more information about the students and they could get information much more quickly in the one place that they used to have to go on the place to find. They worried about what students would think they worried if it was ethical so we had to spend a lot of time on transparency and on making sure everyone understood that whatever staff saw students would see too. We have questioned the models that we're using to calculate engagement and we're doing some work with solution path this year and with Nottingham Trent University we use the same software to try out different models perhaps for different cohorts or different types of courses. Once people got used to it and liked what it did they asked us a thousand questions about can it do more things. We've had to be quite firm and manage people's expectations and not have too much scope creep coming into this because it could do a lot of things but it's not going to in the short term because we have to get it right first. People want to talk about what good engagement looks like so we give people an engagement score and then someone has to look at that and think about is that what we expected. That's challenged our departments quite a lot because they've had to think about what engagement they expect what exactly do they expect students to do in terms of the BLA and in terms of lecture capture and all the things and stuff. Some of the work we'll do next year is around this and departments thinking about engagement. Everybody wants to know what's the impact so even though we've only had it in the whole university for a couple of terms everybody wants benefits, realisation and impact whatever and I think I say it's too soon. People want to use it too so people like the students union are coming to us and say well can we put our data in there as well can we track this we've had to again say no because a lot of the decisions were made earlier that this was about academic activities and then the name we learn the engagement activity portal it was it was agony getting the name but it did matter and we've just surveyed some students and found we've got a really really high level recognition of our icon and our name so that's quite freezing it was quite painful experience to get the name in the first place but it's something for people to latch on to that they can talk about so we're now on this new leap team even though our project is actually called supporting student success through learner analytics we're now the link team let's see if we can change this what we've learned then is academic staff do see the benefits of having this information at the fingertips sharing with the students and one of the key things in our software is that people can make notes of meetings with students and share them with the students you have to be a bit careful obviously about what you're writing it's all transparent but that's being really beneficial you don't have lots of emails and follow-up emails because it's all just there in the system students need to have useful information in there to make it useful for them to use it so we've been working hard on getting marks current year marks into our system because along with attendance information and a couple of other things that's what we find students are interested in looking at some students are much more engaged than others the ones we get telephone calls from or sorry not telephone calls we have a an inbox the queries are from the students who've got 99 percent attendance and think they should have 100 rather than from the less engaged ones so that's a challenge digital skills are very important when you're approaching something like this but a good use of interfaces by father most important thing because if it's easy if it's like any other website or any other interface that you navigate your way around then people will use it and we've had no trouble with training people they don't need to training actually because it's so intuitive to move around and that helps a lot so the skills that people need are about digital skills there's skills around understanding engagement thinking about engagement and interpreting what they're seeing and then knowing what to do after that a collaborative approach which is around my slide again a collaborative approach with our academics works and we've found that we have had a lot of collaboration we have a lot of cooperation with academic staff and with students and with the students union so we haven't had any problem with that at all and I'm afraid I can't get back to my previous slides so thank you I'll do two very quick things then everyone will challenge the data the data is very hard to get in the first place and then everybody will say it's wrong so the data is a massive challenge and good communication is absolutely critical which I'm sure everybody knows but we've spent a lot of time working on our communications plans looking back and checking that as well using every communication channel that our discourse on okay I'll finish there because I don't want to talk for too long so I've got some contact details on there if people want them I'll hand back to me control that's perfect thank you very very much Linda can you hear me right yes I can excuse me sorry for the the gremlins excuse me we always need a little a virtual crook no not at all they do down to sheer digital incompetence nothing else and I'm sure the people no doubt have an awful lot of questions for you I see have one question has already come in through the UNA sec and so from James how was the waiting decided for VLE interactions lecture capture interactions attendance what was the criteria for giving higher and lower weightings we didn't decide on that that was done by our machine learning exercise so with solution path the first step with them is this thing they call the foundation report where they have a year's worth of data and the data that they have from us I see there's a question about attendance the mark so the other things that we have been we don't have marks in the model we have in the model attendance data we have VLE data so that's logins to moogle but also accesses to course materials on muggle so there's a couple bits in there we have use of our university PCs so logging into university PCs but not personal devices um we have use of our lecture capture system as well which is an in-house development system so we have a few things in there we gave a year's worth of data to solution path who ran did some machine learning stuff which sort of skewed out the model with the weightings and that was fine that was fine for the pilot and that was fine for the first year but now we've got lots of people coming back and saying our department's different we need a different weighting so we're going to run that sort of process again with subsets of the data I think there's an exercise that we're going to take part in this year to see what that does to the model so we're trying to try to make the model work not something that we decide on through intuition or whatever but something that's based on data because it seems like the most reasonable way to do it um I see it's a question on recording attendance we have got some readers and people tap in they've got cards and they tap in and I can't remember the name of the company that provides those readers but there's technical people if anyone who's interested that I can put them in touch with so they tap in and all of that data goes into we have a data warehouse so I suppose that's another important thing which is the data warehouse has made it so much easier to do this because but then we get data out a bit more easily so all of that goes into the data warehouse and then goes out into the software um which is housed on our servers here um it's a question on disability and social economics this is we don't have any of that in our system that was another of our early decisions was that this is purely about academic activities once you are our student so there's nothing there about previous attainment it's kind of day one situation you come here and you treat it the same way we have a lot of pressure to put in that kind of special category data but partly from from our gtpr quite a few and partly also from a not biasing anything point of view we haven't put that in so we made a decision not to do that we could um we have thought that at some point once we've got lots of engagement data we might do someone on alliance analysis around that to help inform other decisions but no not in the dashboard just it's the very little personal data in there but that's the minimum personal data in there thank you very much a few more questions there oh sorry uh sorry i know we have a few more questions there but we might just push on for now yeah and if people want to contact me then that's that's great just do it thank you very much i really appreciate it and so we might just push on as i say if we have a bit more time at the end we may come back um so i'll pass over to cormac i don't know anyone sorry i'll pass over to cormac in just a second so sorry cormac you should have control there and so i was down with cormac in march i think over and go away and what's really really exciting about the approach that himself and a tain are doing and this is something that's very very dear to me is the fact that they're using ordinary available software to really do incredible things and i think there's an awful lot to be sent for that approach rather than starting out by investing a hundred grand in a platform that's all bells and whistles i think the the approach that cormac and a tain are taking to really drive proper student feedback you know effective pedagogy using ordinary tools is really really striking so cormac i'll pass over to you perfect can you hear me okay thank you very much is my sound okay yeah absolutely can everyone hear great um and do i have control yes i do okay perfect um hi everyone so myself and a tain are down gmit and we've been working at harnessing student data for personalized feedback or at least that's how the project started so the initial aim of the project um we'll get on to who the students are in a little bit was to take the data that we have and give the students an overview of their own performance so you can see on the slide there there's a few templates of different uh feedback sheets that we've given to the students but the basic idea behind it was that we'd be able to give each student a snapshot overview of everything they're doing because the data exists there so all we have to do is transform it um but to give you a bit of background there's about 300 students in first year who do chemistry and there's about 500 students who do maths so there's an awful lot of students and as a result there's an awful lot of information about those students there's 10 different lecturers deliver labs the lab happens 20 times a week and they're in groups of 16 and that group of 16 contain any of 10 entry or maybe 12 different entry routes going on to eight different courses ultimately so it's quite a diverse set of students and so there's a lot of different bits of data and it kind of lends itself to well you know if we set them up to do what do we ask them to do is it working so it was all about feedback and I skipped on one slide too many here having control of these slides but to give you some idea of where this information comes from in a typical week what do we ask of our first years so in the case of chemistry they're asked to watch a pre-lab video then they're asked to do a quiz then they're asked to do the lab itself then they're asked maybe to watch another video on the calculations then they do another quiz and the calculations of their own lab work and then maybe we give them some feedback and that's just one week and they've got forward they're doing physics chemistry maths and biology so they have quite a bit going on in a given week and so the questions that start to arise are well you know what can we do with all that information and also is that a fair thing to ask of the students so this is our typical workflow and if you put it together by the time the students get to week six you know they've gone through all of these different uh tests pass fail and usually or as it was they just get some summative information and what we wanted to do was make that holistic and give them all of the information so that they can make informed decisions so that's where the analytics part of this comes in you start to wonder like what can analytics offer you and analytics as Lee was saying at the start of this all of this is done through excel and it's done through microsoft word and i have occasionally straight out into mini tab but that's not a necessary thing it can all be done through excel but if you think about our first year module an individual first year student might have 50 data points per term just solely in terms of assessment and attendance they do 20 quizzes 12 labs some theory ca's a lab exam and attendance at the different labs maybe tutorials so that's 50 points per module per term nothing to do with how they're engaging just have they completed the necessary uh items and so it's hard for the first year to keep track of what they're doing and it's also hard for the lecturers to keep track of the what of what the students are doing because if you think about the electoral situation either you're in a lecturing room with 200 students or you're in a lab with 16 students but you don't know anything about what else they've been doing so that was what motivated this the other part of it is that the feedback thing is kind of two ways because you can try and as linda was saying you can try and get feedback from students but they don't necessarily tell you anything sometimes they don't respond at all and when they do tell you things it's through a very particular lens so the analytics kind of provide an insight into how the students are behaving and what the workload looks like and so that's the first thing that i wanted to have a look at is when you start thinking well are we over assessing them you can look at how they behave in relation to the quizzes you can set up analytics like this so here we have over over one term how many days before the deadline did the students do the post lab quiz and you can see that that's almost a random distribution of how many days before the deadline did they actually do the exercise and what i've taken that to mean and what i think is a fair interpretation of that is that the deadline doesn't matter as to when they do the work they just do the work as it comes up and so it's become a weekly thing it's part of their routine and so they're not stressed out by it so i don't think in this sense we're overtaxing our students or over assessing our students but then you start to see something interesting there's two high stakes quizzes on Moodle one each term and you get a very different behavior you get this almost exponential growth so you can see on the right hand side there i took the last six days as you get closer to the deadline and you can see that the number of students working at it grows exponentially i.e the students are procrastinating so if the students are procrastinating that's a fair sign that the way that they view that work is different to the way they view their weekly assignment so if you set a large assignment you're going to cause stress and if you set a smaller assignment you're not and these are just the things the little bits of information that you can get back from doing your analytics other things that have been really useful are you know we can assess the results that they actually get so in Moodle we get them to put in their actual lab results and we grade those and what it means is that we can look at the lab and we can identify you know 22 percent of the students aren't getting within the expected 10 percent of the expected value and we're asking them to measure a physical constant so we know that they should get the right answer but then you can see different distributions so there's a good number of students who are within two percent of the right value but significantly off and then there's lots of students who are you know it's normally distributed uh around the correct value as well and so that tells the lecture that you know there's different things going on in the class and so the analytics are really useful but getting back to how this project started was creating feedback from lecture to student so if you use Moodle if you use any VLA VLE you'll know that there is a good dataset behind every set of student interactions you can download the grades everything else you might want and what we did from there then was we set out criteria so we set out you can see in the grid here you know what is the kind of feedback you'd want to give a student who falls in each particular criteria and then all you have to do in Excel is set up so the statement that's written there is an if statement and all it's doing is relating the grid here uh above the if statement to a particular grade obtained by a student and so you can edit that and you can say whatever particular feedback is appropriate to whatever particular grade and you can make it as simple or as complex as you want and then you can set it out into a mail merge and then you can create the feedback sheet and what's really nice about that is that all of that is contained within the Microsoft environment so in terms of you know people talk about things like GDP or and data safety all of that is contained or can be contained within the online environment you don't have to take anything outside your uh established environment there's no interaction with other third parties and so in that sense it's really quite safe in terms of looking at the feedback it becomes truly personalized and it becomes really useful you know if there's nine different feedback zones that's two million possible outputs so students aren't getting the same feedback and it gives them actionable changes which really you know from their perspective when we look at their feedback which we'll look at now it starts to change the way that they behave and you can see you can look back in that there's a few different examples of ways you can set them up or you know giving them things that they can aim for or asking them why they didn't do different things but when it comes to the feedback from the students surveying the students so 161 students responded and 94 percent of them said that they found the feedback sheets that they were getting out or that we were getting out useful and then 72 percent of those said that it actually changed how they were studying so it changed their approach to studying chemistry or maths and it gave them a sense of how they were doing it improved their confidence and it reduced their anxiety so it's you know there are single statements from single students and we don't have the data to make statistically significant claims but it's certainly pointing to something that if we were to look further into this it's definitely benefiting the students from a lecturer point of view it's really useful for the lecturers who in the lab to be able to assess a student okay maybe the student is missed two or three labs are they missing a lot of things across the course or was it just two or three labs and then from a department perspective you can send that further up the chain and when heads of school or heads of department are dealing with problem students they can look at how those students are doing the feedback is generally targeted for high attrition times of the year so we're all the time trying to boost retention okay that's just about in the time slot so thanks very much thanks for listening and if you have any questions I'll try answer them fantastic thank you very very much Cormac excuse me as as ever I think it's such an interesting presentation I've no doubt there'll be a good number of questions on it can I just maybe get the ball rolling and ask how you know to what extent would a user require like a very high level of competency say with excel and where to enable this? Actually because we ran a seminar there back in March and there is a learning curve it's definitely not if you're not familiar with excel at all you know there is definitely a learning curve but it's something that if you put the time into like it becomes possible and one of the things about it is that once you have set up the if statements say or you set up the categories you don't have to go redoing that you just do that once and then you modify it for each different case so there is a learning curve but once you guess once you get it going it's up and running you know and I think it's time well invested because you know okay it took me a few days to get it going the first time but now it's a few hours to take the data and create feedback for say 300 students so fantastic thank you very much if you wanted to stick on your your video there as well as up to yourself sure and so sorry so we have do you love playing with trial and error with the if statements in excel? So I mean in terms of playing with the if statements once you have it set up once how I have it how I have it done is that the if statement will just categorize things and then I can edit the comment sheet so I can change the comments and I can change the categories so I don't actually have to go tinkering with that if statement once I have it written for you you know if I said the four or five or six categories then I can just edit the categories away from the statement which makes it a lot easier to use because if you try editing the if statement with the text in there or anything like that it just it very quickly gets out of hand. I can see some of the questions coming up we actually print out the feedback and give it in the labs so each of the lecturers is given 16 sheets and they'll hand it out on a one to one basis to the student and that's probably one of the really valuable parts because then the student is eyeballed by the lecturer and the lecturer says I know how you're doing do you have any questions about this and actually like Linda was saying you often get questions oh I attended you know I'm down for 95% attendance but it should be 98 you know so you catch errors in that sense too. And sorry then just the time before the due date how it's affected by other modules. So for the no I haven't but for the non for the low stakes assessment what you're looking at is several thousand attempts across 12 different weeks of term from all of the cohorts who are not necessarily doing the same other modules so it's I would imagine the effect of other modules is limited and in terms of the what's available for the high stakes one I would have to look at because I don't have that working for me now but I think at the time I checked at the timetable was clear like it wasn't that they were doing other things and didn't have the time to do it. Fantastic well listen thank you very very much again it's really really interesting work that you're doing and I think there are an awful lot of applications to to a lot of other institutions and a lot of other stuff we teach out there so thank you very very much again. Yeah I mean we're happy to collaborate so I mean if people want to get in touch I know I didn't put my email address on there and but if people want to get in touch they can and we'll you know we're happy to talk so thanks. I expect this may become a full-time gig for you. That's great thank you very very much. Thanks. So sorry we'll just take it back and we'll pass over so sorry we're just going to move on. So I'll pass you over to Diane and Nev who now have control and I think that what's really interesting here is the institutional approach and the support of senior management that this project has had. As I say the team here is what works and I think having that kind of institutional drive seems to have been such an important part to kind of successful initiatives and I'm looking forward to hearing more about what Diane and Eve and UCD have learned from their approach. So I'll pass over to you guys thank you very much. Thank you Lee. Hi everyone my name name is Eve Nestor I'm the student advisor at the School of Veterinary Medicine in UCD and I'm here with my colleague Diane Cashman who's the lecturer of veterinary education in the school and we work together with Professor Jason Last who is the sponsor of our project also on our project team is Associate Professor Sue Rockard from the School of Veterinary Medicine Ashley O'Grady who's the head of the student advisory service and we work collaboratively as well with UCD IT services and UCD buildings and services. For the presentation we're just going to talk about the background to the project and the aim what worked and then our key recommendations for future phases of the project. So we're just going to go on to talk about the project itself. So the aim of our project which ran as a pilot project in the School of Veterinary Medicine in UCD this year the aim of it is twofold. The first aim was to offer early support interventions to students who may be disengaging from the program as evidenced by a decrease in their attendance. So this was the metric that we focused on throughout the project and the second aim was to offer students the opportunity to look at their own attendance data as well. It's important to note that at the School of Veterinary Medicine we don't really have a retention issue so if students start to disengage this is usually for other reasons and what we found is that this includes maybe some emotional difficulties or mental health difficulties that are coming up for the students. There are high levels of anxiety and depression well documented research on veterinary students in the veterinary profession more generally. So a typical example of what was happening was that if a student disengaged from the program this usually became apparent around exam time. We run exams twice a year in UCD at Christmas and in May so a student may have failed a module or a collation of their attendance data happened at the end of the semester and it became apparent that the student might have disengaged a little earlier in the semester and this might have led to some difficulties for the student. So we wanted to develop a mechanism to intervene earlier in this process in order to try to offer support to students at an earlier point and our tool of choice was to use learning analytics. So after much planning and discussing and reviewing of the literature we began our pilot project in September 2018 and it's just concluded at the end of April 2019. There were 70 undergraduate first year veterinary students involved in the project. We collected attendance over their 10, 10 of their core modules. They also have two elective modules. We didn't include those in the project and we collected attendance data across two semesters. How it works is that the student at the beginning of the year so in September downloads an app to their phone. When they enter a classroom where their timetable to be in that classroom at that time their phone picks up on Bluetooth beacon that's in the room and they log their attendance through the mobile phone app. The students see what we see so whatever attendance data we receive the student also receives that attendance data so they can monitor their own attendance. During the project the student advisor which is myself we offered early support interventions so the way it worked is that if a student missed more than two classes in a week the following Monday they received an email an automatic email was signed off by the student advisor offering support and just noting that they had missed more than two classes in the previous week. So if there were after three further recurrences of this pattern and it culminated in an invitation to the student to meet the student advisor and then we would discuss any necessary supports that the student that the student might need. So finally just for a hand over to Diane I just want to mention the how we evaluated the project so we conducted a survey on attitudes to attendance both in semesters one and two and in semester two we also surveyed how the students found the technology itself and using it and we also ran some focus groups as well to get further feedback from students. I'm just going to hand over to Diane she's going to talk about what worked and key recommendations. Thanks Nia. Yes so overall really what we found is implementing the piece of technology I suppose with any technological new step that we'd like to intervene in in a school has some initial set up challenges so we work very closely with with our software provider to overcome those. So what we ended up thankfully with was a system that was located in 10 venues across our campus here in UCD that students could sign into and the information successfully did collate into a central repository and from there myself and me were able to see the trends students engagement with that. So technologically it worked which was great for the few one or two little hiccups at the start and we'd have to say as well student feedback to us is that we're they were actually very engaged cohort they're very enthusiastic and very willing to give this a go and try it out and support this initiative which was which was really great very positive from from our perspective as well and the module for nature supports very positive towards this project. Really what we were able to do through this attendance data is as Nia mentioned she was able to go in and see what what the trends were happening with students. The system automated out as we mentioned these automatic emails which accumulated into an invitation for a meeting and we wouldn't have had visibility about that previously really. Historically we have say paper based attendance collation mandatory practicals for example but the visibility of that data is never shared it's pretty much held with the module for nature so it's great to have it centralized and student advisor has that quick and ready access. So students themselves they very appreciate the the benefits of coming to coming to class and and really what that can offer them and in terms of the technology and getting the most paying for their book I suppose out of that what they were reporting to us is that if the data isn't as accurate as it can be that can be demotivating for them and they can disengage with the utility of the app itself. So really another component part as well they said there's motivation from their peers if everybody was tracking their attendance that was quite a strong motivational component for them as well. So overall I suppose we were happy for a pilot it implemented it functioned as it should have done and students were very positive towards it with a few outcomes in terms of dissemination going forward across other conferences and also a lot of people expressing an interest in this particular system the mechanism that we were putting in place. So it's just quickly to wrap up our key recommendations where you're going ahead again next year and taking a lot of the lessons from our first iteration moving them forward. So really one of our key principles is to ensure that whatever data we have and we're collecting that that is as accurate as possible and this is as I mentioned be a strong motivation for students to engage and to find good benefits from actually this this system themselves as well as our own. If the data is wrong we're going to unfortunately target students not the correct students. So really whatever interventions we can in strategies we can implement at a school level we're very keen to try and make that as efficient as possible. For example I always see on my slide there in point D as with every other program classes move left and right quite quickly sometimes timetables can change from the status quo of week one semester one. So about having mechanisms to making sure that the student is tagging data their attendance at the correct location and you know these are things we can address locally as well and also as well I suppose any technological issues we had I suppose three to four week delay at the start. We want to overcome it's really important that barrier of technologies is overcome quickly and we ensure that that keeps going. So let's just summarize our hand back to Mia. Yeah okay thanks Diane so just to finish up we applied for funding to continue with the project under the 2018 Innovation and Transformation Program from the HTA and we were jointly awarded with some funding together with DCU as that collaborative project has started and is moving forward now. Our own project here in the school as part of that will scale up to several programs and several groups in the UCD School of Medicine this September. So we started with one group one cohort last September and we're moving forward now to include four groups coming September and then in September 2020 we're looking to scale up further to several schools in UCD. Thank you very much. Thank you very much guys it's really really interesting. So as you we have questions jumping in here already again we only have a little bit of time left so we might just take a handful of questions maybe if people want to get into contact with you directly afterwards would that be all right? Yes thank you very much. So what were the teething issues? Yeah well I suppose when I take the technology one of the easy ones to start off with we just had a single sign-on authentication issue it was a two-step approach with a PIN and our mobile providers had a few blocks completely unknown until we got going but we removed the two-step authentication went down to a single sign-on using the password username that they wouldn't be familiar with here in UCD. Once that was a dealt with we were fine. So from a technological perspective that's the only thing I would say as well from a minor perspective I had to do a bit of tweaking we have these beacons on the wall I should grab on a showy beacons on the wall I just had to adjust some of the settings on the actual check overall that was okay. And then sometimes like Diane had mentioned it wasn't really a teething issue but if the class had been moved so they'd not be able to update the timetable in real time so that would cause a couple of issues and like Diane mentioned as well if the data was in any way inaccurate and so if the student received an automated email offering support what the student felt and I think Linda had mentioned this as well as part of her presentation that they had attended everything and like Linda said it was interesting it would be students who had very good attendance who would be saying no I was present for everything and you know you're telling me I missed a couple of classes so yeah it's just kind of working through those things with students as well. It's very interesting how that's a recurrent theme across all three presentations and so we might just take this then as the last one from John. Any key advice on the hardware and app side if anyone else is looking at iBeacons? Yeah I suppose one thing that we learned after this year was about location so for example we have some venues where there's a health and safety concern mobile phones are not allowed and so students would be obviously they were reluctant of course they've been told off not to bring phones in and so as one of those things we just didn't account for swipe cards would have been an appropriate tool for those types of venues and the other thing to say as well is that if a student is scheduled say two slots concurrently one after the other it's very easy to forget about tagging in or checking in for the second class simple kind of cultural user type issue usability type issues but I suppose as well as trying to find that technology that's easy to use quick to implement and has scalability to it and again we talked about GD Core previous as well what kind of permission levels can be provided to different levels of individuals who require different access for example you know I shouldn't be seeing you know certain data that need as a student advisor can see how can system restrict that access and so on so it's looking for things like that when you go to have a look around at services and as well the one benefit about the one we chose was being able to see those dashboards we've seen lovely visuals from our previous keynotes it's about visualizing that data so you can actually make these quick assessments otherwise sometimes being met when loads of spreadsheets can be quite hard to process so what kind of visualizations can they offer with your analytics is also very handy fantastic thank you very much Crystal I see we've a problem that question there for me as well I'm afraid we might might skip on if you don't mind I'm really sorry but could I ask you to make contact directly with Neve and Diane and because there's just one other piece I want to get you all out in time to get a sandwich before you go back and there's just one other piece we just want to have a quick just show you quickly but one of the things that we've done as I say like a right well I think two of the things that we've seen today are that individuals and institutions across the country are coming across a lot of the same issues kind of both with where we are and with where we're trying to get and I think from from the work that we can see going on in with Linda and with Cormac and with Neve and Diane is that there are very innovative very effective solutions out there as well and I think one thing I've really been struck by over the course of this project is the the willingness and in fact active desire for institutions and for individuals across the country to work together so with that in mind we set up this padlock and I know padlock has its its its potential drawbacks as a tool but just for this purpose essentially as a notice board so we'd encourage you very strongly either now and kind of into the future my hope is that this will persist as a as a resource that if you're having questions if you're having challenges come into the the padlock page you can see the URL is there you should see the the little pink circle down at the bottom and so if you just click that you'll be thank you very much Cormac my my beautiful assistant and so you should be able to see that be able to post questions that you're having or issues that you're having or even you know things that have worked out well for you and be sure if you're asking a question or posting anything please be sure to put your email address because I'll keep an eye on it myself I've kind of been very fortunate to meet a lot of people around this sector so I've some idea of who can help kind of who with what but I would encourage you to put your email address in it so that other people can find you and to those who maybe don't have questions and I'd strongly suggest going in anyway because there things will come up that you can answer and I would encourage everyone to go into it I know a lot of people will think the stuff that I've done isn't really that relevant there isn't really that important and that's been so far from my experience everyone that I've seen every every kind of institution that I've been to there is stuff going on that would be really really helpful to other people that they often underestimate the value of and so please do excuse me this this will be as successful as we can make it and I mean otherwise really that's it I'd like to say a very big thank you to Catherine and to Colin for for making sure I didn't have a heart attack halfway through this and for for passing my my stroke medication intravenously and I'd like to say an enormous thank you to our speakers and Linda, Cormac, Neve and Diane I think you will all agree that we got really really better sound like Donald Trump we got really really excellent we got the best speakers and that I think the presentations that we had were perfect and I think that's reflected very much in the attendance and the participation that we had today I don't know whether you guys can see it but we've had kind of teetering around 55 people involved throughout the whole thing so thank you all very very very much and finally thanks to you guys for participating and for taking part it would have been awfully lonesome here with AG and so I'll sign out now and please I'd strongly encourage you please do keep an eye on Adlash as you know we've been recording this and I'm going to send out make access to the recording accessible after the fact now go and get your ice creams or your carrot juices or mojitos or whatever floats your boat of a sunny Tuesday thank you very very much everyone so good bye