 Ieisio, yna yw'r ysgol iawn. Mae ydych i'r ysgol iawn. Mae'n hyfryd ar y dyfodol, rydyn ni'n gallu bod ychydig i mi oedd yn siŵnwyr i'n credu i'r ysgol iawn. Felly, rydyn ni'n gallu i'n credu i'r ysgol iawn. Rydyn ni'n gallu felly yn siŵn yn ymwneud i'w gynllun, gyda y dyfodol, mae'r cyfrif droisi ar y ffordd leol? I was having a late night conversation with some people that I had met about our role and things we did and the issues we came across. So I'd like to acknowledge them but they were the inspiration and they started this off. It was Graham McAllanay and David Reed from Sheffield and Alex Spears from Liverpool. Alex Spears was the person who came up with the name, the Fresh Interface, so that's his. If you get some good stuff out of this, great. Anything that might be contentious is all my responsibility and my views and not my institutions. A lot of this is from my own empirical observations and conversations I have with people. I don't have all the answers. I've got some things maybe for you to think about and I have a big question at the end for you. So I don't know about you but I'm pretty sure that you probably agree that we all are very busy. And we don't have much time in our practice to reflect about what we do. We know reflective practitioner is a good thing. I don't think we really have much time to do that. What I'm going to try and give you is a framework perhaps to think about how you can situate yourself. And have some time to reflection and this big question I'll give you at the end. So cast of actors for our journey, usual suspects really I think. Educators, learners, techies, technology, the institution, the Fresh Interface and learning and teaching. There'll be something hopefully a little bit interactive later on so just try and hold some of those in your head. That would be good. So I'll go through a bit more detail some of these just to focus us onto what we're looking at. So educators we work with, academics, tutors, teachers, they have a particular skill set, a particular set of priorities. Typically I would argue that these are what they are. They have the pedagogy subject knowledge delivery and they do the teaching. They're at the sharp end of what we're all about really. Techies and the hard IT professionals will look after our systems. They have also a very important set of skills and important role to be looking after integration, security and they keep it all working for us. We couldn't do it without them. We love the technology, multiple systems, interoperability issues, bugs versions, features that are foisted upon us. Usability and training issues, an institutional way of doing things or a lack of and the important point that the technology is done. It doesn't help us with teaching and learning unless it's used in a creditological way. So a simplistic way of thinking about it which I think some people have. And I'll just add at this point that this is meant to be critical of anybody. It's trying to be a frank conversation so that we can think about what we do about some issues. This is the model that gives us teaching and learning. I'd like to unpack that a little bit. So if you think about the educators in technology, they're happy to use the technology on the whole. They're interested. They want anything that's going to improve their student experience. They just want it to work. They just want something that they can use. They don't want something like a car. They just want to get in, turn the key and drive. They don't want to be messing it out under the hood, getting oily. Techie and the technology, that seems to have a good fit and is a very good fit. They have a particular remit though and that remit, it doesn't really include the pedagogy typically. Educators and techies, I think this is a problem area because I think largely they talk different languages. Educators thinking about one set of things, techies about something else. So I think sometimes it's very difficult to translate ideas between the two. So I think that with this model, teaching and learning doesn't really happen. Not in a good way. I think it's more like this. So this is the silos that Bonnie was talking about on the keynote on the first day. So what can we do about that? What's the solution for this? So introducing the Flesh Interface. This is the learning technologies type of role. We have broad skill sets and we cover lots of areas. So we know about the software. We know about the hardware. We understand about the pedagogy. We know about the technical. And we have the people skills. That's really important. So Techno Nerd to UberNorm. Well I was very lucky. I managed to find a picture with me when I had hair. So there I am. Nerd was coined apparently by Dr Zeus in 1950. But it's been quite a derogatory word for a long time I think. Someone was bookish, studious, sits on their own in their bedroom reading their book. Wide Techno Nerd. Well technology arrived, things changed. And then the nerd was somebody who sat in their bedroom playing on the computer, programming. The stereotype of eating lots of pizza, drinking lots of soft drinks. So that's the Techno Nerd part of it. UberNorm, well things still changed. Things changed all the time. So all of us I think have multiple devices, at least one device. So my phone that I keep in my pocket, although the architecture is different, I'm fairly confident in saying that there's more computing power in this, this computer, than we sent people to the moon with. And who were the people that sent those people to the moon? Well, the nerds were definitely in there. And things still changed. Now it's quite typical, not just with our group, but societally, to see people with this computer in their hand walking down the street. It's all part of society. It's embedded now in the society. My brother, he's a build-on, he raises motorbikes. He has no interesting computers at all. But he uses them because he can see the value in them. He can look up parts, he can order things, he can find things through his motorcycles. It's all around it and it's part of us. So it's a normality. UberNorm, well because we have that sort of step back and the other skills of people's skills and interactions. So we are that sort of UberNorm over the top of the normal, just day-to-day usage of it. So I think this is a more complete picture in the context we're looking at. So the TFI, I've used abbreviation a bit later on. So I think we are the glue that brings it all together. We can talk to the people, we can get the ideas translated. So I think that's a much more accurate picture of where the teaching learning happens with the learning technologist role involved. That's all well and good and hopefully not too contentious. So just some things to stop to lead us into some other questions. So how are we viewed by other people? So who might consider us, this is the bit where I'm hoping somebody will answer me, who would consider us as techies they can talk to? Shout out, sorry, out of our list, it was our list of people. So the educators, the lecturers, they can talk to us about what they want and we can help them with that. Crazy academics, well I think that's the view that the techies have us because they know we work with the academics and they can talk to us. Learners, as part of the help desk, they don't really know what we do, but they know they can talk to us and we can help them because we have that particular understanding of what they're trying to do. This is a key one, the institution. So we're not quite sure what you do, but you seem to be something that we need. So what institution I'm thinking about, people who maybe are more senior management in a strategic level, who are a bit more further away from the actual sharp end where the teaching learning takes place. And similarly about being valued. So I think educators value us highly because we understand their needs and can help them. Techies value us quite high too because we have the knowledge we can talk to them. Learners value us when they need us, we're pretty invisible otherwise. And the institution, we're not quite sure what you do, but you seem to be something we need. I was talking with a colleague, an ex-colleague, and he was an institution where they were looking to get rid of some of the learning technologists. And in the meetings they were saying, well we can give it to the technical support people, they can show the lecturers where to point and click. And so the question was, is that what you really think we do? So we're getting to the juice now I think. The institution, it's a complex animal. I think that in technology it sits very near the heart of it. This is where we face a lot of challenges. The reality role is varied and it can be more seen yet. Of course it will be quite lowly. I did a good year this morning before the fire alarm went off. One of the things was, learn tech is a professional field. But it's not really well recognised. It's starting to become more so, there are degree courses for it, and a C-Mult of course, a professional body recognition. But we need to increase our influence and impact so that we don't get things that are just put on to us and say, well this is the video software, now you have to support it. We need to feed in before that to make sure that the pedagogy is understood and taken into account and that it is the right tool for the job. So I don't know all the answers, but I know who to ask and I know what to ask, I know how to ask. I can talk various different languages within the context of this domain. I went to a really interesting session yesterday on evidence bases and business cases. And Kiriaki Agnostopoulou, and I have practised that, but I apologise to her if it's wrong, from Bath Spa she's looking at this area as well. I look at the wider context too, because there is a wider context. Neil Morris of Leeds at the same session was talking about us speaking the language of us, of people above. Thinking about how those people that deal with the strategy talk how they think, what they do. Again, from Peter Goodyear, we need to find ways of relating what we are doing from one level to another, because I think we're caught at a particular level and we need to break that barrier. So, we're skilled people. We're people-people. But I think largely we're within those three circles, we work within those three domains. So my question really is how, this is the question to you, because I don't have an answer to this, how we do it. It's not going to be a simple answer and it probably needs more research to see where we are in different institutions. But I think it's important that we do it and engage with it, because if we don't, there's a risk that we just get left here down low and the pedagogy and the things that we can help with don't get considered as they should and we don't get the solutions that need to the best learner experience. So, we need to take our profession to the next level, I think. One of my ex-colleagues from Plymouth, Luke McGowan, helped me yesterday. He had a look at this presentation with me and gave me some comments on it to help me, which was really useful and I incorporated some of them in. One of the things I didn't have times in corporate though is he suggested that as it was the fleshy interface, as we are that, that a little symbol there for us should be a fleshy colour of some sort. I didn't have time to do that, unfortunately changed it. He said that the green one, it makes me think of a dead fleshy interface, a zombie. So, let's not be zombies. Let's not let this all happen around us. Let's do something to lift ourselves up and to make the difference we need to do. I don't know how to do that, but I just think we need to start this conversation. That's it. Anton, yeah. Thank you. Thank you very much, Ian. Now, do you have any questions? Any questions for Ian? I dodged a bullet there, didn't I? That's good. Thank you very much. The next presentation will be Calibrating Educational Design Quality by Integrating Theory and Practice. Is there a tool for supporting students who are being by some answer? So, this one's a little bit different. This isn't anything I've done yet. It's an argument for something that I believe that we should do. So, I'm going to present what I think is a massive issue for us as a sector. What I think our responsibility is as a sector and why I think learning analytics can help us. So, we have a problem. Or, certainly in the press, we have a problem. This is an illustration from The Guardian. It came from a Guardian blog post and it's a student's illustration to do with mental health. So, when I'm talking about wellbeing in this context, I'm talking about student mental wellbeing. This is what I'm going to focus on today. And this is in a response to a comment to a student from their tutor where they felt really overwhelmed and their tutor said, well, that's how you should feel in your first year of HE. Which wasn't very helpful. Some of you may have noticed there's been a lot of press in The Guardian lately. Certainly the education pages around a mental health crisis. It's reaching the THE. There have even been comments in the Huffington Post. So, there is a bit of a problem. So, on a national perspective, every seven years, the NHS run a thing called the Adult Psychiatric Mobility Survey. This is the best survey we've got on mental health in the UK. On the current survey, 15.7% of all the adult surveyed had a common mental disorder. So, these things around anxiety, depression, obsessive-compulsive disorders. That extrapolates to roughly 20% of the entire adult population of the UK. So, that's a fifth of our adult population have a diagnosed common mental disorder of some sort. But what we have is a big gap in these disorders within our 16 to 24 year olds, which are young adults in our institutions. In 1993, it was 8.4% of males and 9.1% of females. This gen to 19.2% of females, sorry, it was 8.7% and 19.2% and 9.1% to 26% of females by 2014. So, we've got this massive growing gap between males and females. We also find that it's more common among young females and unfortunately, even more so in black and black British women, that actually jumps to 26% of the population in 16 to 24 year olds. So, that's kind of the national picture. What we do find is that with regards to suicidal thoughts and tendencies, it's more likely to report this or willing to admit that they have these thoughts and these feelings and that they've tried to harm themselves. Whereas, unfortunately, young men don't. And the statistics on suicide are that actually, it's the young men that actually achieve their aim, but they're not the ones that talk to us. And with regards to our demographic for our students, which most are under 25, approximately 75% of adults that get a mental health condition have developed that by the time they are 25. And this isn't something that ever quite goes away. It's something that will come back with other stressors further on. So, there is an ongoing long-term potential issue. So, what about our HE perspective? We've had a 220% increase in declarations of a mental health illness on entry to HE across the board since 2010. So, these are only the ones that are reporting a mental health condition on entry. And that's jumped 220%. Yet, we know that also our students support services have seen an increase in 150% of appointments. And 29% of students are sharing clinical levels of psychological distress. So, an all-party, so this says, it makes me wonder, is this the tip of the iceberg? So, a report, a survey, I think done with the NUS, went to the all-party parliamentary group of students in December 2015. 78% of students surveyed believed that they had some form of mental health issue in that year. That's an awful lot of our students. So, we have a problem. University of York have done a really nice piece of work around this, and they produced a very nice report back in 2016. And they identified three key things, really. Financial concerns is something that's really upsetting our students. Difficulties in the labour market. What's going to happen to them afterwards? Are they getting jobs now? Can they get jobs in the future? This is echoed by the Institute of Public Policy report that's come out in the last couple of months. Young adults that are working part-time are 43 times more likely to suffer a mental health condition than those that are working full-time. Our students are graduate students that don't get a professional job on graduation under-employed. They're also about 20% more likely to develop a mental health condition than those that are employed appropriately. It's not a good picture. When we add to that the pressures of modern living, so digital technologies, the cyberbullying, the victimisation, there's a caveat to that, though, because those technologies can also buffer them by creating a sense of social support through their social support networks. Also, it's this simulation to university life that is totally crucial. If they've assimilated, they are much better. They have that support mechanism, they have the buffer. Those that aren't settling, this compounds everything for them. Currently, we've got too much variation in the extent to which our universities are equipped to meet this challenge. This comes up a lot in the University of the UK report. It's come up in the Institute of Public Policy report that was published this week as well. That should be in the what's causing it slide, I'm sorry. The role of our HEIs, our Higher Education Institutions. If we take out the variance in demographics and so forth, 49% of the variance in student mental health is from the combined effect of individual and institutional level measures. That means that we have a massive role to play in our students' mental health, so whether it increases, it decreases, or whether it stays stable. I happened to post question last year what's happened to our pastoral care. As our numbers have grown, that seems to have been squeezed, and there's been a lot of questions around are we supporting our students properly with the terms to their wellbeing and the personal tutoring systems. But it's also well known that personal tutoring is key in that assimilation for students feeling supported. It's one of the lists of the recommendations of things that should be in place in the University of UK mental wellbeing guidance as well. That's part of our duty of care to our students, so whether students have pre-declared, then we do have a role duty care, but we also have a duty of care with regards to health, safety and welfare in general of our students. So it's key in that point. And in particular back in 2013, the QAA report recognised the importance of personal tutoring, but they also spotted that there was massive problems with inconsistencies in how that was delivered within institutions and across institutions. And when you look at some of the feedback from students now, that's still the case. And certainly in my institution there are pockets of really good practice and pockets of not so good practice. But reading some of the policies, not all of the policies necessarily are that helpful. Some still put too much onus on the student to seek support, whereas it's known in depression around literature and health that the ones that most need the support are the ones least likely to request it. So putting the onus on the student to seek out the support on the services and the self-referral is immediately you're losing on that, so there needs to be a rethink in my mind on how that works. So what am I getting to? So I'm going to kind of talk a little bit about what this looks like, how this presents for a student. I've used the Dementals from Harry Potter because it's not necessarily a pleasant thing, mental ill health. It's actually all about behaviour change. There are changes in people's behaviours. You will see academic struggles and failures, grades will start to drop, they'll struggle to submit things on time. You'll get absences from class and other commitments if they're regular sports players that may not be appearing to play, those kind of things. You'll start to get excessive substance use, so heavier drinking, heavier smoking, potential legal substances. Loneliness and isolation comes into it. There'll be social interpersonal difficulties, so they'll start to struggle to connect with friends, family, things start to break down in that respect. Changes in self-care and lack of self-care so people may not be looking after themselves so well. Or there'll be this subtle differences and things may not be immediately apparent but you'll start to see it over a period of time. You'll get extreme risky behaviours and their ability to deal with things like regular stressors they'll become more frustrating more quickly. All of our students will get that at some point, potentially. That's not unusual. I, as an undergraduate, went a bit crazy for three weeks with my taste of vodka but that lasted for three weeks and there weren't other things. When these things start to combine generally that there is a clear problem but this requires us to know our students. If you teach a large cohort of 200 students you're less likely to necessarily know that student as well as their personal tutor if the tutor has regular meetings and so on. Again, this QAA report recognising points of person tutoring I've repeated that, I'm sorry. I've lost my role I did it the wrong way around. But yeah, you need to know your your duties and you need to know your students. If you're going to spot a behaviour change you need to know what the behaviour was first is my point. We know that social support and perceived social support are in effect. The internet can help with that to a certain extent they will talk to their friends they will talk to the family they will seek guidance but not all behaviours are good so spending lots of time on the internet in itself is not necessarily a bad thing but when you're using that as your coping mechanism to seek support and guidance that you're not getting from your institution or from your friends and family then that becomes an issue. Right, so I'm going to put a tenuous link with Learning Analytics because a couple of years ago Microsoft did an analysis of Twitter and again they found that it was changes in the use behaviour of Twitter that allowed them to predict when a depressive mode was going to come on for someone but these were people that were already suffering from depression so they knew in their training data when these depression periods happened and then they were able to use the changes in behaviour to predict when an X1 would be so we know that we can use behaviour and certain online behaviours to predict so I ask can we use this to predict this so Learning Analytics this is Siemens and Geistavitch's definition from 2012 it came out of Learning Analytics conference and it is the measurement collection analysis and reporting of data about learners and their context for purposes of optimizing understanding learning advice which occurs right so my argument is can we use this for the wellbeing bit so a lot is said around retention and in fact the key application is early alert and student success course recommendation, adaptive learning and curriculum design so a lot of the stuff that's in the early alert and the student success around graze dropping off around non attendance is much the same data that we would want to use and feed to our personal tutors so they can start to spot these patterns in our tutors but we have a lot of big elephants in the room so we've got a lot of data and a lot of data that we could repurpose for this and we can repackage and surface to our tutors but there's a lot that we need to factor in so immediately we get into the data protection issue if I shouldn't a lot of some of the student mental health gets close to disability which means that we're immediately into protective characteristics which is even more scary on the data protection there's very little good or significant research that shows that interventions based on learning analytics work so Exeter did a survey they started out with 400 papers they then narrowed it down to two that they could do some analysis on and of those four of them I think actually only assessed the effect of the intervention based on the learning analytics so we don't have a very good empirical basis to say that these will work and this is quite a big and quite a scary and a really important thing to get wrong we still have silos issues we don't necessarily have the communications between our professional services departments that are supporting students with our personal tutors we don't have the communications necessarily between the academic teams and the tutors some tutors are not even in the same departments as their tutors and they're not aware of the course so we don't currently have a flow of data that we need and it's a flow of data that we're going to need to keep getting and also if we are passing this data forwards and back because it then becomes a liability question as well as to well you had it, I developed this to suit some various other bits and pieces interestingly in the last two weeks there's been two big reports so the university of UK produced a new step change report which is the mental health guidance report and in it they've suggested that we align our learning analytics to student wellbeing so they're starting to see this but they're also saying that we should promote a diverse inclusive and compassionate culture within our institutions and these are key things that will help us or certainly that is aimed to help us to prevent or at least limit the extent to which these issues evolve so my kind of conclusion is, sorry Alison Wonderland and who are you this isn't a question that our personal tutors and our pastoral support staff should ever be asking our students we have the data there that we can pull out that we can present so they know who they are and we need to make sure that they know who they are but we also need to make sure that they are capable of dealing with that data and they know what to do with it and how and when to get in contact with the students and it shouldn't be well I might see you once a term if I feel like it but I don't really want to also interestingly the IPPR reports your mental health said that for the sake of academic rigor we should not be causing mental health and distress in our students so my argument is that yes it's big and yes it is scary but if we get this right this will make a massive difference to our institutions serious difference to our students and I think this is worthy of our attention and it is something that we should be looking into and I can close if you've got any questions I will take them I also have a massive list of references that I will later share Thank you very much for answering Do we have any questions? Thanks Samantha, really interesting talk I wonder about if you talk about the potential of learning analytics alerting staff could it also be could there also be the potential for early indication to the students themselves look you know your behaviour is you know actually suggesting to the students themselves before they might even know Yeah it can but I think we need to package that very carefully so while I'm saying this this isn't a be on end all for me it's all about something that a tutor has where they can have a meaningful conversation with their tutors you spot something that might not be a little bit right, you need to know the normative pattern for a group how your students normative pattern relates to that group and how their current behaviour is relating to their norm so there's a lot that you need to to know but yes we ideally want to I would necessarily give them a dashboard but I'd have a very subtle conversation with them as a tutor and academic maybe Yeah OK Let's go to the next presentation Our next presentation will be Students' Expectations of Learning Analytics by Alexander, Dragon and Ricardo Hi I'm Alex Wainwright I'm a PhD student at the School of Psychology here at the University of Liverpool and today I'm going to talk about students' expectations of learning analytics So the aims of the talk are to firstly highlight the importance of stakeholder perspectives in learning analytics services and then I'm going to outline the development and validation of an instrument designed to explore students' expectations of learning analytics services So taking what we know from information systems research the inclusion of stakeholders in the design and implementation stages of a service does reduce the likelihood of future dissatisfaction as the service becomes reflective of what stakeholders actually want In terms of learning analytics researchers have called for higher education institutes to allow stakeholders to be involved in learning analytics implementations However finding suggest that the actual level of engagement from stakeholders such as students has been quite low A notable example of limited engagement has been the development of learning analytics policies which tend to be led by what managers, researchers and practitioners believe students want It's reasonable to assume that intentions behind such policies are to improve learning performance or to provide additional support But nevertheless these may not be reflective of what students want from learning analytics and the creation of a service which is not representative of students' expectations is an example of an ideological gap and this is the main cause of user dissatisfaction So to offset the possibility of creating this ideological gap there is a need to explore students' beliefs towards learning analytics services Now in the case of Arnold and Sclaeter they sought to explore students attitudes towards the university collecting and analysing educational data The main issue with this study is that and the authors do highlight it is the instrument used is not validated So the findings they report are questionable On the other hand you have the recent research by Shumaka who has explored students' expectations of dashboard features Whilst this is an important step that students want from a particular service we must understand that learning analytics is not predicated on the inclusion of dashboards alone There are many other aspects to learning analytics and we need to start understanding students' expectations towards each and every part of this But despite these issues mentioned these examples do represent an important step in enabling students to have say in learning analytics service developments Given these limitations of previous work we sought to develop an instrument to explore students' expectations of learning analytics services For the next few slides I'll present the theoretical foundation on which the instrument is based on and then I'll discuss the four themes identified in the learning analytics literature that guided the development of questionnaire items So expectations are a fundamental part of human cognition They influence our motivation and even the judgments we form They are not too similar from beliefs where a belief is a probability judgment between an object and an attribute A expectation is only discernible by the point of time at which the judgment is made So expectations are framed as beliefs about the future But the term of expectation is quite general and it doesn't allow for different types of levels of expectations to be measured So to solve this we looked at the research by Thompson who decomposed expectations into ideal and predicted Whereas an ideal expectation is what an individual hopes for in a service Whereas a predicted expectation is a realistic belief of what they expect as a minimum And by conceptualising expectations in this way you can get a better understanding how satisfaction emerges So if a service meets a user's predicted expectations you're more likely to find that they are in difference as this is met what they expect as a minimum But if the service then meets what they their ideal expectations are likely to see satisfaction emerge But failure to even meet the predicted or ideal expectations is likely to cause dissatisfaction By thinking of expectations in this way we can enable researchers to gain a better understanding of what students may desire from learning analytics services and also what they expect as a minimum So we can then use segmentation processes to understand what features are most important to students The first theme identified in the learning analytics literature is ethics and privacy expectations And there has been a lot of literature about ethics and privacy and a particularly important piece has been the delicate checklist which provides a series of advisory points for higher education institutes implementing learning analytics in an ethical manner And one of the main points here is informing students about the practices undertaken So enabling them to understand what data is collected how it's analysed and how it will be fed back And this connects to the second point of consent which has been debated quite extensively And there are two positions There's one where people believe students should consent to each and every element of learning analytics from the collection of data to the distribution of data to third party companies On the other hand some authors believe that consent should only be sought for interventions But even within this literature these decisions about how informed students should be or whether students should be providing consent are again being led by what practitioners and researchers believe students what's best for students as opposed to asking students what they expect But a good example of actually engaging students in this debate has been presented by Slade who found that students actually want universities to obtain consent before undertaking any analytic processes And together this literature identifies a series of points that does require more input from students Agency expectations refers to learning analytics services not creating a culture of passivity in higher education At the end of the day students are responsible for their own learning They are active agents who set their own goals and enact their own strategies to attain such goals And it's not for learning analytics to remove the ability of students to make their own decisions There may be occasions where a higher education institute may offer additional support to providing regular updates on how students are progressing to particular goals But we should always be mindful of whether students expect to make their own decisions based on the analysis fed back to them or whether they expect the university to make sense of the data collected and provide them with more informed feedback Now learning analytics has previously be dominated with research attempting to identify at risk students This is led by the belief that if you identify them early on you can implement interventions that will offset the likelihood of them dropping out Although research findings don't necessarily support this belief as of yet Beyond merely predicting dropouts, learning analytics has moved into other areas of education For example, trying to improve student-teacher relationships by enabling tutors to have a better understanding of how students are performing and whether issues are present in their learning progress But as with agency expectations researchers are making a lot of inferences about the type of service students would like to receive in exchange for their disclosure of personal information and we actually don't know what type of services students want As previously mentioned there has been investigations into dashboard feature expectations We should always be mindful that this is not the end of learning analytics there are many other elements that we need to understand what students want from The final theme is meaningfulness expectations This refers to feedback from learning analytics services being relevant to students in order to promote a positive change such as motivating learning As with information systems perceptions of usefulness are intrinsic to its acceptance So if the feedback provided through learning analytics services is not pedagogically meaningful to students then you're unlikely to use it So we've included this theme to cover students beliefs about the applicability and the relevance of learning analytics feedback to their learning Now with these four themes we generated 79 items that were subject to peer review and the peer review allowed us to refine to 37 due to certain items overlapping or being difficult to interpret These 37 items were then rolled out at the University of Edinburgh in a pilot study 210 students completed the questionnaire and provided qualitative feedback on each item The quantitative data was factor analysed which led to a reduction to 19 items and the qualitative feedback was used to slightly alter the wording of some questions due to the complexity of them So following this we then re-rolled the questionnaire out at the University of Edinburgh and we received 674 responses which due to the items being re-worded we re-analyzed it using factor analysis This left us with a final 12 item students expectations questionnaire Now the 12 item questionnaire can be explained by a two factor structure of service and ethical expectations Ethical expectations relates to the beliefs of students towards providing consent for personal data usage and providing consent for third party data usage Service expectations on the other hand relates to students beliefs about receiving regular updates on their learning and whether teaching staff have an obligation to act and these factors are applicable to both the ideal and predicted sub-scales But although we've developed this 12 item questionnaire we still haven't validated it So to achieve this we've distributed the questionnaire at the University of Liverpool and here we've received 191 responses which were then subject to complementary factor analysis Following this we found that the two factor structure did have an adequate fit which strengthens the validity of the instrument But although the purported factor structure does have an adequate fit it is important to understand whether this instrument is in variants across sub-groups As up until now there has been no research exploring students perspectives of learning analytics across the various sub-groups available such as gender or faculties So to extend the complementary factor analysis we used mimic modelling which allows researchers to explore whether there is differential item functioning within an instrument through the addition of direct effects of a covariate on factor scores or an item of interest So we may find that female students have a higher ethical expectations overall than male students And this approach is important as it can use to overcome the limitation of researchers assuming that all student groups hold the same expectations of learning analytics services Rather we need to start tailoring policy decisions to meet the needs of the various sub-groups available So for those who may want to use this instrument the use of mimic modelling can enable a greater understanding of whether expectations of learning analytics or learning variance across sub-groups or not The next steps in the project are to firstly assess the reliability and validity of the instrument cross-culturally So to achieve this we have distributed the questionnaire at Talent and the Netherlands and we are waiting to run the survey in Madrid And from this data collection we will be able to assess whether expectations change across different institutions and it will also enable a greater number of universities to utilise this tool in their implementation of learning analytics So as previously mentioned the creation of a service that is not reflective of students' expectations is likely to create an ideological gap Therefore the service does not reflect what students want or expect from learning analytics which is likely to create dissatisfaction from them So this instrument gives higher education institutes who are interested in implementing learning analytics the chance to explore students' expectations of such services In doing so they can begin to align services with what they call the beliefs and create a learning analytics service that students are satisfied with Thank you for listening Got any questions? Do we have any questions? Thanks very much for that presentation Really good to see your emphasis on ethics and on the student perceptions because I think it's something that's all too easy to ignore Obviously this is still in development but when do you think this would be ready as a service for other institutions to pick up and use? Or is that in your plan? We're hoping to have as many institutions involved as possible If you'd like to use the instruments just send us an email because it's part of the Sheila group so that they have a website I should put it on Any more questions? Okay Thank you so much Our last presentation will be Digital capabilities in curriculum design by Tindio Varca-Adkins Thank you and hello everybody Thank you very much for attending and your stamina You're still here so thank you I'm going to take you slightly away from analytics and take you on my PhD journey so what I'm presenting today is my PhD project I was told do a PhD that you're passionate about so it's digital capabilities for me and the main area is curriculum design I think you are probably familiar with this definition of digital capabilities whether we call them skills literacies, competencies but that's the one I'm working with throughout this project and my main motivation was if you imagine my role is a central learning technologies role I would be working with staff from various disciplines on design workshops where they come and think about how the designing courses so some links about our previous presenter John and colleague and I think one of the things that interests me or interested me is how we can work with them and how to enable them to create curriculum that is ready for the 21st century and we use the digital elements model, the six element model in these workshops we ask people to think about what skills they already have what is it that they would like to develop and also what skills will be important for their students to develop to become successful professionals in their own discipline and I don't know how in terms of how you are from your discipline I'm in a central role so I usually would be working with the people from different disciplines that I probably don't have much to know about apart from having some ideas and obviously we have been very successful in working with them on flip classrooms some of the education and activities but with this project what I wanted to really discover is go out to the discipline that I don't really have much knowledge about as a fieldwork and really look at how technology has transformed or disrupted that particular discipline so that when I am working with these academics I can ask really good questions perhaps challenge them and see where the gaps or strengths are so that I could also then advise other disciplines but have you thought about doing that so a quick show of hands who in this room has let's say humanities social sciences backgrounds what about sciences some of you what about both hybrids I expected that as well in learning technology land and also I guess it's a question of whether you're working closely in learning technologies so similar role with a particular field and closely working with them and you know the discipline or whether you're probably more in a central role so a bridged version for today of my PhD aim there's other things going on is what I'm going to talk about is who is a digitally capable engineering student and the other field I've chosen is management but I really just won't have time to talk about that today so I know if you've been familiar with let's say the GIST digital literacy programme there's been lots of work on it let's say various universities have gone down the route of choosing digital literacy as one of their graduate attributes and have asked programmes to map how each programme develops these skills for instance Oxford, Brooks and Bath so I'm building on their work and in terms of where I feel I guess my study may add to it is that I am literally triangulating so I am looking at three perspective staff academic staff employers and students I'm also talking to employers on what digital tools and practices they are using in engineering and in management which has been really useful for me okay I'll take you back to my living room when I was growing up this thing took up a third of in our living room my dad was an engineer so I saw a lot of his back and him on this design desk and I was always jealous of his rubber really good rubber to rub out and redesign things that he was so they were his tools a few years ago and I won't let you know how many years ago it was then but now they saw obviously with this research what I'm interested in now is what engineering management looks like now and I think from this morning if you were at the keynote Peter Goodyear showed some really interesting images of learning spaces especially in the STEM subjects so these are the kind of things going in and exploring in more detail but today I don't really have time to go in much in depth now but I've been really uncovering really interesting areas of how technologies have impacted on engineering okay so I did promise a poem for you if you read my tweet this morning and this is also if any of you is interested in research I read Pat Basley's book on qualitative research methods and she was recommending that as you are working through your transcripts trying to make sense of them by the new experiment and perhaps write a poem of the salient highlights or bring in images draw a diagram so really this is my research repertoire as well expanding and I just hope you will enjoy it anyway it is an example of a very good cross disciplinary work because my husband is an English teacher so he's going to do the voice for it and so what I'm trying to do here is I looked through the transcripts of the engineering ones here and I tried to draw out the salient points about the discipline the signature pedagogy and also what is important for them to develop in students and also what the approach is around digital capabilities development of the students so hopefully this will work we open boxes pour things apart bikes, trimmers, spark plugs, cars we simulate, model with graphs solve problems with applied maths we collaborate from day one on all things complex and human we draw on global resources, join forces just like in the real world although MATLAB simulink and CAD are core you don't need to be the master of all after baptism by a five day wildfire fight your way through the digital mire and only with wit and the need to inquire just like in the real world and also perhaps it is a question for you in your discipline or your background or the people you were working in what would this poem look like what sort of things would you highlight in your discipline and what I will do now is just take this poem section by section and basically just relate to some of the highlights just to engage with you with some of the findings that I'm putting together so the first bit is open boxes reverse engineering taking things apart and seeing how they work is a particular part in engineering partly because you have to look at what the competitors are doing apparently and then perhaps see how they work and whether you can copy it and then documenting this process representing it through wikis digital technologies is one example for instance that I got from academics that they do and also there is a general inquisitiveness around the subject and the being, breathing and living in the project that also came through in these but when I relate this in terms of the discipline and when people were talking about digital technologies as part of the learning activities of doing with students I think the main idea that they immerse students in the task so the task is absolutely the primary you know you open that box you see what's in it, there's things around you we have given you resources it's what Peter Goodyear was talking about in terms of learning design and the digital is sort of around and what I'm getting from how digital technologies appear in the curriculum that if you look at this model and again I won't really have time to go into it but they basically focus on the top two circles so where the staff feel that their role is around either in gendering into students, information media digital literacy is part of this task or digital problem solving and just to take that on board digital problem solving the key feature in engineering now is the move away from paper sketches on to simulation and models but it's not just an instrumental tool of a particular digital technology it is the way how they analyze and model things and things through things with the problems okay and I think again just from employers it was very interesting to hear that even in the last 10, 8 years they do say there is a predominant shift not just from moving design and manufacturing on to virtual simulation but also to other parts of further than the product chain so even they would be simulating how materials arrive on a particular side before they actually do the arrival and also job roles and skill sets are changing was another feature so for instance one employer was talking about how there used to be the engineer and the technician relationship and the technician had the skill of drawing something out in AutoCAD but all the analytical skill the engineer had and they had a more or less face-to-face paper based fax based way of communicating now with the expansion of designing virtually or technicians also are raising their skill level to using some of the analytical features in the software and therefore it also means that engineers will need to have higher skills so there is some erosion and re-alignment of the kind of roles that are needed which is internal impact on the curriculum and I'm sure this happens in many other disciplines as well ok so taking the second stanza the collaboration teamwork and global work intercultural work for again two key features of engineering and this is I think to me I think I always thought of engineering as quite a sciencey you know you do a lot of maths discipline but I haven't appreciate how much they are working on complex human social problems so many of the tasks involved I don't know design a humanitarian drawn that is one way because once it arrives in the area of crisis it can't come back so it will have to turn into something normal or another student was designing something to help a disabled student then for children hospital some hydroponic units so I think that there is the appreciation that the discipline has lots of other links with the others and because as mentioned teamwork collaboration global working is quite prevalent in this area I would have expected communication and collaboration perhaps some of the elements that are more explicitly taught or covered in the curriculum but interestingly again is the general approaches that students are immersed in the tasks and perhaps they might end up creating whatsapp groups it happens organically it's not something that the staff member orchestrates I talk a little bit about digital identity in a bit so the final bit is was really around the use of digital tools technologies so I mentioned that matlab so there's some core software that is used in these areas and they are absolutely vital for students to get to know but the whole idea again is apart from this year one is baptism by a 5 day wildfire I mean that was mentioned by a number of the wildfire was kind of the old name of one of the these tools students apart from this 5 day training in the first year they don't get any more training specifically but they will be immersed in these tasks and then there will be lecturers around that they can drawn for support or peer support and that's how they are exposed to this so generally the idea is that developing confident agile adopters is similar to the Oxford Brookes graduate attributes so students are more required to pick things up quickly rather than learning particular software and this is another poem I won't read this one but this was written on another transcript where the lecturer students were into Formula 1 so designing Formula 1 cars Ferrari with two car companies manufacturing companies and I think what I wanted to illustrate in this so there's a section that says sponsors need you in a highway linked in jacket so the idea that for these students who were engaged with sponsors who were paying for the materials that they were building the cars with the use of social media was much more prominent and needed than let's say for the other engineering students so the idea that once you start looking into a discipline you notice all these sub-disciplines and sub-specialisms and then the idea that they might prioritise different areas of digital capabilities in this case developing a digital identity was very important for the students because they did have to acknowledge the sponsors and tell them look we have done this now we are doing this and okay your money is in you know safe hands and finally I think what I wanted to draw out is the VLE because I know a few of presentation we talked about VLE use and it's something that you know it's not always something very exciting but what I found interesting as employers and staffers speaking about is that some of the student learning tasks such as submitting assignments via the VLE on time, on deadline was almost like getting students to becoming a professional because what happens in the workplace is that there will be an organisational system an enterprise system that they will have to be using they will have to meet deadlines and rather than perhaps as with engineers they will be working with various clients who will need to ask changes and there is a new process that they will have to go through and if they do it via email they can be open to and liable to perhaps when things go wrong to things going very wrong in that sense if they hadn't followed the new process so in this sense they were seeing students engaging with the VLE almost like getting used to the idea that okay this is the new process that you have to follow and this is something that's going to be in the workplace please do not email me your assignment because that's not the process that you need to be following okay so obviously I haven't talked about management I don't really have time to do some comparisons but where I'm up to and I'm still yet to do the student collecting data from the students which I am really looking forward to so what does it mean I haven't said anything drastically unexpected disciplines are different they might prioritise different areas of digital capabilities and it's really how I'm looking into because that's what I can grab and use in my practice supporting other disciplines so if I wanted to draw some conclusions it will be around that it's such a mapping according to disciplines is key and the programme level approach is needed so many times staff were saying that from year one and year three there is a development into how much dependence we give students so again year one we expose them to the software but then more and more we will rely on them to be using these tools a lot more independently and us for support and what the finding that perhaps some of the digital task that staff have exposed students to are focused on those two areas of digital capabilities just makes me think about there needs to be some conversations of whose role it is that those particular areas they need to be developing perhaps in some cases for instance in communication and collaboration staff didn't feel that they are the ones who should teach students about how to use WhatsApp or how to use communication it wasn't necessarily their role it was something that need to be part of the conversation because for instance where does the curriculum cover where we might do professional communication and in engineering for instance they do at the very beginning they do teach students how you email your lecturer so that you are not in your lower case high I'm in the pub hello so they do get some ideas of how to start professional collaborations so I think that's what I had to time for it's still my journey so if anyone is interested I'm happy to speak more and I just wanted to acknowledge my host site Liverpool, the Lancaster University and my supervisor where I'm doing this study and also I've done two expert interviews with Helen Beatham and Rhona Sharpe who's helped me to sharpen these ideas and also my participants so I'm happy to have that time I think that's what I wanted to say thank you thank you very much Tundi any questions? seems like you are all tired now and you want to leave, no more questions? okay thank you very much thank you Tundi