 Thank you Can I first I think I'm in for a fantastic introduction and can I think once was and where's Mark sitting? There he is for inviting me. So I feel very honored. I'm a little a little bit feeling under the weather So if I'm not so enthusiastic as I'm normally I'm just Shout so the reason why I like interaction is that I hate lecturing I come from a master's university and we do problem-based learning at master's university So standing in front of a crowd is really boring so There's going to be lots of interaction you can choose what we're going to discuss today, which is quite interesting So just to pick your brains also to get to know who you are and to understand where we're from Every do you know what formal informal learning is? Yes, you know, I guess what social interaction is Between two people or more people. What is what is learning analytics? What is learning analytics? What's the what's the word that comes up? Data Sorry, I didn't Okay, what more What do you what's the feeling that you have from learning analytics? Yeah, so it's not only student learning. It's also about a process. Yeah Yes Yeah, so is Dublin City University using learning analytics Okay, it's quite interesting. I always do this kind of exercise and I've done this version in different versions I think now five or six times and you're the first audience That hasn't mentioned negative elements yet learning analytics. So I'm very happy to be here Yeah, I just saw a fantastic piece. Sorry, I forgot your name Mark's. Yeah, I didn't know it was mark because but they can't be too much And yeah He's he's marked to Yeah, yeah, yeah 2.0 He's doing some interesting stuff and he's already targeting students based on their behavior. So that's really so You don't really have a negative feeling with learning analytics are you doing Mm-hmm. Good question It I didn't how does it work here? Have you informed your students about this? Did students opt out how many Okay, okay Yeah Okay, yeah, why? Mm-hmm. Mm-hmm because where's the boundary of Where's the boundary of what happens in terms of learning inside the classroom or inside of yearly and outside that's yeah But what's new in what way? Yes, we've been yeah When I applied for this readership in learning analytics people were like, yeah, but you're an economist why I Mean I I've only started publishing and learning analytics with learning analytics in my in the title one year ago But all the I mean in the 60s people were already collecting loads amounts of data and trying to help our students Interpret the data Yeah Yeah, yeah We are we think at the open University that we are the first Institute in the world to have an ethics policy for Learning analytics and we tweeted this and it's in times higher education. That's far. No one said the contrary But many people at least at the open University are very concerned About learning analytics and when we did a consultancy with a hundred students Many students on the one hand were like yeah I don't want you to follow me and I don't want you to know that I'm posting Something on the internet or that I'm disabled or that I'm Particular collar or that I'm a woman with three children now I'm divorced or whatever, but at the same time many students indicated Yeah, but I've been dropping out and I'm failing with my course and I was really surprised that no one contacted me So there's on the one hand this concern about privacy and yet the other hand this need at least in the open University of a kind of I Need this personalization that there's lots of tensions and I'm if I can find remote remote which I Have no idea where I left Did I put it in my pocket? Here we go So today I want to have a debate with you and to start the debate. I mean So the definition of learning analytics the classical definition It's the measurement which you set before collection analysis of reporting of data about learners and their context For the purpose of understanding and optimizing learning to help them to learn In an environment that is a code. That's a classical definition that was only listed in 2011. I personally think it lacks the kind of Social-cultural elements so my colleagues and my team Rick Ferguson Says that we should focus on social learning analytics. That is basically how do people build knowledge together within the context of their social-cultural environment and actually quite like that definition, right, so Has anyone seen a picture like this? I mean you have because you've been to the Euro call anyone seen this What's the story of the picture? Yeah Cool, I should I don't have to come here. It's cool. It's cool. So the problem I have With this picture is with the VLE or the LMS system and all this analytics and then a kind of a visualization and the teachers then Reflecting on all these visualizations and then comes up with some kind of informed judgment what students should do I like it as a simplification But I think the real power Would come if we would start to link Student To this and what you're doing. I think links to links is one also linking The students to the teacher so so the one of the principles of the Of the learning of the ethics policy that we designed for learning analytics is that students are not a number students and learning analytics is a moral practice so Teachers When they're judging students, they're not just numbers in their graph There are stories behind this and I think we all agree so we all so I'm normally I have to explain what learning analytics is and I always take a metaphor of football Who is this guy? Everyone knows this guy No for the people who don't know this who this guy is who is he? Louis van Gaal yes and The story in terms of learning analytics and to show you in a kind of layman's and what learning analytics all about I'm gonna focus on my hero The and basically the rise and fall of my hero So I have a very short YouTube movie and given that I'm from the Netherlands. I'm just gonna share this Do you remember this goal? What a bull go I Was sitting in France with my wife on a on a on a on a square We were the only Dutch people on the square all the French were supporting the Spanish And I we were one nil down and I was like, oh, this is the most fantastic moment of the day And then we scored another time and we scored another time And we after the fourth or the fifth goal people were tapping my shoulders You know, you shouldn't you shouldn't celebrate so loudly so No one guessed that we would even make it to the next round because we thought already have an awful team They they lost the previous previous European championship. So we thought they're never gonna win. So when we the thing which you probably don't know is that Footballers are continuously tracked in terms of learning analytics So you can see exactly what Robin van Persie is doing Continuously and you can do this real time with a service called squaca So this is This is learning analytics 2014 so what what what do you see in this particular picture? You're nodding like what No, no, so it's an all the players movement and where the passes are going and whether they're going back and whether they're going forward But it's it's lots and lots and lots of data. It is data useful To interpret who is going to win or not Yeah, you keep This this is real. Well, okay, this is screenshot after the match So this is like after 90 minutes, but you can do real-time analysis when when you're playing and especially if you pay a certain fee Then you can watch it real-time Yes, yes, yes Me neither me neither. So this is a simplified picture And what was again, I forgot again what was so this Spain is playing that way. They have one goal in that direction from that position and they had loads of I mean That even one went completely off target here and the Dutch had fantastic goals on that side And then the analytics allows you not only to drill on a kind of course level But it also allows you to drill on each individual player And you can see in this predictive modeling from Percy is comparative from Percy Previously so continuously what his performance is and then initially he was well not great Or just similar to what he's done before but then he scored his magical goal And then he came he went from strength to strength in terms of his performance And there is this other guy I in a Robin. I don't know if you know him He's allows you play but from Percy. He was like he was my he's my hero So the fundamental question is then of course We lost in a semi-finals So I'm just wondering He got he got five stars during his first play and during the semi-finals He got only one and a half star and if we have all this learning analytics all this data to our disposal Why didn't this fantastic world-leading coach intervene? So what I would like you to do is to talk to your neighbor For one or two minutes. This was a this was the match against Argentina this fantastic screenshot And what I would like you to do is to discuss Where if you would be Van Hal if you would be the teacher Where would you have intervened in the performance of Robin from Percy? One minute go ahead I Do you remember the answer I Okay If I asked the group over there where where Should Van Gaal Van Gaal apparently you pronounce it in English, but it's weird. He's Van Gaal Why why should he have intervened according to you over there No, no, no, you're preparing you're comparing him to the previous from Percy So all the all the dates you have about from Percy and how well he's in terms of you know the positioning and how well He's playing so it's like and the history of his previous from Percy You're using that and in comparison to the previous history of how well from Percy plays That's his benchmark That's right. The thing is if you're comparing you to Robin, which is here or not But if you were the performance of Robin might be Unpredictable and it's that unpredictability. Yeah, that is what swings the game. Yeah, I See some puzzled. No, I disagree Yeah Yeah Yeah Okay, so you're saying after 15 minutes Yeah Why not So for me to understand what that means I need to know what the other players around him are doing and how the performance Yeah, he is dropping relative to his teammates. Yeah, then maybe there's something to be read about that But I wouldn't be satisfied to make decisions just on that individual Mm-hmm Yeah Yeah Yeah Yeah, yeah, I mean if we if so until until a certain point in time he was under par he And these genius players, you know, they just need one chance and then they score So, I mean it's what I think it's nice as a kind. This is a metaphor of teaching, of course is on the one hand data is extremely messy and It's only what you actually can measure, you know, the number of passes the positioning where people are walking But you don't really know What's what's in the minds and hearts of people? And you don't know what you're saying is the the interactions between those and also you don't really know You know the informal part of learning that is not captured so What I'm gonna talk to you today about is on the one hand my title says that I have to be very positive About learning analytics and I am But I think at the same time there are some caveats that I would like to discuss and we know for example In hindsight that from Percy like me was a little bit under the weather that day and you could have He scored a brilliant goal on Saturday the other week Yeah, nearly nearly the same but then he then he missed and then ten minutes later He scored just fantastic or so so it's like it's very difficult to to go into the psychology and that you need a little bit of luck Etc. Etc. So I'm gonna propose something really weird. I'm gonna give you a menu option And I only have one hour and there's so much to talk. There's so much data we can share I also have loads of things which I didn't put on slides, but we can discuss so we have a kind of Interaction so you can choose three courses Or you can say I want to have something else and just shout what you want so I could show you one of the studies we did on the power of Formal and informal learning So I'm just going through the menu now at the moment I can show you the power of formal and informal learning between teachers and How they learn inside and outside their practice I can show you a couple of studies That show how do students actually make a mix? How do they pick and choose in terms of all the VLE tools that they have at their disposal and how does that relate to? their behavior and performance I Could show you a fantastic study. This is this is the main piece of this is a In an adaptive mathematics online environment with over a hundred twenty different variables that we measured Over a period of time or I could show you a study Which is not yet published, but it's at the OU how we compared 40 modules and Looking at okay, how does learning design of those 40 modules link to the experience of students? but also how that linked to performance and What's underlying all of this so if I can just do a very simple show by hand so you can select three courses? given the time and then if there's time left over just to dress and so Three courses you can choose. So I'm just gonna who who wants to see a number one One two three four five six seven eight nine ten twelve. Let me just remind myself Because we're in analytics twelve. Yes Number two who is interested in number two One two three four five six seven eight nine Okay, who's interested in number three? One two three four five six seven eight nine ten eleven twelve thirteen. Yes I can only vote three times. Yeah. Yeah, I'm not and I'm not checking if you're actually yeah number four One two three four five six seven eight and Number five who is interested in five? Oh So you're gonna skip the most interesting study Well, let's see how far we get and then So the power of formal informal learning So I have a kind of provocative question Which is basically and this is linked to the informal study that we did If 80% of our students learn more outside the classroom than they learn inside the classroom or inside the VLE What are we measuring? right The next one which links partly to This the third study is what kind of data is actually useful to collect in terms of VLE and the third one Which you haven't chosen so I'm not gonna discuss this then Just a little bit of context in our latest study we distinguish between Learner data and learning data so learner data we it's about From Percy how tall he is how fit he is how which size shoe he has what kind of shirt he's wearing You know the kind of baggage that you bring Before the plague before you come to the ground and learning data is what some of you also mentioned You know the process what's what's actually happening when there's him when he's on the page and plays all those balls And how can you then measure those and we think that learning analytics? There's so everyone talks about learning analytics, but what are you actually what are you trying to measure? so this is a study we did in a Face-to-face institution so not really learning analytics in a way So one of my PhD students who was interested in a medical program using problem-based learnings a problem is learning everyone's Is working in small groups of 12? They continuously meet for half a year in those groups of 12 and then the assumption is is that they primarily learn Within those groups of 12. That's the assumption, right? So then we used a principle called social network analysis to see okay, are they indeed learning? Only within the group or I'll see are they also learning outside the group So this is the picture What does what does the picture show you? It's very similar to the football pitch, you know with all the passes, but it's something different Any idea? Yeah Yeah so this student Has indicated of the 400 students in my medical program. I'm learning from one student and All the other 399 students independently from that student verified No, I'm not learning from him or her Well people who are very central in the network are people that Many people go to but it's also they might go to many people to discuss their learning So what do you think the colors are? the color yeah the colors Black and white and gray. Sorry. It's not a it's not a color indeed Yeah achievement data. Yeah, it's achievement data And what does it what does it show you? mm-hmm so if you look with a Quite weird angle. You see there are lots of white students on the fringes and More you get to the center the relatively more darker the colors so the higher the grade Yes, so Yes, so it's not and that's again this learning analytics. This is Data on the general picture So we then thought okay, but there are loads of other factors that influence this So we'd look Darker when you get to the middle Because you always find on the extremes of the thing. Yeah. Yeah, yeah It's sort of what we would expect somebody who is unconnected with their yeah Yeah, there's There's a person here, which we talked about before who only has two friends But they're very dark Yeah, so I like modeling so The size is the kind of betweenness effect so are they sorry, that's a social network jogger It's like how many people are connecting to you and how many people are you connecting to others? So the bigger the size the more you're a kind of bridge builder Yeah, that's a very good question Yeah, so this is just one model to look at it So we try to for example control for prior knowledge. We try to control for motivation We didn't look at this particular case introvert extrovert, but in other studies. We've done this We looked at what extent they feel feel academically integrated into the and then you get into this weird thing called structural equation modeling So we looked at okay What actually predicts student learning and of course previous performance on modules heavily influences how well you're doing over time No, no, yeah, yeah, yeah, yeah, so so that's also what there isn't in in a structural equation modeling way We can then control for the city the direction Walks in that way if it's the other way around then so it's it's an assumption we make in the data modeling So what we found quite surprisingly was that you know the way students whether they are feel integrated or not or whether they are Motivated or not didn't influence but to our largest surprise The number of people that people were in in terms of the network was the best in terms of the number The best predictor for performance So it's like it's it's weird in a sense. So with whom your networking Has a massive influence on Your performance Yes, so we I should have explained this in a little bit more detail so we asked them Who are your friends and we specified you know going for a beer or going to the movies or discussing personal issues And we asked them with whom do you give information in terms of well how to solve a particular medical problem or Where to find a good article on our drosses or whatever and to whom do you give information? So we have a kind of fine grained Quantitative understanding of this And then Juliet afterwards did loads of qualitative studies as well to further impact those reasons So we were initially surprised that there was this massive social network effect So then we look even further and we went back to the to the these small groups of 12 and we assumed That most students would primarily You know learn this is one group this is another group and this is another group etc etc and We were in for a surprise So there are approximately 40 groups And you have to bear with me a little bit. This is an another graph, which is difficult to explain Everything which is below zero Means that people in a group are more inward focused than output focus everything Above zero until one that's an index. They are more outward focused And not inward focused in terms of whom do they give information? They seem to be mostly Outward focused, but also from whom do they get information? It's mostly outward focused. So what we found that the vast number of groups except these three learned more Outside their group Then they learned inside So the only thing they did was they had once a week a lecture on I don't know what heart Functioning or whatever and then the rest of the time they always met with a tutor on this small groups So Designated So it's really it's Yeah, so in terms of via Lee activities They are assigned to a particular group So if you would track via Lee behavior You would only track certain part of the learning if if the largest part of their learning occurs outside the classroom You know when they go to the library when they're sitting here having lunch, how do you? How do you measure that? Yeah, thank you for asking this I Don't have to study here with me, but If you just google me This is a Spanish woman. We did exactly the same study in a classical Spanish University And you find exactly the same that most students or even if they're Allocated in traditional lecture rooms and they work in small groups. They mostly learn with people outside a designated room in Finland with konfirman's a primary school children same mechanism We did lots of studies in in in the UK of course and again you find that most most groups Even if they're working together for half a year They continuously maintain informal links outside those group because we're human creatures If you decide we're in one group together, I will still publish with My mates from the Netherlands and from Spain and I still have my friends from Sweden where I was in God knows one ago So it's it's it's quite intuitive This goes against learning analytics in a way because you can't collect that data unless you do social network analysis I mean as anecdote many football players when they go from one club to the other the first thing they do is call Their mates who are actually playing there is actually a good thing to go there So, you know, it's it's part of life networking is continuously important And they sit at a different table every month And he's very good memory so he remembers everybody's name in class Yeah But so he every month they sit a new table, but every when I can extract this information extracting the data But he has the same Small number of friends. Yeah on yard. Yeah switching tables. Of course, of course and Research by Juliet and it's published in plus one. She she measured this over a period of two years afterwards and The first group that students were in is still the same group after two years. It's like 80 percent prediction accuracy So we're gonna skip the next study given time. So I will just give this slide later on So most students learn 80 percent of the students Yeah, I like to link with Pareto Have there been attempts to Yeah, we Such a great crowd here Another study which I haven't included here is we we try to manipulate this So Juliet for example the next study she did the 400 students 50 Two groups of 50 were continuously assigned with the same people and the other 300 were randomly assigned and you could see over time that there were two separate islands of these 50 people just by the way we assigned people over time and another study we Mixed people based on our social networks And then you can just see you can manipulate of course The network and it has a profound influence over time So you we as teachers can change that and particularly in a cross-cultural settings if we put students In a mixed cross-cultural group and we put them together for a very long time suddenly. They're all friends So that's really cool. So you can influence this From the students themselves they're saying the timing of that intervention is very important Yeah, and all the variable variables of the idiosyncratic nature of lecturing Yeah, you know who uses technology and yes, but the fact that when they get together in first year They're going to pick Facebook for their social network Yeah, as you can get in with an intervention before that they will do their forming Yeah So in a way it's it's creepy because similar in learning analytics We can manipulate this But nonetheless, it's So Yeah, so what what our research seems to indicate but it's still I mean work Consent the one of my PhD students is doing this again on Monday. She's manipulating students in a different group So if you put good students together with slightly weaker students Then the weaker students over time become better because they're in this smaller group So you could then start to play around with those kind of variables While the if you if you if you believe for God's key You learn more from teaching then from learn from learning in yourselves in a way They also learn from it, but the evidence at least is a little bit Yeah It is it is it is fascinating and in a way and that's why I don't like this initial learning analytics Picture because it's learning is much more messy Then you would expect so I skip to study twos go to study three This is an online course. So you for those who came for the learning analytics Element. Yes, there is data and there's online behavior So these are students again in Maastricht relatively small sample mostly international students and they in contrast to Dutch students who didn't most international students don't have Economics in secondary education. So we give them a kind of online crash course before they start and in line with my own philosophies of learning Those were social so they had to meet online in discussion forums to discuss economics problems and Work on it and at the same time every Every week we had video conferencing and then one of my PhD students He coded, you know that they use webcams that they use the whiteboard that they use chat You know that they use audio and then he looked at That's how students choose those technologies does that actually influence How they're learning and does it actually influence performance? So coming back to a question to you because I've been talking already way too long So this is a kind of the structure of the course. They met online synchronous and then asynchronous and again online synchronous asynchronous Which one which element do you think is the best? predictor for How well students are performing over time Asynchronous you read my paper You read this paper. Why do you think it's asynchronous? Yeah Yes Yeah, and we thought that you know the fact in the community of inquiry model Teaching presence is very important and also social presence is very important and Initially in my PhD. I only focus on discussion for but it's quite difficult to give The social feeling of learning and we thought oh you can actually see you know a teacher here I am with a slightly different haircut You know that that That the ability to to to help your students is much better in synchronous environments So then we again did a path model and you again see then You see a picture like this. So what do you see in the picture and this is amazing because I actually see people counting and looking at It's like normally when I give this like oh my god, this is way too complex So on this side we have we knew what kind of motivation students had Extrinsic motivation is I I study economics because I want to be rich or my parents told me if you study economics You're gonna be rich Intrinsic is oh, I really want to know why we're still so much in a difficult economic situation And I really like economics and a motivation is I literally don't give a toss why I'm here So why do you think that intrinsic motivation is the only line that links why do you think that is the case? Yes So students who are really intrinsically motivated to study in this case economics They're engaged They don't probably don't need to tutor even and Why is it that? There is this line going through from posting behavior and Why is it that? There is no link with the video conference you think no significant links, sorry But why isn't it not important to be present in this case at a video conference at least not in the first model So the weird thing is I mean exam dropped out for some reason on the slide So intrinsically motivated students do well they post a lot They post more over time and they then do well And of course they attend more video conferences after time But what about those extrinsically motivated students what what what kind of what do they do? Yeah Yeah, and and they unfortunately are more likely than to to drop out And That's quite Difficult and then I think in many online environments. You see that the ones who do extremely well are People who are who have a particular kind of motivation that is encouraging them to learn So you're sure you don't want to see the fourth study Let's go first to the fifth one because I have to finish at two Okay, yeah, okay, I will quickly skip through and I can always come back So just ignore this for the moment Okay, okay, okay So this is not published yet, so please don't cite this or turn off your large part So this is at the OU who are actually working so it's nice to link it with my own real world So we have this thing called learning design Where we have an online tool which I can show the link later that maps what what the learning design philosophy is of each individual module and Then a learning design specialist together with the module chair goes through each of the design decisions that a teacher makes and Loads of stuff. This is I think a health health course But I can't tell this because it has to be anonymous and then we we we basically tag the kind of activities that Students are expected to do based on the learning design. So in this case 40% of the activities are assimilative so reading You know learning and a bit of writing 2% is finding new information in this particular module 5% is discussing with peers 21 is productive. It's like in health care It's actually trying to you know to put on a wristband or to to I don't know put in a needle You know to to learn what it's like They didn't use any experimental stuff here Very little adaptive and then of course assessment. So for each of these modules we had That's how we've mapped around a hundred We mapped. Okay. What's the learning design? Do you have something similar in your Institute that you do it in this way? Because one of the things in the innovative pedagogy report is that we say okay learning analytics is great But if you don't know what in terms of data, you're actually looking at it's quite difficult to make Link so taking the metaphor again a football you would expect that the graphs of the Spanish team Will be very different than the graphs of a German team Right. So the way the German coach designs the the team play or the way we design is very different from the ones Okay. Yeah Who very difficult question it depends on on on the size of the module so in the in the large level one Course a year one courses. Let's say 4,000 students each Course has a specific tutor we call him an associate lecturer who works with a group of 20 very intensively together And but at the same time they get support from well from one or two teachers over this 4,000 students Oh, sorry, so yeah, yeah, yeah, yeah, I should have mentioned this And this is people from my institute from the Institute of Education Technology So learning design Specialists are working together with module chairs. Sorry. That's jargon from the OU with with Module convenes To map all this out So this is a very this this takes our learning designers three days per module So it's a tremendous amount of work But we think in order to understand learning analytics. We need to understand the strategy behind Yeah So then what we what we did and I'm hoping that you can see kind of a narrative is we try to see okay our teacher designing Unique courses or are there kind of patterns that are fairly similar? So we found out four different Categories and I will explain them in a minute So the first category is what we call constructivist Clusters so people have lots of things in terms of content. I See not in yet where lots of reading that students have to do very limited experiential stuff and Assessments, that's one category of courses then another recipe of courses is loads of assessment activities during the module and then limited other things and then there is a mix of well, a relatively more finding activities and lots of experiential learning and The fourth one which we call a social constructivist, which is lots of communication lots of these productive activities Moodle yeah, yeah No, so this is basically how the the module confeder has designed the module Based on the kind of the blueprint Yes, yes So perhaps I shouldn't say this in front of a camera, but it's quite surprising at the vast majority of our courses are Classical Knowledge-driven Yes, yes, yes, you should come and work at that at our university and tell us I'd like to know how Moodle does Yes No, no, so this this is purely based on on the kind of battle plan the recipes that that Module chairs have produced in a way they could use any kind of combination You know have ten from Percy's in your group and then or have none of them It's basically it's whatever people decide to do Just to add yeah, my own thesis is on work in a college, which is a non-line Yeah, I found that despite what the literature is saying Learning should be social construct. Yeah, it was mostly transmission. Yeah Yeah, so information. So let's let's okay. Let's Which one do you think is the is more beneficial for For learning in terms of performance of these four who thinks that one is the most beneficial Constructivist so loads of reading Who thinks that too is loads of assessments continuous assessments? One Yeah, yeah, yeah, yeah Learning Yeah, but performance and by by how many people actually met in the OU one of the things is that many people drop out because of distance education So you would hope that a learning sign would help students to continue to study. So learning is being measured by the achievement Yeah, fine number of people who passed the course and with the number of grade Which one who thinks about a social constructivist is the best way Okay, let's have a look. This is this is very tentative. This is only again a health and safety warning only 40 modules This is not the truth. There might be other designs. So with all this health and safety This is what students think so we I've linked this with 20,000 20,000 students so typical student surveys and linked this with the design of the course And what you see is that if I mean, it's not always significant because there are only 40 modules But in general if there are lots of assimilative activities, it's always positive. So students like this If there are lots of finding information activities in it or lots of communication activities They report They who fill in the survey Hmm Of course, they don't know that, you know, they can't compare it But at least if you compare all the surveys of all the modules together, it's quite puzzling to us I probably have to turn off the camera So one of the things my criticism I'm not criticizing my fantastic institution is only students who Pass a certain amount of activities then get the invitation to complete this so Some students who who drop out in the beginning are not asked this questionnaire so You have to take this with a pinch of salt. So then let's look at we like objective behavior, right behavior of people so then we looked at VLE behavior, so this is the number of minutes that students spend on a weekly basis and Just ignore the third one because this is very weird It's we only had one module at this point in time that we could link with VLE behavior So what do you see any patterns in terms of VLE behavior in terms of this design of how modules were implemented Yeah, so she constructivist is like Extremely high And then over time Similar right so lots of activities to do Yeah, yeah Yeah Yeah, yeah, and of course I have to again take it with a pinch of soul because these are averages and certain modules Finish after 12 weeks certain finish after 40 weeks some continue until 60 weeks At the light the degree at the green the gray the solid gray Yeah, but that's that's due to an odd thing. This is only one module. So there is and This is probably when students had to complete an assessment. That's where they go back to the VLE So if you look at individual modules, you see this is a constructivist so hardly anyone spends more than 200 minutes Per week in the VLE and it's mostly, you know, it goes with peaks and troughs linked to assessments. This is the social constructivist So loads and loads and loads and loads of activities pursuant to the sum one actually goes until 1400 So immediately you see okay to just by designing the course The behavior of students is completely different and that's intuitively logical, right? if you then link this to Again the seven categories same picture So the seven categories and then how students Over time click and behave You see that in contrast to the one we said before finding information and communication They didn't like it if there are lots of finding and communicative activities students spend tremendous amount of time online And then the crucial bit and that's what our provice chancellor likes. Let's look at performance Same same way of portraying If you have loads of assessments, it's relatively negative in terms of the number of minutes spent online so performance You get a completely different picture than the first picture from the students so students liked a Simulative in terms of experience, but if modules have loads of assimilated activities the number of people who passed the course is Substantially lower and also in terms of grades which we which I didn't show Then modules who have other activities so in a nutshell what I'm trying to say very inarticulate and this is very much work in progress is learning design seems to have a very strong influence on behavior and performance so we can again manipulate So to come out to my conclusion and I you didn't see this But I will send you the paper is that VLE data the raw one doesn't provide very good Proxies for learning and if you would have selected study number four you would know why So the implications of learning analytics to partly you can't understand this because you didn't select study four but What we continuously seem to find in learning and it is just clicking behavior doesn't really explain Learning it's primarily if you link it to assessments and if you link it to emotions that you can understand it perhaps social network analysis could be a good way to measure this informal learning and Then a fundamental question coming back to this when Van Gaal should intervene I think what I'm trying to show you over time is it's very complex There is not a clear point in this if I take a random graph like this where should you Intervene if this suddenly goes tremendously down from 1400 minutes to does it that mean that the course is not successful So it is it is I mean, I think we're at the at a kind of Crossroads To be honest, I'm not that familiar with Yeah Oh, sorry Yeah Yeah, and also if we would collect more courses it probably would become significant and also a link which is quite intuitive is the more workload you have The lower the performance Message where people just don't yeah Yeah Yes, if you would if we would have selected study for But I will post this all online and you can you can read it afterwards Yeah, yeah, and then and study for it actually shows that emotions were the best predictor for learning Yeah Yeah And I think that's also one of the things which is so important is that this Learning analytics is a moral practice is that behind the numbers are so many complex elements And Well, it's Thank you for your time and attention and thank you