 Just to recap, we've seen that communication is a critical element in learning and we've seen that emotional competence is also critical for success. And what I would like to do is move us up into higher education and talk about applications of emotional measurement in online learning. And first I wanted to thank the Levergham Trust for funding the research that I'm doing here, CAST, an organization that I worked at previous to coming here where I started embarking on these questions and let me hire education grant for supporting this work. So first, I'd like to talk about, you know, when we talk about emotions, here's just four discreet emotions. And one way that researchers look at researching emotions is they identify certain discreet emotions that they want to examine. And so here we have frustration, happy, calm, sad, and those are just four different discreet emotions. One is ways that you would label an emotional experience. And if you wanted to, you can actually organize discreet emotions in dimensions. And here, what I've shown here is the core affect dimensions of arousal and valence. And what this has done is it's put the negative emotions on the left side, the positive emotions on the right, the emotions that are low energy on the bottom and the ones that are active or high energy on the top. And this is one way you can actually think about categorizing emotions. This is actually considered to be a universal categorization of emotions. So this has been researched across cultures. Now why would you want to categorize emotions? Because there's a lot of them. I mean, when you went through the exercise with Bart and sort of gave your list, the ones that were spoken were probably not all the ones that people thought about. When people look at the role of emotions in learning, they research a large number of discreet emotions. And here, this is just a sampling list from a LACE report that Bart worked on. And if we take these emotions, one way that I chose to categorize them across energy level and valence is first to use a sentiment analysis to put all of the negative words on the left, all the positive words on the right. And then I plugged every word into Google Scholar to see how much research is going on with respect to each of these emotional words. That's not a perfect metric, but that was just one way to sort of organize the energy in terms of how much research is going into these topics. You can actually use, and so you can see the arousal level from the bottom to the top and the valence from the left to the right. You can use the sentiment analysis dictionaries to do both arousal and valence, and here is the same set of words organized in the quadrants of high energy positive, low energy positive, low energy negative, and high energy negative. This is a useful tool in thinking about how to organize information. This is actually assisting me in doing my lit review as I go through this process and thinking about how to look at the role that emotions are playing inside of education. So I've illustrated core affect, but I wanted to give an example of what research design looks like when it's leveraging core affect for how it's going to measure the emotions of its learners. Before I came here, I was working on an organization called CAST, and we worked on a Department of Education Office of Special Education Policy, five-year, ten-million-dollar research project called UDO, and in UDO, the idea was to do or model after core affective measurement of emotions, and you can see on the left a design for self-report, where we're working with sixth, seventh, and eighth graders in North America, and this is a pilot study with 353 students that I'm going to talk about today, but you can see low energy words are on the bottom, high energy words are on the top, negative words are on the left, positive words are on the right, and students could provide their emotional response to a reading that they just completed to sort of say, I found this reading to be both interesting and calming. Simultaneously, they could make comments in a discussion, and this article is great just like Garen's presentation, so you could see, actually when you read that you laugh a little bit, but you can see that there's an emotional expression in the written text. There's techniques that people use called sentiment analysis to do an analysis of text to organize that information around low energy, high energy, negative, and positive, so this was an opportunity to start on my research thinking about multiple traces of data organized across core affect to get a better understanding of the emotional state of the learners, and when I did an analysis, I'm just showing one scatter plot, and I'm just going to walk you through it really quickly. On the left, these are the self-report emotional responses, and on the right, this is the analysis of the discussion content, and what you can immediately identify or I'll immediately highlight for you is that the reactions are all positive, generally speaking, and the discussions are again very positive, but there is a distinction between arousal in self-report in terms of how it compares to the analysis of what people are writing, and that's because what people are writing are generally in a low energy category in the language that they're choosing, whereas when they're self-reporting, they're reporting through the full spectrum of arousal from low energy to high energy, so there's a discrepancy between the analysis that I've done on self-report and the analysis that I've done on conversation analysis and sentiment analysis. Now, that's exciting because why is it different? There's two different reasons that I can think of that are very plausible explanations. As sixth, seventh, and eighth graders, they may not yet have an emotional vocabulary to be expressive across the entire spectrum of arousal levels in the words that they're choosing, so they may actually need some support in how they're going to use emotional language to describe their current state, and that's part of the goal that the React design had was to organize emotional responses across the spectrums of emotion to sort of support students that may not have a broad vocabulary of emotional response. Now, it could also be that self-report is subject to providing socially desirable responses, so when you ask a bunch of students, how are you feeling? I'm high energy and happy. These things are great, positive, and up, up, up. They might think that that's the response that we wanted to hear, and so it could be that the discrepancy here is about a self-report problem or a mixture of both. So what's next? I took the work that I was doing there, came here, and I'm working on a pilot so that I can use the labs that we have at the Open University to do physiological measurement in conjunction with self-report and conversation analysis to get a sense of where the discrepancy is coming from. So we can actually measure arousal levels with heart rates. We can actually measure valence with facial responses in the software that Bart was illustrating earlier in the talk, and we can look at what the physiological indicators tell us, and we can compare that to how people are self-reporting and to the words that they're using in how they're being expressive about their conversation, about the learning activity. From that pilot, the intent is to improve the measurement that I'm using on conversation analysis, because again, communication is critical to learning, and again, emotional competence is critical to learning. So I think that getting at good measurements of emotional content and written expression is going to be a very helpful tool in terms of improving learning environments. We'll take that measurement, move out to a randomized controlled trial with 1200 students, and validate the measure that we're using in terms of written communication. And again, a wonderful opportunity, as we discussed earlier at the Open University, is the opportunity to bring research to scale. So with the validated measure, we can take our steps and move into a scale study, looking across courses and seeking a more comprehensive understanding of the role that emotions is playing in online learning. So effectively, what we're looking at is research questions around measuring emotions. I'd like to look at how are we going to display that information back to students, and what support are students going to need in order to use that information to improve their learning experience. So effectively, we're looking at research studies that are guiding us towards supporting the emotional intelligence of written communication. So what? What are the implications? Well, we can see, as we're talking about the development of emotions and the role that it plays, the effects that it will have is at the student level, student families, parents, teachers, course designers, and researchers. When we talk about emotional competence, we should have emotionally competent designers and emotionally competent researchers that are creating environments to support learning. So how do we do that? This is an example of what a course design might look like. It might include a reading, a quiz, a group assignment, and a test. Typically, the feedback that we provide to people includes feedback about their cognition and how effective they are at producing their understanding of the learning activities. And we measure things like behavior, like attendance or how much time were they spending on the website. But we don't necessarily get at the emotional measurement, and we're not surfacing that for the learners or the designers. So what would that look like? This is just an example, fabricated example of a student. So imagine here we have a student who's done pretty well with reading comprehension checks. They've done pretty well with their quizzes, and they've done excellent with their tests or excellent with their quizzes and doing good with their test. But there was a really negative or poor performance on group assignments. Now, from that, I can look at, well, how much time did you spend on that? What were the clicks? What were the resources that you used? What is the feedback? What's the reason or the rationale for why that assignment didn't go well? As an individual, if I had access to an indication of my emotional state, and it showed me that I had expressed positive low energy throughout the majority of the course, but during the group assignment, I was expressing in a very negative high energy manner. That would give me some information to then potentially change my practice and learning. That would be feedback that would give me some insights into what led potentially to a 65. Now, as a designer, if every single student had a negative high energy reaction to a group assignment, that might give you some feedback as a designer, that there might be something wrong with that assignment. So I think, again, thinking about the emotional intelligence in the design side, as well as in the student experience side. So this is where the research is headed in terms of surfacing this information to support improved practice and improved learning. And one of the things that we wanted to highlight today was that not only are we researching these questions at the Open University, but we're teaching a variety of skills that if you are also interested in exploring these questions, you too could take a course and start pursuing an exploration on this topic. Thank you very much.