 Hi, everyone. My name is Ryan Hazen. I'm an Instructional Technologist and Moodle Administrator from Carroll College in Helena, Montana. Very quickly before I get started, I'm going to be taking questions for this session via Twitter. Please use the MootUS15 hashtag and then hashtag 4P for four pillars. I tried to keep the characters down so you could make your questions as verbose as possible on Twitter. Go ahead and do the questions via Twitter. I've got a bag full of goodies from our mountain moot that we hosted at Carroll College. We just did our fifth one this year. I've got some t-shirts and some neat stuff to give out to people who pose great questions and get cognitively engaged with the presentation that I'm giving here. This is about research, but it's more about an online course about online teaching that I teach to Carroll faculty based on the research that I did that other people have done. I wasn't actually doing research myself. I was aggregating research and trying to create a course on this. The academic department at Carroll came to me and said, hey, we need an online teaching course about online teaching so we can certify teachers and know that they're creating quality content when students go to access their courseware online. I said, okay, let's do this. The course that's eight weeks long, it's divided into four cycles, and each cycle is two weeks. I had five readings, four activities, 12 discussion posts, four assignments, four evaluations, and four journal entries. Of course, that really is an understatement of how much engagement this course creates. So you might ask, what literature? Here's the list of the literature. These slides are also on my presentation course on the U.S. 15 Moodle site. So you can go through and you can get a closer look at this bibliography. But the first thing that I noticed was that everybody who was talking about online teaching seemed to want to reduce best practices in online teaching to a set of principles that were easily digestible by professors as they were going to teach. It kind of makes some sense. So I thought to myself, you know, we've got this literature by Reagan, identification of competencies for teaching success, a big long list of principles, nine principles for excellence in web-based teaching, ten principles in effective online teaching. This principles paradigm seemed to really emerge when you go out and look at the literature on online teaching. So what emerged for me? Well, I wanted to make it even simpler than that. I thought nine or ten principles was too much. I had four cycles, so I said, let's reduce this to four principles. And what I came up with was be present, be engaging, quality content matters, and know your tools. So let's talk a little bit about being present. Here's what Lawrence Reagan had to say about it in this article about online teaching. Show up and teach, practice proactive course management strategies and establish patterns of course activities. Henry and Meadows had some very similar stuff to say. Sense of community and social presence are essential to online excellence and sometimes the little extras go a long way. And finally, I'm a little bit nervous about quoting Martin Dougy Amis when he's in the room, but I'm going to have to do it anyway because, well, his research was really good. So he's talking about the instructor, Peter, was doing close readings of student journals and started kick-starting discussions with carefully crafted questions that were designed to tease out what those students were already interested in. And this is a really good explanation of what I mean by being present online. Now, I know none of you have ever dealt with this, right? Where the student has an assignment, they toss her book aside and head out to a party. Well, one thing that you can do to try and combat this is have really clear expectations, view by times, and patterns of course activities. So that was the reason for the cycling of this particular course. And you can see this emerge in lots of other literature about this. So once you have view by times and your students have clear expectations about what they can do, they finish their assignment before they head out to that party. Now, you're not going to stop your student from going out and doing whatever they're going to do, but you can give them a timeframe that makes them engage with your material in a responsible way. And it's important for you as the online instructor to model this for them. This is, I think particularly, let me get back to that, precise clear expectations. I noticed this trash can is like right over there. And I noticed that precise clear expectations are just as important in the real world as they are in the online world. Does anybody notice what's going on here? Trash only? No trash? Have you ever wound up with a contradictory statement in your online course that you restore it time after time, you can find this happening really easy and you have to pay attention because if you're not clear with your expectations, your students don't know how to engage appropriately with your courseware. And this is what one of my professors, a theology professor at Carroll College, had to say, first I was skeptical of the view by times. I shook my fist at E. Ryan. Then I found these view by times helped me. If I actually viewed by a certain time, I inevitably began reading and reflecting a bit, giving me more time than I otherwise would have had to engage the material. I believe it also upped the quality of my appropriation of the material. I take back my initial fist shaking. Now aside from my professor shaking his fist at this E version of me who was out there causing him consternation when he wanted to be doing something else, I was enamored with this idea of myself having an, I wasn't present. I was somewhere else doing something else while Eric was shaking his fist at his laptop. But I was present in his mind because I had established my expectations clearly up front and I had followed through with those expectations with him. That created presence in his mind of me when I was not there. That fascinated me. So that's what I mean when I say be present. Not just get in there and engage in the discussions, but be present enough to have your students know what you would be telling them if you were standing next to them right then. So let's talk about how to be engaging. Now this is something a little bit different than being present. Again, I'm going to quote Martin from his 2003 Ed Media article. For each topic, students were required to read two, sometimes three papers and make journal entries. Reflecting on the professional implications was reserved as the main subject of these discussion forums. So students are not just reading, they're reading and doing some reflection. This is these feedback cycles that Martin was talking about in his keynote. Getting you involved and getting you engaged, doing something with the material rather than just being a passive recipient. You have to be active. You have to be engaged to really truly learn something. Henry and Meadows also had a lot to say about this. These guys from the University of Lethbridge. In the online world, content is a verb. This is a great principle. In the online world, content is a verb. Do not forget that because content has to be active. It can no longer be passive. Also, sense of community and social presence, this is going to come up almost every time. I'm going to talk about that a little bit more. Sometimes the little extras go a long way. I encourage you to read this article and contextualize some of these principles in terms of engagement. Here's what Lawrence Reagan from the Penn State World Campus had to say about that. Help maintain forward progress with little nudges. Finally, in the core competencies document, encourage active learning. Give prompt feedback. Emphasize time on task. Engage in learning activities. That's verbatim what this principle is. I think of engagement as being two basic categories. There are a lot of more categories of engagement, and you can get really specific with this, but I'm going to try to stay broad here. First, I'm going to talk about cognitive engagement. What cognitive engagement is, for those of you who may not be familiar, this is my favorite definition. Bonwell and Eisen from this 1991 Higher Education Report. Students must do more than just listen. They must read, write, discuss, or be engaged in solving problems. More important to be actively involved. Students must engage in such higher-order thinking tasks as analysis, synthesis, and evaluation. Anybody know blooms? We're going to talk about that in a second. Active engagement can be defined as instructional activities involving students in doing things and thinking about what they are doing. I really want to focus, first of all, on these two parts. Read, write, discuss, or be engaged in solving problems and getting students involved in doing things or thinking about what they're doing. So how do we do this? Well, it's not as simple as it seems. You can say that, yes, but let me give you a concrete strategy that you can use. We call it chunking. Carol, I've heard other terms for it. Basically, you break up knowledge and comprehension activities into smaller chunks and insert engagement activities in between those chunks of content. So you might give a student, say, an online video, and then they look at that for five to 20 minutes, and then you want them to engage in something about that video, where they either write an essay or maybe do a multiple-choice answer question or a matching question that's carefully designed to help them engage with that content. And then you want to send them back into a new piece of content. Maybe a web article or some other sort of article that you want them to look at. And then, again, after about five to 20 minutes, you need to engage them in some sort of activity that's going to force them to think about it. Because if they have to respond to a multiple-choice question or to an essay question or to a draw it question or to whatever sort of question format you're using, they can't just ignore that material. They have to run it through their brain. They have to do some work with it. And there's really good cognitive science research to support and prove this. If you're interested in that, read Daniel T. Willingham's Why Don't Students Like School. Very, very good book. So, yeah, you need to chunk your materials, especially online. Also, automated feedback for formative assessment. Remember quizzes in Moodle? I think Diane... I can't remember Diane Dell. Dr. Diane Dell was talking about this earlier in Apachakucha. And she was talking about using quizzes for formative learning activities. Not just summatively, how much do you know? Can you prove it to me? But to get you involved with the content. And the great thing about Moodle quizzes is it gives you this automated feedback so the student knows right away if they're getting it or not. If you structure that quiz well, it's a learning activity, not just an assessment activity. Another thing, here's what Maria Brosnan, a nursing professor at Carroll, had to say about this. It's been a great opportunity to experience the role of a student. She gained a better understanding of the types of learning activities available on Moodle, as well as how these activities can be implemented to achieve different learning outcomes. Enjoyed the short videos, followed by the various activities. I've noticed how much better I understand the information after completing these activities. That was really positive when that came up naturally, organically in our forum. So I'm back to the two types of engagement. Now I'm going to move from cognitive to social engagement. These sort of act in two different places. I'm going to quote Martin one more time. If you want people to be socially engaged in their learning, you have to privilege a focus on collaborative discourse, encourage individual development of meaning through construction and sharing of texts and other social artifacts, and apprentice learners into communities of practice which embody certain beliefs and certain behaviors. I want to stop here and point this particular thing out. You can be a mediocre online teaching about any subject except for online teaching. In this course, I had no choice but to walk the walk. You have to walk the walk because you have to apprentice your learners into communities of practice by embodying the beliefs that you're trying to teach them. This is paramount to remember for people that are teaching pedagogy online. You have to walk the walk. So targeted messaging is another way to engage students. Perhaps you guys have seen this course participation report in Moodle. You can tell when a student has or has not posted to a particular activity, and you can send them a message based on that. You can say, hey, I can see that Ryan Hazen and Billy's student did not engage in this activity. I'm going to send them a message and I'm going to say, hey, something like this right here. Let me show you my targeted message that can get connected back to my... Here we go. So after every assignment and after every due date, I sent a message to the students in the course. If you did not complete, you got a message like this. Hi, I see you didn't complete your evaluations for evaluations one. I had to put a zero in the grade book, though you have three more chances to improve your evaluation grade. And I go on to discuss how important it is that we're socially engaged with one another, that the other students in the class are relying on you to give feedback to them for their own growth and learning process. And then I wrap that up with a reminder, hey, thanks for reading. Don't forget, this is the next assignment that's coming up in our course cycle. But equally important is to send messages to those people that did complete the assignment. Perhaps even more important that you send messages to people that did complete the assignment, telling them they did a good job in thanking them for being there for the other students in the course that need their support to learn as much as they can. And this came from... My pedigree is in K-12. I was an eighth grade English teacher for four years. I had a paperless classroom and iPads, a learning management system and all that stuff. And one thing that I learned about when I was in the K-12 environment was a ratio of positive to negative interactions with students, a ratio of positive interactions to corrective interactions. And this graph that you're looking at right here, it's hard to see from there, but this is from a 2011 report where they had a baseline, which was at the beginning, the gray line is the amount of time students are disrupting the class in the K-12 classroom, and the black line is the ratio of positive to corrective interactions. So what you see is an inverse relationship. Now, a positive interaction can be as simple as, hey, you brought a pencil to class today. Nice work. That's an expectation I have for you. You remembered your notebook. I'm really proud of you for doing that. I know it's difficult for you to get to your locker from third period to come in here for fourth period. Or, hey, you really took the time to sit down in your desk and you didn't fiddle around with anyone else. That's what I like to see in my classroom. And what you notice is when you start doing even these little things, students start to seek more of them. We are hardwired to want positive feedback from our people that are in our lives. And if you as the teacher take the role and do that online, this works just the same way. You send those messages out when people have completed their activities in appropriate ways and say, I appreciate that. That's important. This class is not as good as it could be without you. That matters. And that's social engagement. So this is what a couple of professors had to say. Ryan Howes from Spanish, I particularly appreciate the emphasis on social engagement. Sam Alvey, who's an amazing biology teacher. Social engagement is really valuable in helping students from various backgrounds and preparation work on material groups and learn more effectively. It's your job to make your online class a place where people can make these social connections with their peer groups. It does matter, especially in places where people come from diverse cultural, socioeconomic, and ethnic backgrounds. So that's being engaging. Let me move on to quality content. Lawrence Reagan in his 10 principles document says quality counts, and I particularly like this one, number 10, double click a mile on my connection. That's important. You can see what students are seeing. That's why we're doing this online teaching course where the professors are the students. They have to do this. So, Henry and Meadows, this is a good one. Technology is a vehicle, not a destination. And great online courses are defined by teaching, not technology. So I want to talk a little bit about some basic pedagogy here. I'm going back to this Bonwell and Eisen definition, and I want to focus in on this part now. Students have to engage in higher-order thinking tasks, such as synthesis, analysis, and evaluation. This is a direct reference to Bloom's taxonomy. And for those of you who are not familiar with Benjamin Bloom, he and a bunch of other people came up with Bloom's taxonomy, which was a hierarchy of cognitive activities. And they came out with this in 1956, and it's still relevant today. That is an incredibly long shelf-life for any academic pedagogical theory. This has been in the academic lexicon for over half a century. We should be looking at it and paying attention to it, whether we're teaching in person or online. So it wasn't just Bloom. It was Bloom and all these other guys, and I bet if they knew how long this theory was going to last, you know, Crattwall might have said, hey, let's call this Crattwall's taxonomy, alright? And in 2000, Crattwall actually did get together with this guy, Anderson, and they reworked it. They put synthesis at the top of the pyramid. And let me give you my quick and dirty on what Bloom's taxonomy is. Okay, so if you can know something, if you have knowledge, you can tell me that 2 plus 2 equals 4. But if you don't comprehend it, you can't tell me that, by extension, 2 plus 3 equals 5, and 2 plus 4 equals 6. Comprehension is a global knowledge of an idea, whereas knowledge is just the ability to repeat back to me what I tell you. Application is your ability to take that arithmetic and complete, like, a word problem with it. Two trains are traveling. One's coming from St. Louis, one's coming from Cincinnati, one's going 40 miles an hour. When are they going to run into each other? That doesn't give you a math problem. You have to come up with a way to figure that out using what you know of arithmetic. Analysis is your ability to take apart someone else's application of arithmetic and understand what they did. If you do that math problem, you can look at it and work backwards to figure it out. Evaluation is your ability to say this solution to the problem is better than that solution to the problem. This one's more elegant. This one's tighter. You did extra steps you didn't need over here and you got the wrong answer or something like that. And then, finally, synthesis is your ability to have applied, analyzed, and evaluated so many times that you come up with something totally new, like algebra. If we're talking about arithmetic all the way up, everybody at some point had done so much algebra that they synthesized a whole new idea out of their experiences with arithmetic and moved into algebra. This has happened a few other times in mathematics. That's the end of my mathematics knowledge. I was an English major, so keep that in mind. Moving on from Bloom's taxonomy, how do I apply this? How do I tell my students to engage in, quote, higher-order thinking activities? Where this wheel that indicates verbs that you can use to direct students to engage in particular cognitive tasks? So if I wanted a student to analyze something, I might say, I want you to deconstruct the new deal. That means take it apart and explain why each part comes together. And then I want you to maybe critique the new deal based on its effects after they implemented it. That's your ability to say, hey, was it good? Was there something else that could have been better? Was there something particular wrong with it? And then finally, if I wanted you to synthesize, I might say develop a different economic plan that would have addressed the United States' needs during the Great Depression. So all of these questions imply knowledge, comprehension, and application, because they're above them in the hierarchy of cognitive tasks. You can't analyze the new deal you can't evaluate the new deal if you can't deconstruct it. And you can't synthesize a plan that would have done something different for the United States in the Great Depression if you hadn't evaluated the new deal for yourself. And if you don't have that whole pyramid, you're not really understanding the topic in the way that you possibly could. So when you privilege those higher-order thinking tasks, your students are doing a lot more than just answering that question cognitively. There's all this sort of deep knowledge and comprehension and application activity going on. So that's really important with quality content. And coming from this particular professor, Jamie Dolan, a sociology professor at Carroll, this is a tremendous compliment. I've been surprised at how much one can really do with an online course in terms of pedagogy. I had really thought that it would be a lesser teaching and learning method. I am actually thrilled to realize how much I really didn't know and how much I have learned and can learn in regards to online teaching. Again, organic discussion post. This wasn't in an evaluation or anything. And having seen the course that she put together when we finished this online teaching course is just thrilling, absolutely thrilling. The student evaluation data has been overwhelmingly positive, a ton of engagement, a lot of good stuff getting done there. So finally, I'm going to go last to know your tools. In this course, Lawrence and Reagan would say quality counts. Again, double click a mile on my connection. And then finally, Henry and Meadows up at the University of Lethbridge. This is a good one. A great web interface will not save a poor course, but a poor web interface will destroy a potentially great course. It's not so important that it can make your course great, the web interface, the tools that you use. But it is so important that it will ruin your course if you don't construct it appropriately and use the right tools. So again, Martin was talking about this in that 2003 paper. Tools for peer rating were added to forums. The separate ways of knowing, separate and connected was added as a scale. And this was at the very beginning of Moodle. So they're starting to add functionality. They're starting to throw in tools based on their experience teaching in this new tool, Moodle. So some of those tools that we have looked at in this course include lesson, quiz, workshop, schedule, or wiki forum, glossary, database, questionnaire, and choice. And the way that the course was structured is we would do the reading. We would have an engagement activity with the reading, a quiz or a lesson module where you had to do things while interacting with the reading. Then we would have a discussion about that. How might I use this in my particular discipline? And then we had a workshop where you submit a link to whatever the module of that particular cycle was in a different course that you had made. And we used a common rubric based on the reading from that week to evaluate each other's modules that we created. And then you look at your evaluations, you do a personal reflective journal entry to try and ingest that and look back at it when you're done with the course. So those are the four things. Raging quality content matters and know your tools. I invite you to go on to the US MOOC 15 site and look up this presentation because I just restored my online course on to the Moodle for you guys to look at. It's got the lessons, it's got the discussion forums, it's got my little video tutorials in there, it's got links to all of the literature and the full bibliography, the syllabus and all that. I really hope that you guys will take a look at that and benefit from it because I've benefited from so much that the Moodle community has shared with me. So thank you guys a lot for everything. Should I get out of here, Michael, or do I have time for questions? I was going to look at Twitter here and see if we've got any... Don't project Twitter. Was there somebody else who's using that hashtag? Oh, okay. That would have been... Hold on, let me see what's going on there. We're all adults here, come on. Is anyone in the room got a question for now, though? Hello? All right, yeah, we'll do it live. Go ahead. You may have already said this, but how many faculty went through your course? Oh, I actually forgot to say that. The first iteration had six faculty and then I went and did a presentation about it to our faculty assembly and the word of mouth got out and then for the next iteration I had to go west to be part of the course. I had to break that into two separate iterations of the course with 15 each and I'm halfway through the third iteration altogether right now. So I've got students doing stuff right now, too. Any more questions? We actually gave them a $200 stipend to take the course. The first thing, and they're not required, although I think academic wants to go that direction, and the stipend does not justify all the work that you do. I first said to the professors, expect about four hours of engagement per week. I think I based that on a slightly quicker words per minute than most people use, so I had a lot of professors who said it took me six, eight, or sometimes even 12 hours a week to do the work, which was great and I'm glad that they engaged at that level. But at the same time I was trying to keep it slim to do it during a semester when they were also teaching. So yeah, I've got some things about that, but it was a stipend and voluntary, but after this iteration we'll probably have a requirement before you teach online. Any more questions for Ryan? Did you find any from that Twitter feed that were mentionable? I hope that's not too long-winded. Is Jason here someplace? We've been going back and forth about the whole issue of engagement and I was trying to get at the question of does formative assessment with automated feedback free up time for instructors to do more engagement because of the whole problem of online teaching taking more time than face-to-face generally speaking? I think that at the outset, no, because you have to construct that feedback and do intelligently designed questions to provide relevant feedback at the right time. So once you have the courseware constructed in your second iteration, I found with this course, I took maybe 30% of the time of the first iteration of the course because I had that courseware already set up and it was just a matter of tweaking it. So yes, it can, but I would be wary of telling a professor it's going to save you time out of the gate. I would say this is going to require some serious investment up front and a really heavy maintenance of the social atmosphere as well. It takes a lot of instructor time. So the place where I think online teaching especially helps with efficiency for professors is the workshop module. You can't feasibly assess assessment, assess student assessments of other students work in a way that the workshop module does. So that saved a whole lot of time and just augmented the output of the students. Yes and no. And it allows you to do more as what's more important. You're engaging your students more intensely. You'll be happy to talk about that more later. We've got time for about one more question after this. For the course content we know quality control is very important. I'm wondering whether you use any guideline checklist like quality matters to double check the content? I didn't. The main checklist that I was looking for is essentially the four principles that I was talking about today. I didn't deploy any out of the box quality checking like quality matters. But the check that they have here at the University of Minnesota I think that was circulating around is an excellent document. I took a look at that and I would use that in my institution any day. So that can be useful. As far as targeted messaging that's really neat what you were doing both for those who didn't and those who did participate. How is that different than just giving feedback in like an assignment or grading spot as well as you must use some system for making that happen like automating it or copy paste or something? Well the course participation report can only send two messages. There's the did complete and the did not complete and for the students that did not complete there is no feedback box next to the assignment because they didn't turn it in, right? So and also course participation can give you views so I would say have you viewed the reading, have you viewed the activity because I didn't just have a complete the activity by Wednesday at 11.59. I had view the activity by 11.59 on Monday and then complete the activity by 11.59 on Wednesday. So that reading activity you have to look at it two days in advance which emphasizes time on task and if they have not viewed that activity by the view by time then you send them the targeted messages and say hey I see you didn't view it. You've still got two days to complete it but you're slipping, you know jump back on the on the Moodle train. So do you just copy paste those messages I copy and paste them or modify them or write them. I keep a list of the ones that I sent in the last iteration of the course in a separate document. I can copy and paste them in and it also helps if you put in specific things about the course like the activity says something that Billy student talked about this week in the discussion and think about that. Making connections between the cycles was a really high priority for me in that targeted messaging and in the discussion forums as well. Alright, thank you very much Ryan. It was an excellent example of taking theory and testing it in real world situations with data.