 Hi, my name is Tracy Tagohama Espinosa, and I teach a course at the Harvard University Extension School called the Neuroscience of Learning. It's an introduction to mind, brain, health, and education, and in this video, we wanted to share with you 12 steps that we use to create this very successful instructional design. Now, this is a realistic video. We want to begin just sort of by framing this. This is not an easy thing to do, but it's absolutely effective. And so we want you to think a little bit about whether you want to invest the time and actually have that paradigm shift of thinking that it will take to actually create a course like this. We'll do this within the worked model, this framework of our course so that you can see exactly how we put it together. We'll go through each of those different steps, and then we're going to put everything in the balance, you know, the pros and cons of using this particular instructional design methodology. So there's a whole lot of ways to take your courses online, and they range from things like webinars and MOOCs, things that have pretty much no contact between participants to this very high-touch environment of doing online learning in the structure that we're going to recommend. So there's a huge gamut of things, and you have to decide where you want to be on that spectrum. Now this video is really out to feed the curiosity of those people who want to challenge themselves to actually make the best possible online course. Many of you watching this video have already seen the tour of our online classroom and canvas. Please feel free to do that to get a sense of how things are laid out. So what we want to do in this video is give you a behind-the-scenes look at how all those pieces came together. This course is taught 100% online. It has a balance of synchronous and asynchronous activities. We use Canvas as our LMS, and we use Zoom for video conferencing. It is a flipped course. We meet synchronously once a week for two hours. We use a lot of things that are both teaching as well as evaluation tools, discussion boards, quizzes, reflection papers, semester project, but we don't use them in exactly the same way that you might be very familiar with. We'll explain a little bit about how we use those a little bit differently. For example, discussion boards are not used to check people's reading of a chapter, but they're meant to build community, right? Quizzes aren't meant for giving this end grade or summative kind of evaluation, but rather, more formatively, people can take them as many times as they like. We just want them to have vocabulary and core concepts before they come to class. So, similar tools, different take on them. We typically have a luxury of having people from several different states in the United States, as well as from many different countries around the world, and, of course, caps at 50 students. I'm the main instructor, and we usually get two teaching fellows. And the evidence we've been gathering has been done since 2014, documenting these last seven iterations of the class, and to measure the impact of different changes we've made to tweak the course to be its very best. We, of course, have all of the official feedback that we were given over the years, that we would love to have you stop the video and go ahead and read some of the things that people say about the course, but they really go into the instructional design, and they celebrate the ways that that facilitated their ability to learn how to learn within the structure of this class. We're really pleased when we get a bunch of comments that say things like, this is one of the best classes they've ever had in their lives, or that they felt that this was a highly personalized experience, and I invite you to have a look at those comments as well. So we're going to pick apart this structure that helps students sort of get a groundwork idea of basically what is a class about through the pre-class video. We invite them to explore using a huge variety of materials that we put into the bundles, which are many libraries curated for each topic. We have a lot of opportunities for them to talk to each other, especially in the small group work in the synchronous classes. And we have policies that encourage students to rewrite any assignment they'd like, using the feedback to improve their learning outcomes. This means we have several things that are individual activities that students do on their own, other things that are group activities, some things that are online, some things that are offline, as we try to leverage all of these tools in the best way possible. So let's look at these 12 steps. First, we have to decide on the course objectives. And some of you may call course objectives things like educational competencies or learning outcomes or learner outcomes, right? These are the things we set up in our syllabus by saying, at the end of the course, learners will know or be able to do or value. And this hovers around the big question, what knowledge, skills and attitudes will learners have at the successful completion of your course? So keep that in mind as kind of a north as you plan those objectives. And also keep in mind that the OECD definition of educational competencies is accumulation of knowledge, skills and attitudes that an individual can have. And knowledge are all the things that you could basically get off your phone, Google-able knowledge, dates, facts, formulas, theories, concepts, historical dates, right? Then we have skills, which is the application of the knowledge. It's one thing to know A squared plus B squared to C squared. And another thing to know when you use it, solve what kinds of problems. And then at the innermost core of educational experiences, our attitude and value shifts that occur. For example, when we begin to value teamwork or that we are people who are in solidarity with one another or that we appreciate perseverance, want to be resilient in the face of challenges. We know that anything that a student learns within your classroom context is some form, some combination of knowledge, skills and attitudes. So as you do this first step of deciding what it is, where you want to be at the end of the semester and you decide your list of objectives, just keep in mind this longer list of knowledge, skills and attitudes. And we have some example of course objectives, the way we've articulated them, for example, in our syllabus. They look like this. They're generally started with a bunch of verbs. We're going to explore, examine, highlight, learn, consider and translate different information. And we also give students that hook of an overarching goal. What is the overarching goal of the class? Maybe you'd like to try that as well. This leads us to our second step. So once we've decided on the objectives, we have to decide what order we're going to cover, what topics. Some people consider the topics to be the curriculum of the class or the way the syllabus has been laid out or they call them weekly topics. This is what we might indicate in our syllabus is being something like at the end of this week or at the end of this module or topic, students will have the prerequisite knowledge for the following least topics. So basically the order of the topics is important because they respond to a kind of constructivist design that has a learning trajectory that has logic to both the student and to the teacher. And the main question we keep in mind as we do this is what is the logical order of the topics that builds off this neuroconstructivist design or this neuroconstructivist trajectory of learning that it makes sense not only as we read through this list but basically in terms of the way that the themes build off of one another. This means that once we have our list of objectives we really have to break down our course structure and we do this by weeks. We have 15 weeks in our semester and we break them down in this way and we have identified a topic per week. So we think about well at the end of the day the last topic I might want to consider is neuroethics but at the beginning I definitely need to have my introduction and that means I have to have these fundamental basics of neuroanatomy but oh yes, I better stick in affective neurocentering that actually fits in very well at the smack middle concept because it has so much to do with neurophysiology but it also has to do a little bit with psychological terms. We know that there's pillars of attention and memory that need to be talked about, neuroplasticity. So we fill in the main topics in a logical order so let's talk about how we express these topical orders. What we do in our syllabus is that we list by week the name of the topic for example so we have each week listed out 1 through 15 and then we indicate what the main topic will be for that particular week. Additionally we also indicate there some of the other section topics so we're going to go into much deeper detail. In order to decide those weekly objectives we have to do a teeny bit of research. So weekly objectives, some people will call them unit objectives or module objectives. In our case each module is a week so we can call them weekly objectives but this is what you might put in your syllabus that says something like at the end of each week learners will know or be able to do or value very much the same structure as the understanding by design money that we use for the global objectives the same thing now at a weekly level. And this pretty much responds to the question you know what knowledge, skills and attitudes will learners have at the end of a successful completion of this week's topic. So this means once we have our structure we have it laid out in these 15 weeks we know what our topics are. If we take the example for every single topic we're going to do the same thing but for example in the mind-body connection we then look at it at this level of knowledge, skills and attitudes what are those sub levels. So we have a global objective for this week but we also have these specific objectives which are these subtopic areas. For example in mind-body we look at sleep and dreaming, physical activity and nutrition but also for example with a nutrition we have another subtopic there of the gut brain access. And we came to know this because by defining these weekly objectives we had to do a lot of research and that research also helped us build up bundles. These are the many libraries we use to curate information for students on that specific topic for that week. So going back to this layout in our syllabus not only do we have this level of information in the syllabus but within each of the different classes when we begin the class we also explain those subtopics for example on plasticity. We're going to be looking not only at the definition of plasticity but questioning this idea of critical periods looking at some case studies and understanding protective and risk factors that are related to plasticity. So we indicate that in the live class slide deck which we'll get to in just a second. So I mentioned that we do a lot of research to figure out exactly which subtopics and what information is most up to date. Well we also use that as we are researching to create things that are called bundles and in our course we create at least one bundle per week. For example in the week of plasticity we have plasticity and epigenetics so we have these two bundles of information. If you click on any of those within our course it takes you to another page in which there's a long list of resources that we came upon as we researched the topic as we updated the topic for the semester. And what we've done is take the time to hyperlink each of these articles or videos directly to their source so that the students don't have to do a lot of fumbling around to find the right resources. We do this because we want to differentiate homework. Students get choice. They're asked to come to class having reviewed at least one item within the bundle but they get to choose what and what's fascinating is that we can see that many students dig into far more than just one thing. They also use the bundles for example as jumping off points for the semester projects which we'll talk about in a minute. These days for K-12 teachers we also want to motivate you not only to think about bundles as being scientific articles and videos of Nobel laureates speaking about their topic but you can also include things like podcasts and links to other platforms like for example Khan Academy or Teacher Tube. Or you can include links to apps free apps that students can use like Duolingo to improve language or to educational gaming resources which can also enhance skill sets. The fourth step is to create the pre-class slides. So since this is a flipped class these are the slides that are going to be used to make the video pre-class. So we have pre-class slides some people just call them power points or slide decks or the flipped video slides and a key step as we're emphasizing throughout is to keep in mind learner outcomes as we do the stage of planning. And so our guiding question here is really what is the best use of my synchronous time. So the idea here is to offload some of those dates facts formulas that knowledge based information that's vital but you don't want to spend a lot of time just dictating that to our best use of our face-to-face time or synchronous time might be better used in debate or in discussion or for clarifications or going deeper into the information or using or applying skill sets right. So we offload the other important information but not necessarily something they have to do in synchronous time with us to an asynchronous activity as homework. And we create this slide deck based on what we have decided are the objectives of that weekly topic. So we create that pre-class side deck directly from the information we've gotten from our research on the bundles and those knowledge skills and attitudes for each topic. And depending how you might want to flip your class there's different ways or links of slides. Sometimes people will decide they want to flip an entire class lecture so they have a video that might be 20 to 60 minutes long. Other times people choose to do little bite size nuggets of highlights. So maybe these are three to eight minute videos having two or more of those. And yet other teachers just make a quick video introductory saying, hi kids this week we're going to be looking at dinosaurs and they're eating habits and you're going to be watching a video that's been made by whatever. And you just use yourself as a bookend to introduce somebody else's video which means that your video itself is really really short. So depending on your choice of how to flip different types of slide decks. If you do what I call like a real flipped class is really your entire lecture this will permit you to go far deeper into the material than you have ever gone before. But you have some choices to make. You should not flip everything that would be in a regular class. And so if you ask yourself what is the best use of my face to face time you need to offload the things that aren't the best use of your face to face time. So if we want to do things that are more application based or if we want to introduce a new problem or collaborate together or to rehearse a skill set that's great use of your synchronous face to face time. So we need to give all the prerequisite information in the video beforehand. This means we're trying to leverage the use of this technology to support our learning goals not just sort of shove whatever we can into technology and make some good choices here. One way to think about this is to use Bloom's taxonomy right. At the basic level of Bloom you have a lot of definitions and facts and just being able to understand core concepts. I would suggest using that taking all of those dates, facts formulas, theories, information that's really important for students to have but they just have to memorize it right. Use that stuff at the bottom of Bloom and use that to flip. But also remember your pre-class video shouldn't be like full of like just that boring stuff. You really have to use that hook and engage students in the information. Sort of frame things perhaps as a problem, case studies as some kind of a narrative to give your videos a little bit more life. So once you have the pre-class slides we make the video. So the pre-class video is sometimes called the flipped video or just the video or the weekly video. And again keep chanting to yourself this understanding by design concept what are the objectives? How am I going to evaluate this? What are the activities I do? And at the end of viewing this what is the knowledge skills and attitudes my students will have? And again we do this by asking what is the best use of our synchronous time and then we offload the things that might not be the best use of that synchronous time. So after that slide deck the slide deck is what creates the pre-class video. So the pre-class video based on the pre-class slides highlights those key ideas that basic elemental knowledge of the topic of the week. The sixth step then is to use that same slide deck that you created to make the video to create a discussion board prompt. Now some people just call this the discussions or the chats or the chat forum or the class forum. Remember however you should still think about this in an understanding by design structure. What will the students be able to know, be able to do and value after they participated on that discussion board. And for us the discussion board is a space to create community. So a guiding question we might use is how can I get students excited around a topic that stimulates debate and encourages a learning community. So this means from that same information we can create this discussion board prompt. And the discussion board prompts have the same instructions every single week so students don't have to use a lot of cognitive load there but rather use their energy on the prompt which we try to keep pretty short. So they look at the prompt, they see what they're asked to reflect upon and what they're asked to give evidence for. And they make that first attempt at their prompt, they write out their ideas and then they're asked to respond to two other learners prompts. And so this creates a back and forth between concepts and between ideas between individuals in the class. So similarly based on those slides it came from those specific objectives we create the quiz questions. So a quiz might just be what you know as a test or a knowledge check or a section review or a chapter review. The idea is that we still ask ourselves why would I have somebody do a simple multiple choice true or false test? I mean that's very superficial knowledge isn't it? It is however it is fundamental to being able to apply that information. If you're unclear about if one theory said one thing and another theory said another it's very hard for you to be able to discuss and use that terminology. So we use the quiz in our class because we know that frequent low-stakes testing is a fabulous way to enhance memory and it also gives us the assurance that everybody shows up to the live class with a certain level of vocabulary so that we can have deeper discussions in the class. So a guiding question here might be what is the basic knowledge and vocabulary I want to be able to use with the students on that topic of the week. So again coming from that base of knowledge knowledge skills and attitudes we come up with our quiz. And these quiz are not rocket science basically you know they're true or false there are multiple choice and the beauty of it if they're well conceived and thought out and definitely test these out before you put them up but they are self-graded so you don't have to it's pretty hands off kind of an experience. You just think of some great questions put them up there and then hopefully you ask the students go ahead take it a million times just get a perfect score but try to take it once before you do any of the work to get a baseline what did you know before you've been coming to the class then take it again after you've watched a video and after you've come to the live class and then try to get that perfect score on average our students take each weekly quiz about four times and they have a very high average very close to a hundred. Once they've done that we use the discussion board replies and we use any frequent mistakes in the quizzes we look at the discussion board content questions that some students will submit and we look at the three two ones the reflection papers from the previous week and that helps us develop the live class slide deck so the live class is what we do in our synchronous meeting time so you might call it the live class powerpoint or the synchronous powerpoint or the live class presentation or the live class slide deck all of those would mean the same thing to us and this does not mean creating a totally different presentation this means taking the core of what you used for your pre class slide deck and then enhancing one or two of the key ideas that are there that you're going to go into depth on based on the feedback you got from the discussion board so after they come to the live class what knowledge skills and attitudes would I expect of them so the main guiding question here is you know what is that in-depth understanding I want students to have after they come to this live class so this means that based on the replies to the discussion board and based on the pre-class slide deck and those mistakes that might have come up in the quiz we decide what is important to do in the live face-to-face class we want to remind you that every time you lay out that live class slide deck that you should really frame it you know this is what we're going to do today and also offer that summary in order for people to keep their heads around where they are going with the information and how far along we are with that process so now the ninth of these 12 steps is when you actually have the face-to-face synchronous class with your students now this means that the students have already done quite a lot of work before they get to you that's the typical nature of flipping they have to do a lot of work before they come to the live class most of the work is before they come to the live class however when they get to this live class or the synchronous meeting or this face-to-face encounter here what's really important is that you have created a class protocol this gives students a regularity of expectation of what occurs in the class which reduces their cognitive load they don't have to prepare for something totally different each time they understand the basic structure of what you're going to be doing in that class encounter which allows them to be able to pay attention to the right things at the right moment so this means that within the live class we do several types of activities one important one has to do a small group work and the discussion and collaboration on activities and application of the information so our protocol looks like this don't feel obliged to do absolutely everything but we get in the class a few minutes early and as people come in start to come in little by little we greet them we say hey Jenny good to see you again how's everything going we use their names and we also allow them to sort of you know vent slash transition for more of it is they're coming oh no fine I have I'm just settling down it's been quite a busy week great well tell me a little bit about your week so that they can feel this very natural move from being in one space to coming into your space we engage in a lot of short dialogues they can be about the class or not we let the students sort of drive that and if they have any class related comments we are sure to jot those down and be sure to integrate them into that live class the point is that we want them to feel recognized as members of our community as they come into that virtual door so as we begin the class we basically announce that we're going to be recording and we start the recording right and we thank people for arriving on time I know that sounds counterintuitive it's your class they better arrive on time actually it's their life and it's busy and they have made a way to get to your class and we got to thank them for being there we also use this opportunity to make sure that anybody who doesn't have their cameras on turns them on and make sure that their names are not their emails or whatever that they have written their name the way they want to be called by others and then we do a very quick question in the chat for example write down one thing that really struck you about this week's pre-class video that you want to be sure we talk about today or something like that why do we do this this also helps us take role it's a really fast way to make sure that people were in the class on time and responded but it also helps us put our finger in the pulse of any big issues that might have come out of the videos that we didn't already incorporate into our pre-class slides we also want to remind you to use the chat extensively it's one of the greatest disruptors that we have in educational practice it's like inviting students to pass notes behind the teacher's back but use it wisely and it really does have a whole lot of benefits for example it offers immediate clarifications to students on questions it also encourages this kind of a back-general conversation that dives deeper and personalizes some of the examples for students and of course the teaching assistants are in charge of manning the chat basically reminding me every few minutes if there are some key points that are coming up that we need to discuss in the larger group if you don't have an assistant go ahead and ask any student to do that it's an incredible skill set that they can develop to learn to synthesize and to scan that information and to be able to pull out those juicy points that would be of interest to other students so take advantage of this wonderful opportunity to get those clarifications and use that chat throughout the live class presentation so let's look at how we begin we begin by showing the basic slide deck which will include the objectives of the day and so we explain them and we also display them on that PowerPoint and we ask students if they are happy with the agenda of the day or if there's anything that they really want to make sure we cover and after we've gone over the agenda the very first agenda item is always your questions and ideas and those are things that we have to hold from responses from the discussion board so as we start with students ideas we begin by sort of summarizing the way that we group them or these patterns of responses you know three of you you know Jenny Anna and Tom you really called out this one issue but I thought it was very unique the way that you approached this Tom can you elaborate a little bit more on what you wrote in the discussion board now that gives Tom the microphone and the chance to synthesize some of his main ideas which calls out points from other students and what's so special to see is that you'll often time see them congratulating each other within the chat and you know what a terrific idea Tom or I hadn't thought about it that way or excellent point and this is a wonderful way to give peer feedback throughout the course so for each of these summary points we usually choose one or two students to speak and really celebrate what they have taken the time to write in their discussion board posts. After we've gone through all of the things in the discussion board which can take very easily a half an hour or so to go through those key ideas and to make sure that we've sort of vetted the most important patterns of replies that have come out in the discussion boards then we turn to our pre-class by deck which we've now taken one or two of those big ideas and we've chosen the things that we think need to be highlighted even further and we contextualize that information and we remind the students of that core vocabulary that was in that pre-class slide deck and we talk about the controversies that might occur within the field around that particular topic then we go into breakout rooms in order to get deeper into a group discussion we normally put about three or four people into a breakout room in order to have a really good chance for everybody to have a talk we normally go to the breakout rooms between eight and ten minutes although every time we come back students say oh we could have spent a lot more in there because they get really into it and they have the opportunity in such a small group that everybody gets to talk but before you send them to the breakout rooms make sure that you frame it with a kind of a prompt we use a lot of the visible thinking ideas you know I see I think I wonder oh we tell them that they're signing this come back one thing that they all agreed on and one thing that was controversial or something like that so that they come back they're in a similar framework if you just send them off and you say okay how did everything go you can get all kinds of different perspectives as opposed to something more unified so we recommend that you send them off with a specific task that they return being able to articulate their findings in a certain format and then after they come back from the breakout rooms we have them share in this larger group and when they come back I'll call on just about anybody somebody who has probably not spoken yet already and I'll say so Anthony who was in your group with you now this really forces them after the first or second time they realize she's gonna ask me their names I better know people's names so they get to know each other better and they'll say well I was in the room with Anastasia and Victoria okay terrific and tell me what was the one thing that you discovered you had in common what was the one thing that surprised you or for which you had great differences and this is fabulous for their power of synthesis very interesting you'll see that some people are naturally already good at this they already know this they've taken notes in a way that they summarize their ideas and in 30 seconds they give you the whole 10 minutes encounter then there's other people who have a really hard time synthesizing their ideas and they'll get stuck on what they thought on what they thought on what they thought and it's very important to subtly cut that off and to say I hear what you're saying that was really an important point to you did Anastasia and Victoria find the same things make sure that they realize what they're doing is reporting for the group and can they synthesize other people's ideas that's a huge level of thinking so after you've done that one person two people three people depending on the size of your class you may be calling on seven or eight people to give that summary that can take quite a while but it's well worth doing allowing people to share back groups and to hear each other and always at the end try and say well I hear at least two patterns of things coming out of what you guys did in your small groups what patterns are you guys seeing and that will prompt them to think about globally what is this group saying how are we understanding this information this is fantastic because it goes from that individual thinking to that small group understanding to this larger group synthesis to try to understand where are my thoughts in context to the others you basically repeat this until just about everybody has had a chance to talk also always create a space in the beginning middle or end of this exchange with the students to say by the way if there's additional ideas that came out of your group that you'd like to be sure are also heard please put them in the chat that's a nice way of saying if the guy who's summarizing for you isn't doing a good job add on to that so you can create that space in the chat so everybody feels that their voices are being heard and taken into consideration now something to always remember as you are doing this live class slide deck and deciding what are those topics you want to go into remember it's depth over breadth when you flip a class this means that this kind of more superficial definition information is given in the flipped part so that in class we can go deep we can get clarifications we can get to those subtle nuances of the information so the idea is to give that space where students can go deeper into their own understanding of those concepts as opposed to adding in totally new ideas onto the class so the synchronous time remember should be devoted to helping students get any clarifications they need and to seeing real life applications of the information and always remember let the students lead they generally know what they need and if you ask them and give them the space they will respond accordingly so if you have a long enough class and you can go into a second topic and do that whole kind of small group activity and then come back to the larger group that's terrific we have a luxury of a two hour long class and this works perfectly to break up the time keep the tensions focused and to go into depth and at least two topics in our class but if you have a shorter class you might only have enough time to do one breakout room now after you've gone through these topics gone into depth come back out before you in class do not fail to summarize and contextualize all that information that came out give a less than one minute summary okay we did this today that today and the other thing today we also heard some great examples from Alana in this case study that she offered us and from what Cynthia mentioned call out those particularly good contributions to the class then be sure you put all of that information from what we did today into the context of the larger class one way to do this is to have a connector what we always do at the end of our classes look at risk and protector factors that fits into our topic we say okay so now having gone through all of that information about whatever nutrition or the role of sleep or how memory systems work or whatever it was we ask the students to think of what are the risk and protective factors in your life that might enhance or inhibit your ability to take advantage of that for better learning so you can think of some kind of a protocol where you continuously do this at the very end so that you also give students the sense of closure to the information then finally point towards how this connects to next week's topic so you're building bridges between you know what we did in the past what we did today and what's going to happen in the future and why all of this really fits and makes a lot of sense together the very last thing we try to do in the last five minutes of our class is reflection and we ask students to say three things that they didn't know before class two things that they're curious about and want to know more and one thing that they might change in their personal or professional practice and we use the three two one as a way to gauge their participation grade but it also measures their development over the course of semester and at the very end of the semester we show them an accumulative list of three two ones throughout the 15 weeks for all of the people in the class which really stimulates a great appreciation for all that they actually learned in the class a secondary or additional benefit of the three two ones is that it serves as a feedback to the teacher if the students are calling out three things that you didn't necessarily think were the most important things or if the students are asking for more information they're curious about two things those are big heads up to help you know what types of information they now need to move forward it also gives you some feedback on your teaching have you included all of the right elements or were the things that stood out to them not necessarily the things that you brought up but things that the students were bringing up which you need to pay more attention to in the future reflection is super underrated in schools having downtime having the time to step back and say you know did I really learn anything or what's going on here is incredibly important and we don't give enough time to that so highly recommended it this has been actually one of the things that our students recall years later and tell us this is one of their favorite parts of the class is having just a few minutes to themselves quietly to think about what just happened in the past two hours what what did I really learn here what are the things that stand out to me and finally at the very end of our class when the time is up the time is up and we end on time and we tell them it's in honor and respect out of your time that we're ending right now if anybody has any loose questions please stick around but for those of you have to go go and we say this because we want them to know that we think their time is important their lives are important what they have to do is important what we find very interesting is almost more than half of the students end up sticking around just to try to catch up with those questions or clarify or hang around as they're writing their three two ones to listen to what other people are questioning which we find pretty fascinating but this time is vital because allowing them to stay after class for those few minutes actually saves us time in office hours for example if they're able to just get it out and it's also still recording if they ask a question that's important that we're able to respond immediately that's fantastic because it allows other people to hear I hadn't thought of that but I'm glad she asked it because now I don't have to ask it and that saves us time so step 10 step 10, 11 and 12 are things we do throughout the entire instructional design process it's not something that you have to do at the very end it's something you should think about throughout the big idea here is that evaluation or what some people call assessment or feedback or formative embedded assessment or diagnosis depending on the time that it occurs within the learning process all of this should be something that occurs throughout all of our evaluation it should not be something that happens at the end the biggest idea is that evaluation is a teaching tool it's not something separate from the learning process in fact people learn a whole lot from the evaluation process so the big idea or question to keep in mind here is how are different evaluation tools used in this instructional design what are we trying to measure and why so some of the class activities including evaluation that we do are divided into asynchronous and synchronous activities initially the students are on their own it's the flipped class right they take the quiz they watch the pre-class videos and they respond to the discussion board prompt and they explore the bundles or these many libraries that we have built right and based on some of the errors in their quizzes or some of the questions they might post on the content and logistics board or some of the patterns we see in responses to the discussion board these things help us do the synchronous structure of deciding what goes into the live class so based on the pre-class video we take that we strip it away and only keep one or two big ideas from that based on these other inputs we also have synchronous meetings when we go into sections in our course we usually have one to two sections every week in which we do a deep dive into a certain aspect of the topic of that week and we also have other workshops that help students with their semester projects after any of these live synchronous encounters we ask students to do a 3-2-1 three things they didn't know before two things they're curious about one thing that they want to change about their personal private lives based on the information presented now based on these workshops these are done to help students manage and make their way through their semester long project which is a very significant thinking as writing process we go back to the synchronous work which is basically two big things the first is that there's an ability to do over to rewrite information and there's also the ability to go deeper into the information so we find that even if they don't have to students will retake the quiz over and over and over until they get a perfect score and some of them will re-watch those pre-class videos and others about half of the people in our class watch the recorded two-hour session when we were live mainly because they say there's so much going on the chat that I didn't pay attention to what was being said or vice versa, right so they go back and re-watch the video they're also encouraged to go back and to extend those conversations they began with a discussion board but most of the time people are digging deep into the bundles they are digging into the materials they call it their rabbit holes they usually get a little bit sidetracked and keep reading and reading and reading even though they're told they only have to look at one of the things we have an average of about 100 resources in our bundles and students will look at 10, 20 of them just because they're interested in the topic this means that our evaluation scheme is ongoing it's embedded and it's continual and it's formative for the most part we'll talk a little bit more about that in a second about why we believe in a rewrite policy but more than anything all of these activities because we're online are incredibly well documented we have quite a lot of evidence for how many times people do these do-overs for example, how many times they take a quiz how their different iterations of their projects went we can see this development over time which is incredibly well documented and as we mentioned before because it's a balance between these asynchronous activities students have a great deal of flexibility in terms of time in when they do those things so just a clarification on our rewrite policy it's a lot of work however, having said that this is perhaps the comment that we get every year at the end during evaluations about what was the most important thing that allowed them to learn in this class so do-overs the ability to learn from your mistakes is something that students tell us has a significant impact on their learning and which is also supported by John Haddy's research and this is also part of this educational paradigm shift because typically as a profession we do not bake into our process enough time for a liberal rewrite policy which means to be able to apply this you really have to have a lot of great planning in which you've already incorporated the time for rewrites within your calendar structure our basic policy is that they have one week after they have been given feedback to resubmit to us so this means that we measure different levels of competencies and attitudes with different tools along the way meaning we're sure that we get to the core aspects of a course, attitudinal shifts with the most tools step 11 is another one that goes across the board throughout the entire class and that is communication how does that teacher-student exchange occur nor are there any formal structures there and so from the start of the course it's vital that students know where to find information and how they get help this means that a guiding question here would be how will you communicate with your students to ensure regular contact clear expectations and a seamless flow of information just allowing some kids to maybe text you on your phone and others to send an email and some others will come to office hours it's not good enough you have to create clear parameters and so in our course we use the emails that's within the platform you tell them don't email us to the platform we use announcements we invite their comments to our comments on the feedback that we give them for their discussion boards and their 3-2-1s during the live chat we give them feedback about their comments or where they've taken information and we have an additional discussion board that's only meant for content or logistic questions so they can go ahead and post something there if they say you know I watched the pre-class video and you said X and I've read Y can we talk about this so if they've posted it there that gets worked into our live class presentations we also have office hours that we invite students to come to and as I mentioned before coming a little bit early to class and sticking around a little bit later does a whole lot to facilitate the communication within a classroom the one thing we definitely discourage is using like instant messaging things in which you are on call 24-7 we tell students that we want to keep the communication chain within the Canvas platform because it helps us document and that's been crucial in the one or two cases we've had over the past seven years in which a student might have had some problems that really needed to be kept in the classroom and not in private emails so do try to formalize your communication patterns with your students independent of which of these tools you use all of them none of them just one of them regularity is really the key to good communication if students know when they can find you and how they can get information that's more important than which of these tools that we're actually going to apply so just make sure that you created a really clear structure of how you're going to communicate with the students so the 12th point is the biggest stickler here because it has to do with a total paradigm shift in the way we think about education and this has to do with time the hardest step to consider applying has to do with this paradigm shift in educational practices about what we give the most time to this structure requires you to give more time to evaluation, to feedback to this human touch, this contact with individuals it also requires more time to plan before I used to preach to people that you needed to have a one to one to one structure in your schools where every hour that you teach you should have an hour to plan and an hour to evaluate teaches what I would say oh if only I'm going to go even more extreme on this I think that the current structure we're recommending is actually a three to three to one for every hour of teaching you're actually doing three hours of preparation and three hours of evaluation that totally turns teaching on its head so we estimate if we want to do a reality check now to decide if this is really what you want to do for a class of let's say 25 students over one semester we think that you're going to prepare at least six hours per topic and you're going to evaluate at least six hours per week giving feedback or correcting reflections or mentoring on those projects or doing sections or workshops or answering those emails they send you and just two hours that you'd be face to face with those students plus that time that you would have before or after class so ideally what you do is you'd have the summer to do that prep work you know to do the flipping element to record those videos to get things structured and then teachers should never have more than five perhaps it's impossible time wise because that's working more than your eight hours a day as well as 12 months a year so this really means that the summer off that teachers theoretically get is not a real thing and that they really are doing full time work even though they might not be standing in front of their kids more than 10 hours a week they are putting in so much more time on the back understanding that evaluation process making sure that those kids feel motivated giving that one on one nudge that's needed often in classroom time so it is a change in the shift of the way that we think of teaching and teachers so in summary what are the pros and cons of all of this if you had to think am I going to do this or not it's a big decision think to yourself just as we mentioned right now it's a change in mentality to actually go about doing this and some of us are just sort of keeping our heads above water with these big changes of going online but if you are one of those people who are willing to invest and decide I am going to have the best online classroom possible shift in mentality won't be that hard but it is something that's going to take a little bit of effort right the benefits of all of that is that there's much more focus on this humane aspect of teaching and the contact with the students it's really much more student centered to structure things this way students report that it's far more personalized and that we can differentiate because of the offers within the bundle meet students at their entry points not just where we want to start the curriculum but you have to accept that this means a lot more front end planning this means taking the time to structure things having said that if your course is well structured you just tweak it slightly every year so the amount of investment of your time of structuring the course goes down after time it's far more better documentation we know there's a lot of other benefits here and at the end of the day it really means that teachers are doing a lot more listening and less talking and students have a lot more interaction so this high touch structure to online learning is something that's proven extremely beneficial over the years students report years down the road that they still retain the things that were learned in this format so I would say for those of you who are willing to do it it is really well worth it so in summary we looked at our starting point and we said you have to decide your global objectives of the class you have to lay this out in a very logical and organized way as far as the way that you present your topics so that they have a clear learning trajectory and once you've done that then you have to disaggregate those specific objectives for your class for each one of those weekly topics by going deeper into research and understanding the bundles that you would want to supply to your students so that they have additional resources you can then create that pre-class slide deck and do the pre-class video develop your discussion board questions and create that quiz and from the results of your discussion board and the results of that pre-class slide deck and the results of your quiz you can then develop the live class slide deck and think about those activities that you would do including that small group work where students are applying information or discussing or debating information and after that you create your 3-2-1 that would be a fantastic instructional design in summary since Rebecca somebody who's worked with me on this course for many years created this graphic in order to try to separate out all of these key elements the things you do before during after class the types of activities, asynchronous, synchronous things that are done in groups and alone the things that are digital things that are analog we invite you to have a look at this conceptualization of our Harvard course think about the elements that you would like to include in your own classes and be sure to let us know if there's anything we can do to support your work in creating the best instructional design for your course focus so what did we do with you today in this video we tried to go through 12 different steps towards a successful instructional design we looked at these one by one tried to give you an example of each of these a worked model so that you can decide which pieces of that you want to incorporate into your own work and additionally we really want to make this real this type of a class takes a lot of work but if education, if learning is your goal I really hope that you will embrace this as we mentioned before there's a lot more front end planning but the payoff is definitely worth it the time with the students the depth of the information you can cover and most especially putting that humanity back into education is really evident through this model so if you have any questions don't hesitate to get in contact please look at the other free resources we have on the learningsciences.com and best of luck with your instructional design