 Education of your science, right, scientific research. But for many of us, we wear another hat, and that is as educators. So many of us are currently teaching classes. If you're not teaching now, you will be teaching. A number of you will be teaching. And certainly, all of us have taken classes and courses. So what I want to talk a little bit about are some ideas about how we might improve the experience of education. This is really thinking about classroom education courses and the like, although some of these ideas can be applied very, very broadly. So the first thing I want to start out with is to talk about the dominant way in which we have learned stuff, and that is like this, right? Someone gets up in front, talks for some period of time. There's a collection of people in the audience listening. Traditional lecture. And one question you might ask is, well, if we improve it, how effective is this method of teaching? So a little story. So there's a famous physicist, Nobel Prize winner, Carl Weiman, who won the Nobel Prize, something like 15 years ago or so, for work on atomic and molecular physics, experimental realization of Bose-Einstein conversation. He no longer does that. He actually works full-time at Stanford, works full-time in education research, physics education. But early on, while he was still a researcher in atomic and molecular physics, he was interested in education all along. And it was giving a lecture and explained to his students it was some basic physics course about sound and how it's produced. And when you hear the sound, what is the dominant, where does that sound come from in the sense that what you hear, what is the actual sound producing the vibrating object which actually is responsible for you hearing directly the sound? And explained that it's basically from the vibration of the wood in the back of the violin or of a stringed instrument like a violin. So then in this lecture, same lecture, 15 minutes later, he asked the following question to his students. The sound you hear produced from a violin is A, mostly by the strings, B, mostly by the wood in back, C, both equally, D, none of the above. So it's 15 minutes after he had said what I had said before, the sound is produced by the wood in the back. The question for you is not this question, but this question. What percentage of the class answered the question in the previous slide? This question answered that correctly 15 minutes after hearing that. So we're going to do a little exercise because we use this later. This is actually a useful device to use in the classroom, a way to pull the audience. Let me explain how this works. So for responding to multiple choice questions simultaneously, there's this clever little answer card. And what you can do is you can fold this up. And so you can basically choose whichever one, however you want to fold it in a way, so that you can produce or show only one such answer. And actually, so you have four here, there's another way to do. If you have five answers, you can show a blank. So the reason you do this is when the instructor asks for, so you pose a question like this, then the instructor will say, ask the students to respond. And the students can respond simultaneously and relatively anonymously, rather than raising hands, students can say, like if they think the answer is D, each student can hold the card folded appropriately for their response in front of them all at the same time. And the instructor can then visually, by looking at the letters and colors, get a quick read on where the class as a whole lies with respect to that response. Now, there are more sophisticated ways. There are ways to collect the responses through wireless. There's Clixo called clicker devices. But you don't need that to make this kind of method work to pull the audience to get some feedback. And I'll explain why you would want to respond anonymously in a moment. In some cases, people are just shy about responding, raising your hands. This gives a certain comfort level to responding. But there's also a pedagogical reason which will show up in a little bit when I pose another question. OK, so that's the question for you. What percentage of the class answered a question about the violin correctly? OK. Yes, let me show you the question again. The question's this. In the lecture, the Nobel Prize-winning and very good lecturer, no well-known teacher, prize-winning teacher as well as Nobel Prize-winning in research, explained that sound of violin comes mostly from the wood in the back. 15 minutes later, he asked the question, the sound you hear from a violin is produced. Obviously, in this case, the answer B is the correct answers. The others are incorrect. And the question for you is, what percentage of the class answered B? Which percentage of the class of these four percentages answered that question correctly? OK. So on the count, not yet, OK? Why don't we say one, two, three. Hold up your cards. So I'm getting a quick survey here. There's a variety of answers. OK. I would say a dominant answer. D is the dominant answer, and that's the correct answer. It's not 90%, 60%, 30%, but it is 10%. Well, you might say, well, OK, this is freshman. They're not paying attention. Well, he repeated this experiment. Repeated this experiment, he gave lots of colloquia to physics faculty and graduate students and postdocs. He didn't ask the same question. He was in the course of his lecture, said something. That was not, so it's not obvious. People who jump to the answer typically say something about the strings being where the sound comes from. So there's a certain counterintuitive nature to the question, but nevertheless explained very clearly in the lecture what the correct answer was. And then he polled the students, not in this case, the audience, which was physics faculty and physics graduate students and postdocs. And what do you suppose the percentage of the correct answers from that esteemed audience was? 10%. 10%, yes. So this has nothing to do with the fact that the audience or the class, if you will, is freshman. This is a statement about the impact of lectures and actually how little in the course of a lecture one actually learns. And the explanation for that is that in the typical lecture format, when the learning is passive, so the audience, the students, are sitting there and listening, they're following the argument. It makes sense. They're nodding their head. But in the sense of being able to actually use that knowledge, that is, when asked a question, when they have to bring it into play to recall, it turns out that way of communicating the information because it's passive in the sense of the way it's acquired is simply by listening, that turns out to be not a very effective way of learning. So there have been in recent years a number of techniques to apply in the classroom to do what's called interactive engagement, to get the audience more engaged. We'll say more about the social science behind that or at least briefly touch on that a little bit later. But the idea is the more active the students are in the course of learning, the greater the amount of learning that goes on during the lecture. So one way to achieve this is by so-called peer instruction. And this also leverages another interesting fact that sometimes when a student is learning something new, the explanation given by someone else in the course who has now just understood it, that is, a peer in the course, is more effective than the explanation given by the lecturer. And the way this can be brought into play is the following. So the way this works is in peer instruction. So you give a lecture going on. There's some discussion. And then afterwards there'll be some number of steps. So instead of going on for the entire class hour, there is interruption in that activity. So let me just give the mini lecture for the question that I'm going to set up. So it's Newton's second law, which we can write as F equals mA, where this is the net force on an object. And this can also be written in terms of the momentum, the time rate of change of the momentum, which if the change in time we think about, we can do the finite time approximation for this, which is the change in the momentum over the change in time, the ratio of that for small times is a good approximation. And in fact, when this net force is constant, then this, of course, is exact. Now, we can rewrite this left and right hand side, rearrange this in the following form. That is, the change in momentum, which is the final momentum minus the initial momentum can be written as F net times delta t. So that's the so-called impulse momentum theorem, end of lecture. Now, rather than keep going on, we can stop and pose a question that now engages the audience to actually take that bit, which was just that bit of information, which is communicated, to actually do something with that in a more active way, other than just listening. So he's a question. So I have two disks. And imagine you're looking down from above. These two disks are lying on a table. The table is very smooth. It has no friction, OK? There are different colors here, but they're identical in size, diameter, as well as mass. And what we're going to do is start with both of these disks at risk and apply a pull, OK? It's indicated here. This is a force that we're applying. The same size force, same direction. And we're going to apply that force for the same length of time. And we're going to ask the question. After three seconds, which disk has traveled farther? Now, notice they're not quite the force, which, as you imagine, let's say, you're pulling on a string. And the string is attached to these two identical, otherwise identical objects somewhat differently. So the string on this top guy is wrapped around the edge. And basically, as this string is pulled, or as the force is applied to the string, that unwraps the string. And in the bottom, the string is tied to the center and pulled. By the way, we're applying that same size force, same direction of magnitude, same direction for the same length of time. And the question is, after three seconds, which disk has traveled farther? So that's posing the question. Here's the next step in that process. Each student is thinking about that question and selects an answer. So typically, you give a minute or two. So I'll stop talking and no talking yet amongst yourselves. So we'll have that give a little bit of time for you to think about that. And here's the question again. Now, what I'm going to ask is that now at the end of this period, this is in this step, I'm going to ask everyone to vote in the same way as we had done before, at all at the same time. When I count, at the count of three, you hold up, fold your card up to select which of these answers. I'll show this again. You fold the card up. And at the count of three, hold it up in front of you so I can see, but most anyone else can't see what your answer is. So one, two, three. What you do now is turn to your neighbor and try to convince them that you're right. Compare answers and see whether you're different and you try to convince each other. Huh? What? The purple guy can you go outside? Oh yeah, sure, sure. He could. Go outside? He could? Oh, OK. He could? Just sitting, it's like a hockey puck. No, but the black thing is not a physical mount. No, the black thing is the table, if you will. The edge. No, no, it could go off, let's say. Yeah, it could go off. Anything it would, you do whatever it wants. Anything it wants. Stop, let's stop now. Second? OK. So I would normally let my class go on longer because there's a lot to discuss, but this is physics one, like freshman physics. But this is a non-trivial, non-intuitive type problem applying a basic concept from physics one. So now the next step is, after having discussed, so there was no talking, you turn to the neighbor, you compare to answer, you try to convince your neighbor, what we're going to do is we're going to vote again. So let's see if there's been any changes for as a result of this discussion. So same procedure. If you haven't changed your mind, just hold up the same letter as before. If you have, of course, make the change. And I will ask you to show this at the count of three. Discussions over. You've got to make a decision. OK, we're going to vote now. On the count of three. One, two, three. Not much learning going on here. OK, so I would judge, actually in this case, that there was still a variety of different opinions here on this. Let me, so what you would typically do, if there are difficulties, you would revisit the topic. If it was mostly correct, you would move on to the next topic. Let's revisit this, though. We need to revisit it. So actually, what works really effectively for this is to do a demonstration. Because now you've invested, you're like, OK, can I see something that actually tests this out? So we don't have the equipment here. We can do something that shows this set up with a demo, which does this actually live. But I have, I can do the perfect physics that Lorette talked about in this context by doing a simulation. So let me do this Python simulation that we wrote to demonstrate this. So basically, it's the application of Newton's second law to these two situations. And I have to get in the network. OK, hang on one second. Kick me out of the network. OK, so this was a visualization Python program just implementing Newton's second law impulse momentum theorem. So same size pucks on the frictionless surface, pulling the same force in the same direction. Only differences were applying in the bottom case, the force at the center of mass. In the other case, we have a string that wraps around there, unwraps, without slipping, but essentially applying that force, that same force at the edge rather than at the center of mass. That's the essential difference. And what you see is that as you're pulling both, they both move at the same speed, same velocity throughout the entire motion, throughout the entire time. Correct answer, answer C. OK, answer C. All right. Now, let me just say a little more about this. Just to say, how do you get this idea from this? Since you're invested and you've now spent some time thinking about this, so the way you can think about this, this is the situation. As I said, we apply the same force, same constant net force frictionless. It's that red guy, that pulling force f. Over the same amount of time, that's equal to the same change in momentum. We start at rest. This goes to 0. So at all times, they have the same momentum. They are the same mass. That must mean they have the same velocity at all times. And therefore, they move together. And therefore, neither wins the race. They tie. So that comes out of Newton's second law. So why is this? I think this gives you a flavor for it's a different way to actually introduce or talk about material in the class. And what it does is it provides that engagement makes the learning more active. And we'll say a little bit more about that. Let me say something about this, the measurement. So what's the actual measurements that people have done to show that this really has some effect? So this is Eric Mazur at Harvard, made some measurements. And there's a multiple choice test for mechanics called the force concept inventory. And he basically gave that to his class first when he taught the class using traditional lecture, and then using basically this various forms of this peer instruction. And what he found was that the amount of learning, so on this scale, the way you give this kind of test is in the sort of pre and post manner. You give the test first, let's say at the beginning of the term. The class has proceeded through the term. And then give it the same test at the end. And you measure the gain. How much better did students do on the test after the course than before the course? And on this normalized scale, typically, they will have some score, which is less than perfect. If they improve perfectly, everyone aced the exam. You would get one. But you get something that falls in between, a gain. And what you see is with these sorts of techniques, you get an improvement over and above the measurements that you would get for just standard lecture. There is a larger study that was done, which is in the same sort of format, the amount of gain. Actually measured as a function of the baseline score. And what you see is that this different symbols, basically the open symbols are with the interactive engagement. The hash symbols down here are with the traditional lecture. And for something like 3,000 students and 50 different courses, this is in high school and college, you get the data suggests there's improvements. Yes? Can you say what terrible graph that is? It's a terrible graph, yes. It was pointed at the same point. Well, it's a graph, but it's also been reproduced several times. But this is shaded and this is right. Anyway, point is, this is not my other lecture, this is this lecture. Yes? My point was that snapshot people, this is a terrible graph. This is a terrible graph, I agree with you. But the upshot is that it shows that these sorts of techniques do have an impact on learning. And let me say a little bit about now the background. I won't say a lot about this because there is a lot to say. There are lots of ideas. But there are a lot of aspects of how we think we learn, which are actually quite different from one of the way people actually learn effectively. This is probably one of the biggest ones. The clear explanation. So you think, students think, faculty think, everyone thinks, the instructor gives a very clear explanation of some bit of knowledge. And of course, that knowledge is acquired at high rate. And the kind of metaphor that this sort of way of thinking about learning, that the clear explanation is, learning kind of goes like this. Here's the instructor. The instructor is here pouring knowledge into the heads of the listening. Their students are listening. And if that explanation is clear, then a large fraction of that goes in and this is taken up. Well, when the instructor gives the exam and they find that the students don't do well on the exam, then it's obvious that the instructor is not at fault because the instructor gave a clear explanation. Must be the student's fault. The students, on the other hand, said, I didn't understand what the instructor really said at all. So it must be the instructor's fault. And everybody's frustrated. And this is not how learning works. So let me give you some simple ideas drawn from the science of learning. It's a separate discipline. It's not the content that we typically deal with. It's from areas of psychology, specifically cognitive psychology. And there's a picture that, basically, learning works, roughly speaking, in what's called a cognitive apprenticeship model. And it says follows. You watch someone. This is not just for classroom learning. Any skill can be in sports. It can be in many areas of life. First step is to see how something is done. That's important. In fact, that's the function of the lecture in some sense. Just to have some demonstration of an expert, model what to do. OK. Now, the next step, though, that's not the, per people watch. Because I watch someone, this is, you know, golf. You watch who's a famous golfer, Rory McElroy. Pick your sport. You watch Lionel Messi on the pitch on playing football, soccer. Just because you watch someone who is really expert at it does not mean you can watch that person many, many times. It does not mean that you acquire that skill. What you need to do is you need to practice. That practice is most effective if it's deliberate. That is, it focuses on key aspects that will lead to improvement. And typically, for the learner, the way you determine what needs to be practiced and what's effective at helping you get better is to have a coach. So the purpose of the coach is to provide the feedback that you need. But without the practice, which then the coach can observe and then provide feedback, so that requires practice on the part of the learner. This practice with feedback. And then over time, as the practice increases, the support by the coach is faded and that then to a point where the athlete is ready to perform or the student is ready to take the exam. But there's this sort of process where it's not enough just to see how it's done or hear how it's done, but there has to be this active, that middle step is where this interactive engagement fits in the classroom, that practice that helps in the presence of a coach to help get better. Now let me say a couple of other things. This is a very good book, which the literature in cognitive science is vast, difficult to read. But this book really is a very practical guide. It's called Make It Stick. It talks about the ways in which we have illusions about how to learn and how, in fact, cognitive science suggests the ways in which we actually are effective at learning. It does so in a summary way, and it also provides very practical tips for students as well as instructors. So I really highly recommend that. Let me give a few, just touch on a few of the ideas in this book. First, it talks about this notion of the role of memory. How do you acquire a skill, whether it's physical, whether it's an intellectual, some combination? Memory plays a key role. And the way it could be muscle memory as well, basically it's establishing of the neural pathways, roughly speaking, which really encode that learning in a robust way. And what does robust mean? Well, the process of learning is first there is the process of acquiring the information in what we call short-term or working memory. That is sort of the window into which the gate through which we take in information, which we learn. But that immediate memory is something that we are immediately dealing with is actually very limited in scope. And a standard way to test that is to, in psychology tests, to give a subject a string of digits in rapid succession, well, not so rapid succession. Let's say I call out a sequence of numbers, four, eight, six, nine, and then ask you to recall, how many of those digits can you recall in order from the beginning? The typical answer is seven. About the number in a phone number. And that gives some measure of the short-term working memory. So that's why it's important in a classroom to really be clear, be effective, be focused because in that short-term working situation, the amount of information a student can acquire is really quite limited, the 10%. There's lots that's been said, the amount that's actually at that moment that's really acquired is small because of the limitations of working memory. Now, the process of learning then takes that encoded, that working memory, and then consolidates it in what's called long-term memory. Now, that's effectively infinite. That is, there's a vast amount that as we go through life, we can learn. That is, this transfer, if there's the effective transfer from that working memory to long-term memory. It may be that it's difficult to get at. So it's not just that it's encoded, or let's say, consolidated in memory, but in order to be usable, it has to be able to be retrieved. And so deliberate practice also practices that key aspect of once it's in memory to try to bring it back to use that. Sometimes this happens spontaneously. It may have happened to you when you had a smell or you saw something that triggered a memory from your childhood and the memories come flooding back. That's because all that was encoded, but you were not able to necessarily easily retrieve until you had a cue which allowed you to pull that back. You can make that more intentional and develop cues that allow you to retrieve stuff that you have learned. And the key element in all these tips, which I'll just say a few words about, is that you have to practice, as we said. It has to be effortful. If you don't feel like you're working at it, then it's not very effective at being encoded. But it's not just effortful. It has to be in a constructive way. That's why practice has to be deliberate. It has to be along lines which have been shown are effective at helping you construct to transfer from short-term to long-term memory. Yes? So flashcards are not a bad idea. But let me give you some of the go on and give you sort of the overarching sort of tips and then you figure out detailed ways to implement that. But let me just say something that you said about failure. So actually in all of this, that failure in this effortfulness, there will be failure. That's going to be part of the learning process. That should not be feared, but it actually is your friend. It's not a measure, this notion that somehow, because you may have failed at some aspect, that that's a measure of your ability and there's no way for you to get beyond that. That's actually not supported by cognitive science. Turns out that people, there are, of course, wide ranges, distribution of abilities. But given an ability, there is a huge amount that person with some measure ability can improve upon. And it's through this sort of process of constructive, effortful practice. So failure is not something, even though somehow, and I'll say something about that in the context of testing in a moment where we feel like when we fail on a test and that feels bad and we want to avoid that, but there's sort of two aspects. We want to sort of disentangle the part of failure that's constructive and helps you learn from the notion of the stuff about failure, which leaves you stuck and feeling bad about yourself. Here's one general tip. So just as with lecturing, you listen to something passively, you acquire very little of that. Another related aspect of that is this reading. If you read the text, how many times do we say to our students, read the text? Well, reading itself actually is not very effective. Reading and rereading and rereading, that is, this is in the genre of repetition. Repetition in and of itself, simple repetition, is not an effective way to learn. And there are a couple aspects of it. First of all, it allows an illusion that if you become familiar with the text, the syntax of the text, students get the feeling that that is equivalent to understanding the underlying ideas. And that's not right. So it allows to perpetuate that sort of idea. And then a second aspect of that is in the same category of lectures, poor transfer from short term to long term working memory, just this reading aspect because you become this familiarity is really familiarity in the short term working memory sense but doesn't consolidate in long term memory. Okay, so how do you modify that? Stop and test, like we did for the lecture and the interactive engagement, frequent testing facilitates learning. So if you're reading the text, you stop and try to ask a question about or summarize that in a different way to work a problem related to that if we're doing physics and mathematics but that sort of interleaving or putting in testing and frequent is really helpful. And now I wanna say something about this notion. If you fail at the testing, that is okay. As long as we think about the testing as testing giving feedback, there's testing for feedback, which is called formative testing. There's testing for evaluation that's called summative. The latter is not as in exams, the final exams, if it's very long spacing, not frequent is not very useful for giving feedback. So an effective tip for instructors is to build in frequent low stakes testing which provides feedback to students to make that learning effective to do this kind of, so it forces the practice of retrieval to take what was learned, the application to consolidate, to construct representations in the long-term working memory and to practice retrieving that and applying that. That's why this very makes that much more effective, yes. So what is the rate of this frequent testing? The frequent testing puts a load on the teachers. So how to do that best? I don't know if there's any one way to do that. There is, of course, a balance because you have limited amount of interaction time. So there's some suggestions in the context of, they give a couple of case studies in this book, but something like weekly testing, and the key thing is frequent with an emphasis on feedback, lots of testing that gives feedback. It doesn't have to be long, but it has to be sort of frequent and gives reliable good feedback that helps guide students in the direction of getting better. That's the key element, and you have to sort of figure out how that fits best in your instructional environment for the particular material that you're teaching. A second thing, and this is, I would say, probably relatively obvious, and that is cramming doesn't work, which it actually is effective in some cases for, let's say, passing the exam, or students can improve, because what it does is it loads the working memory, short-term working memory. What is cramming? Cramming, okay, cramming. What it means is doing a lot of studying in a short amount of time. The exam is tomorrow, I'm studying tonight, right? That is not very effective for long-term retention. It can show improvements in the short-term on some short-term goal, like taking an exam, but for really acquiring a material it's not useful. It's better to space. And again, there's something about, there's some time dependence associated with this transfer from short-term to long-term working memory that you have to allow for this consolidation takes time, and that's been shown in a number of experiments. Here's one which may not be so obvious. And that is, you might think, it's commonly thought that if you take, you say, okay, I have to learn a set of skills, let me take the first step and learn that really well, and nail that, and then move on to the next one, and learn that really well, and do the next. Turns out that is not as effective as skipping around doing this, and then moving to the next one before you've really gotten this first step, and do that, and then skip around, and have a varied practice. And the reason for that is varying makes it more effortful, that you're trying to recall when you get, let's say you work on this first part, this first step, you go to the second, and then jump back, and you're trying to say, how did I do this? It requires some effort to pull that forward, and that effortfulness facilitates transfer and consolidation in long-term working memory, facilitates retrieval, okay? Now, that seems very counterintuitive, but there have been numerous psychology experiments which demonstrate this. They can be very simple, have, and they don't have to be intellectual, they can be something like an experiment with kids, take and have them toss a ball into a hole from some distance away, okay? One way to practice is, at set distance, do that lots of times until you get good at it. Another way is, take that and make the, toss it a few times at some distance, move closer, move further, back and forth, okay? And you do that sort of training. Then you let some time elapse, so then that means that you rely on long-term working memory, and now you test those two groups, and ask which of those groups are better at tossing the ball into the hole? One's with varied practice, okay? This is shown in a number of different cases. So it's locomotion as well as the- Everything, yes. That's the way, for example, in training in sports, there's this variation. Don't practice just one skill to completion or fatigue. You practice it for some amount of time, and again, it's this sort of effortful, makes it more effortful when we return to retrieve, consolidates it more effectively in long-term memory, and develops retrieval cues that are more effective, okay? There are a variety of other tips like that in this book, which are discussed. What are some tips for teachers? Well, maybe to explain something, so the notion of explaining, okay? But at least to provide an explanation of how learning works, to describe there are these different ways that are somewhat counterintuitive about how learning is effective. Help them give tips to students along the lines we discussed that help them to study. Build in these kinds of active learning experiences, the peer instruction or frequent low-stakes testing to give them that varied testing of differing types of problems, rather than just focusing on just one type. Lots of things to sort of build in. What about resources? Let me point out a couple of resources. For physics, there's vast number of exercises that have been developed along these lines, particularly along the lines of interactive engagement at these websites in more broadly in STEM. There's a website that the National Science Digital Library contains a large number of these. So the punchline is, I mean, I would say the hands-on school really also has this, we didn't stand and talk to you for two weeks, lecture after lecture, right? We talked a little bit and we went into the lab. We worked, you did a lot of hands-on activities. So this is effective in research. It's an apprenticeship model. Science as in the research, doing research is passed on and it's this mixed varied practice. And it's sort of effective because it's effective and there's an underlying reason it's effective. It basically aligns best with how people learn. So try a hands-on approach to teaching as well. With that, I'll stop. Thank you for your attention.