 So, this school has been about research, right, hands-on research, and we're all about doing research because that's a very important part of our professional lives. But for many of us, the vast majority of us, right, we also wear another hat. We're educators, right, we're teachers. We certainly have been part of the educational system as being educated students, but also if you're even at a graduate student level, you're teaching perhaps, teaching laboratories or recitations or you will be doing something like that. So it's important for us to talk a little bit about education. So the first thing I want to talk about is to say something about what our experiences have been. And probably the first thing you think about if you're reflecting about on your own scientific education has been the way you learn science. One major way you learned it was doing what you're doing right now, sitting in a classroom listening to a lecture, right? So in fact, and not only that, perhaps if you are not just sitting there as a student, but also if you're engaged as an instructor, you stand where I'm standing and you've engaged in something like presenting a lecture. So countless times I can tell you that I've, you know, gone to the board and say started talking about, to a classroom of students about some topic like Newton's second law, right? So I'll write it down and say f net equals m times a or if we want to write it in a sort of relativistically correct way, we can write it as the time rate of change of the momentum. And I'll say to my students, well, we can even rewrite this a little bit and say that this change in the momentum we can think about is equal to in related form of quantity called the impulse, right? And what it says is that if I have some object that has some mass, some constant mass, and I can rewrite this as m times dv, then what this, I can still rewrite this and say that if I have some object that has some mass, m, and in fact, if moreover I say that this force is push, net push on it is constant and constant, then the, fading in and out, say something about, right, then however I apply that push or pull, right, the object will undergo the same change in velocity, right? Okay, and I could go on and on for 50 minutes or however long, right? And I could, if I'm a good instructor, I would stop at this point and ask, are there any questions, right? No questions, so I'm very satisfied. I have communicated this important idea in physics and I go on about my business because of course everyone in the class has learned this very, very thoroughly. Well, this is not a lecture about science, it's about the way we communicate science and in fact, in talking about the lecture, the traditional lecture, which is the dominant mode which I would venture all of us have learned science, by which we have learned science, we can ask how effective is that method of teaching science? So I'll tell you a story. So Carl Weiman, who is a Nobel Prize winner in physics, he received the Nobel Prize for the experimental realization of Bose-Einstein condensates, is no longer an atomic physicist. He is a full-time physics education research researcher and he is now currently at Stanford. So before he moved to Stanford, he was teaching a class, introductory class, talking about a different topic. He was talking about acoustics and in particular what he, in the course of his lecture, and he gives very, very clear lectures on the course of the lecture, he talked about this, the notion of how the sound from a violin, where that sound comes from. And he stated in during the course of his lecture that in fact, the sound from the violin while it's driven, the violin is driven by the vibration of the string, what one actually hears is this sort of resonant response of the violin itself, the wooden part of the violin, and predominantly the vibration of the back of the violin. So he said this very clearly and he moved on with his lecture. But he did an interesting thing. In about 15 minutes after he asked this question, or he stated this in a way which I just stated it for you. He asked the students the following question. The sound you hear from a violin is produced, all right, mostly by the strings, mostly by the wooden back, both equally, none of the above, okay? So the question is, I'm not gonna ask you this question, I'm gonna ask you this question. What percentage of the class answered the question correctly, okay? Now, each of you has a little device, which is a means by which you can respond to this, okay? And actually if you're standing, you can grab one, there's some of the seats here. And what you do is you can fold this in quarters, right? And you fold it in a way which produces the answer to the question that you want to display. Now, at this point, what I'm gonna ask you to do is not talk with anybody, you just have this in your own mind, right? The way we're gonna do this is the way we're gonna do this exercise a little bit later. When I count to three, I want you to, you know, think a little bit first, but take and fold the paper up such that when I reach three, you will all at the same time raise the card and display your answer in front of you, right? So I can see, okay? And so what I'm gonna do is I'm going to do a kind of poll of the class that way, okay? So the four responses are, if you think 90% of the class answered the question correctly, which is that the sound you hear from a violin is mainly produced by the wood, vibration in the back of the violin, you choose A. If you think 60% of the class said, got that answer correct, then choose B. If you think 30% got the answer correct, choose C. And if you think 10% got that answer, answered that question correctly, choose D. He stated this very explicitly in class and then 15 minutes later asked the question, this question about where the sound of a violin comes from, 15 minutes after having told the class explicitly the answer to that question, okay? So, on the count of three, everyone raise their card to their chest or so, one, two, three, okay? So what I see, I'm just gonna give you a poll, the dominant answer is C. I would say there are probably the next between D's and B's, right? We have optimist back there is an A, few optimists. But mostly I would say half the class is choosing C. Here's the result. Answer mostly wood by wood in back 10%, okay? Now you might think, well, these were freshmen, right? They don't know very much or they weren't paying attention. So he's repeated this experiment. But in a different audience, with a different type of question. So the setting of the question has to be a question whose answer is not intuitive, right? It's a little counter-intuitive because you think, well, you see the thing, the vibration of a violin, you see the string vibrating. Well, that's where the sound comes from. So if you ask a similar type of question, which you can state the answer explicitly, and yet it's a little bit non-intuitive to a different venue. Let's say an audience of graduate students and professors at various colloquia, which he's given at different universities. What percentage, 15 minutes later, state the fact, 15 minutes later, ask the question, what percentage of the audience do you think got the answer correct? 10%. So this is a very robust feature that even if you state something in lecture, something that is not at the level of obvious and is that everyone whose their intuition would immediately come to, but something that could be stated explicitly but strikes one, new learners as somewhat counter-intuitive, a large fraction of the audience does not understand or learn or remember that even just a short time later. So this points out actually a shortcoming that is not really often realized by lecturers. That when you are lecturing, and for you, the explanation as a lecturer, seems perfectly clear, there were no questions in the audience. So it must be that most people understood what you say. That's in fact a major fallacy, that in fact it's often the case that during just standard lecture, where this audience does nothing other than just sit and listen, that the audience is very passive, a large fraction of the audience, a large amount of what's communicated in the lecture is not understood or remembered. Now, there are ways to improve that. Here's one way. Here's one way to improve the retention. And this is by a technique which is called peer instruction. It falls under a very general category of what's called interactive engagement. How many have heard of peer instruction? Raise your hand. OK, we have a few people. So let me give you a little example of how this process works. First step, you give a mini lecture. We gave the mini lecture. Gave a mini lecture on Newton's second law. In particular, talked about this form of Newton's second law in the impulse momentum theorem. Change in momentum is equal to the impulse, the product of the net force times the change in time. So you present the material like you would in a lecture, but you don't keep going on for the entire lecture. You stop. The next step is the following. You pose a question. So here's another question. We're going to use our cards again in a moment. But the question's the following. So I have two disks. So you can imagine those are on a table. So look at a table from above. The two disks are identical in mass. What I'm going to do is, in this case, have these two disks start out at rest. They're on this table. The table is very smooth. It's frictionless. Starting at some instant, apply a force. So the red arrow represents a force that you're applying to those disks, each disk. It's the same size force, the same direction in both case. It's applied a little bit differently. So the top guy, there's a string wrapped around the edge of this disk. And so the string then is attached to, let's say, someone who's pulling on this with some force, which is represented by that arrow. The direction and the strength, the magnitude. The second case, second disk, identical disk, identical pole, same direction, same size. But the force is attached to a string, which is at the center of the disk. Start at t equals 0. Both are at rest, and then start pulling. There's a race. And that's the question after some amount of time. Three seconds, let's say. Which of these disks is winning the race? A, the top guy. B, the bottom guy. C, both travel the same distance. D, can't determine need more info. So that's sort of step two of peer instruction. Now, step three, each of you are thinking about this situation. You select an answer, and you vote. But no talking. So here what you're doing is you are coming out with your own answer based on what we discussed in the lecture that immediately proceeded and come up with an answer. So I'll give you a minute or so to think about that. And I'll flash back to the question. OK, so I'm going to cut this off a little more quickly than I would in my typical introductory class. And now what I'm going to ask you to do is vote. Same way we voted before. Don't talk to anyone next to you. You're going to on the count of three, take your card. You're going to raise it in front of you. And I'm going to then survey to see where we are in the class for this on three. One, two, three. Very good. All right, now put your cards down. Now, the next step is why this is called peer instruction. Turn to your neighbor and compare your answers, and try to convince your neighbor, if the person here or she is different, that you're right. So go ahead. Start talking to your neighbor. Yes, yes, yes. He was Jennifer's here? OK, so I'm going to cut off the conversation for now. But let me just say, this is very typical when you do this in the classroom. People are talking or saying, no, I'm right, no, I'm right. And this sort of thing, you're arguing back and forth. So now, what happens next? Next step, after you've discussed this with your neighbor, the next step is to vote again. So I'm going to then tell you, give you the sense of whether there's been any change before or after. So same thing, fold up your card. If you have the same answer before, put the same answer up. If you changed your mind, your neighbor changed your mind for you, let's say, convinced you, feel free to change your answer. So again, we're going to do this on the count of three. One, two, three. OK, good. All right. So let me tell you a little bit about what I saw. So I'll go flash back. And this is pretty typical. I would judge from the first initial response that the two popular answers were B and C. Roughly an equal proportion. There were fewer for A and almost none for D. After the discussion, what I saw was fewer B's and more C's. C is the correct answer. Let me explain a little bit about that, just to debrief and say why that's the case. So here's the last step, which is you look to see about student difficulties. We really visit the topic. We would spend more time. In this case, I see there's some, I might lecture, there are still some people who are unconvinced. So I would say a little bit more about this. So the key thing is to look at this impulse, Newton's second law in this form. So we're looking at this change in the velocity. And in particular, these do this start at rest. So you're looking at what the speed is. The speed at some, let's call it the start at t equals 0. So this would be at some t, starting with t initial is 0. This would be some speed as a function of time. If they're the same mass, the disk would experience the same force, the same net force. It was frictionless. The only force was that force represented by the red arrow. It was in the same direction, same length, same magnitude, same net force, same mass for the same amount of time. That must mean that any instant in time they have the same speed. Well, if you start at rest, they both have the same speed at any instant in time. They both must travel the same distance. That doesn't mean they have the same motion. As you could clearly see, there is one has this additional rotational character. That's because the energetics are different. But we're talking about Newton's second law, the dynamics. And they must both travel the same distance. So that's the process. The key thing here is, let me say a little bit about why one goes through this sort of, let's say, extra procedures. What happens? First of all, you slow down. What does slowing down do? It reduces, we'll call, the cognitive load. So it turns out, this is a fact of human learning, ballot biology, that our short-term working memory, that is, when you're listening to someone or you are learning something new, what you are first in the process of learning is you are taking that information into what is called the short-term working memory. That has limited capacity. You can only take, your data rate is only so large. And many experiments that have demonstrated this in human learning. So what you do is you focus on fewer topics. The second thing you do is you make it active. And this is another aspect of human learning. The more you engage with the topic, converse with your neighbor, the better you're able to learn. So this idea of, when you're an instructor, and again, many of us think, when we're instructing, the key thing is, I've got to know that physics right, have to explain it clearly. In fact, teaching and learning is not just about the content. That's important. But there is this other very interdisciplinary aspect of the how people learn, cognitive science, which comes into play, which most instructors of any sort, any science and engineering, really of any academic field, often fail to appreciate. So what we're doing here and what these sorts of techniques do is they take results from cognitive science and apply them in a particular discipline in the process of teaching and learning. And that's something that in teaching and learning in the United States and many countries, this is becoming more widely appreciated. There are greater efforts, greater and greater efforts to be more explicit about not just the content part, but also including, at the same time, weaving together the important aspects of how people learn to make these sorts of ways of learning more effective. Now, let me show you some data, okay? This is from Eric Mazur at Harvard, who introduced this technique. And this again, this is from the 90s, basically, this sort of when this first came out, measured by some a test, it's called the force concept inventory. So it focuses on mechanics, Newton's laws, and measuring basically what's called the gain, they gave the test beginning of the term, gave the test at the end of the term, same test and ask the question on average, how much better did the students learn at the end versus the beginning? If they started out with some score and then they learned everything perfectly, that means they would have a gain of one, okay? In a typical lecture, what you see is there is basically something like a 20 to 30% gain over the baseline, okay? And this corrects for the fact that you have students of differing abilities. So you normalize the fact that you are looking at from where they start and how much they have to go to learn to get perfect learning, okay? And this sort of result independent of the sort of incoming skill of the learner, what you see, this idea of the traditional lecture, typically something like a 30% gain. With this sort of active learning peer instruction, you can improve that to something like 50 to 60% or more. And this is in a particular subject in introductory mechanics, but this is a result that is pretty robust with respect to active learning in physics. This is more data, it's a little difficult, not doesn't follow Tufti's, you know, optimal way of presenting the data, but I'll explain to you, this is basically a summary of hundreds of different, or tens at least, of different classroom environments at the high school and college level, traditional versus interactive engagement of some type. And what you see is again, there's this gain, this is perfect gain up here, this is 80%, 100% would be here. Again, what you see is typically in a traditional lecture, pre, the gain you see is in that sort of 20% range, but with this active learning, it's clearly typically generally larger. So this is why in fact we see in classrooms in the United States, this effort to engage or use interactive engagement techniques is very common. There are classrooms which are, instead of using cards, they're outfitted with wireless sensors or actually smart phones, students vote with their smart phones, but this sort of process of stopping the lecture, posing a question, having students interact, this is a very common thing, and it has data that shows that this improves learning. Let me stop at this point and see if there are anything, any questions or any comments about this? Yes, you're welcome. So, yeah, right, so you're asking the question if you actually looked at individual learners, what you're saying is that there may be many or perhaps a majority of people who will definitely gain, but you may see a substantial number of people who actually regress, right? The answer is that that doesn't happen, right? If you actually look at individual learners, there's a distribution of gains, they all gain, and basically they shift, okay? So that's the sort of general result. Yeah, okay. Yes. So the data that we have that's most extensive are for typically in the undergraduate level for mainly introductory courses and not to say that everyone gains, but the vast majority of people perform better. So are you saying when you're engaged in this, you actually at the end of the semester do worse than at the beginning? I see, well, this is that, yeah. So I'm not, yeah, yeah, but, right. So what I say, okay, what I want to say though is that in these sorts of measures, it isn't the case that you see, if you look at the distribution, there is a substantial tail that falls below zero. Typically, the shift, there's a distribution that all learners, even at the sort of lower end of the average, will be shifted away from zero in the positive direction. So that just to say that's the sort of typical result. Yes, Bruce? Yes, Bala. So lots of things do happen. So for example, the question, you can ask a question that is too obvious, right? Or on the other hand, it can't be too hard, right? And sometimes when you ask the question, the students as a whole, for example, the sort of situation where you don't see much of a change is where you have a large fraction of people who really don't understand. So asking a very difficult question, a question for which they're not prepared for, you won't see much change. In which case then, you learn from that though. So you've given the lecture, but now you're gathering feedback on where your students lie. So there are a couple aspects to it. It's not only the peer instruction, where students, so the idea behind peer instruction is that students can often explain the concept better than the instructor because they're newer at coming to an understanding of that concept. Whereas an instructor has learned that, has become what we call an expert, and has forgotten how to explain it in a way which new learners can understand. So that's the idea behind the peer instruction. But at the same time, in stopping the lecture and asking these sorts of questions, you're also gathering data as an instructor to determine whether you have to spend more time on this topic, whether you can move on something along those lines. Mesh, just a second. So what is the evidence that that works? What is it? So the issue is, this is the, so what one has to do, it's okay, let me say something and we'll step back. Obviously people are successful at learning, right? Almost independently of what you do, right? So the fact that you have people who are successful, I mean that says that they certainly sort of maybe in spite of the circumstances rather than in support, that the sort of learning environment was supportive of that, but maybe more that they learned in spite of the learning environment. The issue is, it's first of all, there's a distribution of people that have different people, have different ways, this is a technique that is better for some than others, but I would say push back to you, you can't look at and say, well, obviously in the, let's say you're Russian schools, you have very successful people who've learned a lot, right? But you haven't asked the question, how representative is that of that group that passed through that system? Okay, so this is what I would push back at and from my reading that most, so I would say the comparisons that have been done are compare what is done, what is learned in some traditional setting, traditional lecture being that setting and then try another technique like these interactive engagements and do a differential measurement, right? And in those cases, I'll finish, let me finish, in those sorts of cases what you see that some of these techniques are an improvement over what is the status quo. I'm not aware of anything of the Russian school, yes, yes. That's what this is actually, so let me explain the measurement. This is the typical way in which you do this assessment is day one, students step into the course, you provide them with this set of questions, okay? Typically multiple choice and those sorts of questions are, questionnaires are called concept inventory, so they're typically multiple choice, typically more conceptual in nature. At the end of the term, okay, end of the semester, you give them the same question, right? The same questionnaire and then what you do is you compare the difference in that case, okay? So, and the question you might ask, well, if you give them the same thing, you know, does that mean that you're just there, they've seen it before? So, there have been many studies on that effect and it turns out that the fact that they see the same set of questions at the end of the semester has no effect on the effectiveness, the validity and reliability of that sort of measurement. So, but it is longer term. Now, you could ask what happens still longer and there have been studies and there is, as well known in cognitive science, something called the power law of forgetting. So, when you leave a course, you do, and you're not actively practicing, you do start forgetting and it's basically has a kind of power law behavior. So, it will, you will have decay afterwards. So, you're sort of measuring the best case situation here unless the learner continues to engage with that material actively over a longer term. Gordon, cultural. So, this work which started in physics in the 90s has been reproduced in many other science and engineering disciplines and essentially you see the same kinds of results. The control is, so even, so it depends on the level of control but there have been studies which even control for instructor effect. So, for example, you have instructor will teach course traditional and then we'll teach next same semester on student populations where they're incoming, let's say, controlling for GPA and things like that. Again, you get the same kind of measurement but it turns out instructor effects are often, particularly in large courses, very small. In fact, it really doesn't matter to a large extent in a very large lecture course. So, but those have been tested even controlling for instructors. Okay, good, good questions. Pietro. And then, yeah. Yeah. Yeah. And then it already has a 5%. Yes. And then there's a. Ceiling effect. Yes. Yes. So that's what this gain that we're doing here is a normalized gain. So what you do is when they come in, if they're Harvard students or if they're students at, let's say, a local community college, you give them the test to begin with. That sets a baseline. Now the gain, the largest potential gain would be the difference between a perfect score and that incoming score. That's the denominator. Then the numerator is, look at their score coming in, going out, take the difference, take the ratio. And it turns out that, unless you have a really very narrow range, but for practical purposes, Harvard students and, let's say, community college students, you can put the data on the same plot and actually see that essentially gains this notion of the normalized gain in traditional, somewhere around here, an interactive engagement, clearly above. So you can do some controls like that. Other questions, Nirmal? So the idea would be you structure your lectures so that depending upon the results of that inquiry, you then, yes, you would decide either to, if there was a substantial uptake, most people are on board with the concept, you basically won't spend additional lecture time on that, you move on to a new topic. You, as an instructor, though, you see, oh, they're still struggling with this, then you change your plan to focus more on providing additional examples, additional questions. So that's how you sort of dynamically determine that, depending upon the response in the class. Yes, so for example, in a typical lecture, you might do this, let's say, the cycle would be something like 15 or 20 minutes, you made two or three of these in a typical 50 minute lecture. Yes, so that is, so this is an issue that this sort of technique is, because when you're in a lecture, you typically don't have your students, I mean, that's what out-of-class activities and working on the quantitative problem solving is for. So this is really much more focused on conceptual, but the key idea from the education research literature is that too often, students can do the procedures, but then when asked to understand what the underlying idea is, they are unable to explain that. However, if they have a better conceptual understanding, that helps them actually become better quantitative problem solvers. So that's why in the lecture, the focus is much more on this sort of conceptual. That's sort of the appropriate setting for that. Other questions? Good questions. Okay, so I am running long on this. I just want to say a few things about one more thing about this is sort of an example of some things you can think about doing immediately in your own setting if you're an instructor. There are resources which I'll point you toward which give you some examples of how to make your instructional environment more engaging, more active, and I'll point you to those resources. Let me say something a little bit more, a couple more sort of big picture ideas. There is a whole field of study which is education research, and in particular in physics, there is a subfield known as physics education research. And a number of departments in the United States have people who have PhDs who focus on how one can improve ways of teaching and learning of the subject of physics. So people do, you know, write research proposals, they write papers, they do research in the context of the physics department. So the upshot is that this is not some kind of, you know, opinion, but it really, there is a theory evidence behind this. It's very different from the sort of kind of hard science that we typically think of in physics. Even compared, let's say, I would say biology which is very, very complicated. It's, you know, how people learn. It's very much a very challenging subject to explore, but it does provide some very valuable insights that one can use in the classroom. This notion that teaching isn't simply just explaining. And the thing that we're touching upon here is this idea that as an instructor, you forget that you went through the struggles of learning that material long, long ago, and you were at a very different place. You have an understanding that you've come to that is in some sense very distilled and actually in some ways quite removed from the experience of novice learners. So the more as an instructor, as an expert in the content that you're aware of that, and you try to bring in techniques that help make the material more explicit for new learners, help new learners talk to other new learners because they can help explain the students understand the concepts in ways as an expert that you have a difficult time remembering that you may have known long ago, but now you have forgotten. So this, having that awareness is really an important aspect of incorporating these techniques into teaching. I'm gonna say, I'm just gonna skip actually ahead since we're running away. I've got a huge number of slides here, but I'm just gonna jump to the end and point to some resources and some further reading. So where can you get more information about this? This is, I posted my slides on the Google Drive, but there are in physics a number of different, the technique I've talked to you about is interactive engagement. It's just one of a number, many of which are materials you can find discussions about these different techniques on a website called Compadre. And then in fact, there's a larger body of material beyond physics in what's called the National Science and Digital Library. And here are the websites that lead you to that. If you wanna know what other people are thinking about, this is, I wanna point to a couple of reports that are really well worth reading. So recently MIT did a sort of big examination of how they're teaching and how their education, how education MIT is being done. And they have a number of very interesting ideas of what the future of education looks like. This is really well worth reading. But what I would say one thing they miss that isn't on this report is this notion of learning is not just learning the content which we discussed, but there are other aspects, skills that are really important that make you more effective to enable you to take that content and apply it in your careers, in your everyday lives. We've experienced some of that already here and that is the focus on communication. That's an aspect which is known as interpersonal skills. Those are crucially important for your success along with your understanding of the content. There's also a discussion of intrapersonal skills. And those are the kinds of skills you have internally and that would be an example of which is perseverance. Another example of a concept known as metacognition that is thinking about your thinking, taking a step back and reflecting on where your limitations of your thinking lie and using those reflections to improve your understanding. So those sorts of additional skills inter-intrapersonal skills as well as the sorts of things we think about, the cognitive problem solving and all woven together with our individual, the studies of a particular discipline or content. So this report also on the drive talks about the importance of those and it's worth reading. And finally I will mention one other thing is a book which I find very interesting and it talks about the importance of culture. And just say one punchline about this that oftentimes when we are engaged in business or in students in the classroom we have one job which is for students to do the learning but there's this other job which goes unvoiced but is very important and that is students or if you're in a career in business trying to cover up your mistakes or trying to manage your perceptions of yourselves or students aspect, the worry not just about the learning but also about the grades, right? In some sense that often, that second job if you will oftentimes overwhelms the first job. This book talks about how that sort of division is really inherently a problem of culture and suggests again based on science a different area of psychology, adult developmental psychology in ways in which that sort of divide can be closed. And so I recommend that. Unfortunately it's not on the drive but it's a book recently put out by two Harvard psychologists, well worth reading and I'll just leave you with a final message that being active, engaging and having your students be active in research as well as in teaching is an important thing and so I encourage you to try a hands-on approach to teaching as well. So with that I'll stop. Thank you for your attention.