 Okay, so the subject of this school is research. It's the hands-on, you know, school research, school and complex systems. Even the professional development work we've been doing has been focused on how to communicate your research more effectively in different formats. But many of us wear a different hat, another hat at the same time. And that is we're educators. We have students in the classroom. And so what I want to tell you a little bit about is there is some information, even as people who are primarily doing research, there's advances in understanding of how to teach science. It applies to engineering too. We have engineers, basically anything that's science, technology, mathematics, engineering, so-called STEM. There's research in more effective ways in how to teach these sorts of subjects. So I want to tell you a little bit about that. Three aspects first. How do you teach science more effectively? So I'll just give you one example. It starts from some experience that we've all had as students and that starts from the traditional lecture as the baseline. How do you improve upon that? I want to say a little bit about what is the science of science teaching and then although many of us are involved with research and maybe exclusively research, let's see, how many people actually do teaching as well? Raise your hand. Okay, about half of us in the room. And others who are researchers maybe as graduate students but you're doing research full time, it may be in your future that you will be teaching. Or there may be skills that you would acquire in the course of teaching that you would apply elsewhere. So I want to say something about how that teaching and a teaching experience really advances your career as a scientist and more generally just beyond the laboratory and the classroom. So everybody's been to a traditional lecture. We've had a number of these. Although I would say many of the lecturers have made the, broken up the lecture where someone stands up in front and talks and everyone sits and listens. The lecturers have been at the school have done a good job of breaking that up and getting you more involved. But I want to say a little bit about if you take the standard lecture and you're sitting in your classroom for 50 minutes or 70 minutes, you might ask the question, how effective is that as a medium by which you communicate information? So here's a story in a misspelling with a K. One teacher and Nobel Prize winner, this is Carl Wyman who is Nobel Prize winner in physics, won Nobel Prize for the discovery, experimental discovery of Bose-Einstein condensation. He was a lecturer and taught classes and he had this class in general physics and he was talking about subject of acoustics and gave as an example the example of a violin and said during the course of the lecture that explicitly told his students that when a violin string is plucked or bowed, it vibrates, but that isn't what you hear directly. What you hear directly in fact is the vibration of that string is coupled to a bridge, this wooden piece on the top, and then underneath that top piece of wood, the face, there's a post called a sound post which causes the back to vibrate. So in fact that's what couples directly, acoustically, creating the sound waves that you hear. So he's known to give very clear lectures, describe this very clearly. What he did was ask 15 minutes later, same class he asked the following question to his students, the sound you hear from a violin is produced, A, mostly by the strings, B, mostly by the wood in the back, C, both equally, or D, none of the above. 15 minutes later. So I'm going to ask you a question, not this question, I'm going to ask you this question, what percentage of that class do you think answered that question correctly? So we have five choices. Now each of you has, we're going to use this little device more than once. This is a way in which one can rapidly poll your students, if you have a large number of students, several tens or hundreds, this is a way in which as the lecturer you can get some feedback on the question that you posed. So let me tell you a little bit of how this operates, so you can fold this, and then you fold it again. So you produce basically the four answers, A, B, C, D, and actually in this case we have five. The fifth answer is this. So just show this. Now I'm going to ask that everyone, let's say, on the count of three, and this is also practice too, we'll all respond at the same time. We're going to do it in a way so that the rest of the people in the class won't be a little bit embarrassed about our answer, but we're going to preserve our anonymity. We're going to answer by, when I say one, two, three respond, you'll hold the card up next to your chest like this. So people won't be able to see how you respond. You can preserve your privacy that way. So again, the question is what percentage of the class answered the question correctly? Again, let me emphasize, 15 minutes earlier he said what the answer to the question was. Answer A, 90% of the class, B, 60, C, 30, D, 10, or E, none of the class answered the question correctly. So you had some time to think about it. On the count of three, everyone will hold up their answer card. One, two, three. So I would say the dominant answer in this class I'm just polling C. C is the dominant answer. I would say the next most is B. We have a pessimist back here who says zero. But I would say C and B. Some place between 30 and 60%. Here's the answer. 10%. Now you might say maybe his class was a little 8 o'clock in the morning, maybe they weren't awake. He's repeated this experiment over and over again to different audiences. So he's given many colloquia at universities and he's given a talk. He's posed a question and then said some statement about his research and then 15 minutes later asked the question to the audience. Something that he had said 15 minutes earlier. This is the audience with graduate students with professors, physics professors. What do you think the percentage of the audience answered the question correctly? A question that is not obvious deals with some subject that isn't really obvious but if someone tells you it explicitly the question is how much does the audience upon hearing that somewhat counterintuitive explanation. What percentage of the audience retains that? The answer is 10%. So this is a robust finding. It had nothing to do with the particulars of that class. That's exactly right because it has to be something that's not clearly, let's say, intuitive. The issue was that there was a discussion of, in the example of the acoustics for the violin, there was a discussion of why the sounding board in the back because and where, the mechanism just as I gave. So still I think somewhat surprising that it's true. So this is an issue. So the question is if you were thinking that when you explain something very clearly to the audience and you've discussed the chain of reasoning and you maybe let's take the worst case, 90% of your class doesn't believe you and that's what you do day after day, then maybe there's something that you need to be doing to try to make your lectures more effective. So that's really the point of this. So the upshot of the, so the reasoning behind why just passive lecture is in many cases can be so ineffective is that it is passive. That is when you're sitting there, you're listening, but maybe you're listening, maybe you didn't hear what the person said, maybe you're texting, the variety of different things that can be going on with an audience while the lecturer is basically communicating what the lecturer thinks is a very clear explanation. A way to improve the audience understanding, the student understanding is to get students more actively engaged. It is doing something active rather than just sitting there passively. And there are a wide variety of techniques for doing this in the classroom. One of these is known as peer instructions, very widely known in many, widely used in the classrooms in the United States not just for physics, but many types of different classes. How many have heard of peer instruction? Just curiosity, okay? So let me just sort of give you the idea. It can be done in a large lecture and we're going to go through an actual example which will be an actual question which is rather counterintuitive. So you can see how this procedure goes and how you take the traditional lecture and you make this change which I'll show you in a little bit with data. It helps improve student understanding. So first step is you give a lecture like you would. So you're presenting information, but you break it up into segments. So when you think about it as being, let's say, in a 50 minute period, you have what we'll call a mini lecture. Okay? So our mini lecture for today is Fluid's Question from Fluid's Statics that one sees in many introductory physics courses and it's on buoyancy. So called Archimedes Principle. And so I'll give that lecture in less than 10 minutes and the point is, the idea is, is that when you have an object and it's immersed in a fluid, then the corresponding force, the upward force which we call buoyancy, is equal to the weight of the fluid that was, the volume of the fluid displaced. So here I have my object immersed. It displaced this volume of fluid and the weight of that amount of fluid is equivalent to the force, the upward force, the buoyant force on that object. Okay? And you could talk about it in more complex ways and talk about the pressure field and integrating around that. But that's sort of the classical statement of Archimedes Principle. Now if you have something floating, then basically the upward buoyant force, the buoyant force is the amount, again, the amount of fluids that's displaced, in this case the floating object doesn't displace its entire volume, but basically the volume below the waterline, if you will. Okay? So that's the basic picture and basic idea. Any questions about that? Okay? So I've given the mini lecture now, step two. So rather than keep going and keep talking to you about more ideas, the next step is to stop and pose a question. So here's our question. Sunk in treasure. So we've got a pirate ship that's floating on the sea and it's carrying a treasure chest. The treasure chest is dropped overboard and it sinks to the bottom of the should be ocean or sea. Okay? Now what happens to the sea level? Does it rise? Does it fall? Or does it remain the same? Okay? So again, think about this. This is floating, treasure chest on here, pirate takes it, flips it, drops it overboard, it drops, it sinks on the bottom, sinks to the bottom. What happens to the level of the sea? Does it rise? Does it fall? Does it remain the same? Okay? So you pose the question. The next step you give each person a chance to think about it. Okay? So each of you think about it, but you don't talk to your neighbor yet. You're going to think about it. But just like we did before, we're going to vote in a moment in the same way, again, preserving your privacy. I'll give you a little bit of time to think about it. Good audience, good student body, because most times students try to talk, then I have to shush them. Okay, so let me flip back to the question so you see what the answers are, the choices. And I'm going to get a hand to record sort of roughly what I see the numbers are, but I won't reveal them just yet. All right, on the count of three. Sorry? No, I'm not going to show them. I want to keep them here. Okay, on the count of three, let me see your response. One, two, three. Quite a mixture here. Okay, let's see. Okay, so if you just hold over a minute, let's see. The advantage of doing this is I can see pretty quickly. I can already see roughly the numbers, and that's why the colors allows you as an instructor to see is most of the class on board or do they measure? It's a good way to measure. I am colorblind, actually. These colors are saturated enough for me to be able to tell. Okay, so the next step is after that this is where the method gets its name, peer instruction. Turn to your neighbor, compare your answers, try to convince each other that you're right. So let's do that now, and you can talk. Chat with your neighbors. Sorry? Yes, you need to. Yes, that's right. No one's checking, no one's texting, no one's falling asleep. Okay, let's pull it back together now. Okay, so now the next step in this is after we've done this is to vote again. Okay? So now you've had a chance to chat with each other, and I'll say a little bit about what one typically finds in this, but I'll have to see whether you follow sort of the typical behavior of the audience of this question. Okay, so we're going to vote again. We're going to vote again. They're still discussing. Okay, are we ready to vote? Okay, you guys are great students. Just can't stop talking about it. This sort of level of discussion, this is what typically happens when you pose questions. I didn't see anybody texting. I didn't see anybody checking their email. Everyone was talking. So this shows you that part of why this technique is at least getting people engaged in the question at hand. Okay, so we're going to vote again. So same protocol on the count of three. Hold up the card of your favorite answer whether the C level rises, falls, or remains the same when the treasure chest is dropped overboard and sinks to the bottom. On three. One, two, three. Okay. So I'll tell you what I see. I'm going to do a rough count, but basically what I see is typical of what does happen. What is the correct answer? The answer is the water falls. Do you believe that? No, let's don't believe it just because I say it. Let's couple that with something else. Let's couple it with a demonstration. All right, so let me show you. We have here is our pirate ship floating on the sea. It is there and there. This is I dip this out of the Adriatic, nice and green. And so we have our floating ship. So it's floating. And I've got a roll of coins inside there. There's a treasure chest. Now notice the level of the water. The sea. The sea is right up the brim there. So what I'm going to do is I'm going to take the treasure out. Drop it into the sea and watch compare relative to that level whether it rises, falls, or remains the same. So I'm pulling it out now and dropping it back in. So it falls. So what's the idea behind that? Let me just say a little bit about that. So at this stage what you would do is you would recap. So let me just say what the results of the voting. When you first voted it was about an even split between rises, falls, and stays the same. I would say that afterwards that the falls answer did increase but there was still a fair number of rises and stays the same. But there was some progression in the direction of the correct answer. Now that doesn't always happen but that's in fact what happened in this case. And depending upon what happens in that case, let me show you just say something about that. Flip back to the slides that basically depending upon what the reaction is you'll change your response. So if students have had difficulty with that you continue with the discussion. If people are mostly correct after having gone through this process you move on. But that keeps the instructor keeps in touch with where the students are and so that gears the lecture much more to where the students happen to be in their understanding. So this is another important aspect. Let me talk about the recap of this. One way to think about it is why does the treasure, the treasure is not sinking here, it's sinking here. There's a difference in the buoyant force. There's a larger buoyant force in this case than here. It's equilibrium here so the weight of the treasure which remains the same balanced by the buoyant force alone here. When it's dropped to the bottom it sinks and when it remains at the bottom because now the buoyant force is less it needs an additional contact force from the ocean floor. The buoyant force is smaller here than here so that means the weight of this place fluid is smaller here than here meaning you displace less fluid in this case. So you displace less fluid in the sea, the sea level drops. So that's the answer to that. Let me show you why there's something about some data on, does this have an impact? Let me indicate or show what the plots here are. This is a comparison of Stanford trying this technique out in a number of different classes well actually different level of techniques. So this is a standard lecture. They would ask a set of questions and then beginning so they would do what's called a pretest about questions in mechanics, force in motion. They would give the same test at the end of the term and they would ask the question of the total fraction that the students could have improved did they actually improved, did they improve on their understanding of mechanics. So there was a baseline score, they took the test again, on average there was improvement by the end of the term, that's what we would hope after a semester is worth of that class and possible gain that they could have had. After they instituted this sort of interactive engagement peer instruction they saw basically a double of that sort of gain and in addition doing something else, this is not addition, there are two techniques being applied here the peer instruction as well as a way of having students prepare for class in advance called just in time teaching. There's similar sort of data if you couple the peer instruction with lecture demonstration so called interactive lectures that also helps improve still more the understanding of students. So this is an example of data. Let me show you one more thing and then Joe I'll answer your question. This plot busy, it's not up to the standard but it's a very famous plot. It basically talks about again the same type of question. They have some sort of score initially and the question is how much of the total amount of their possible gain do they actually gain. Now the two classes of data here, one if they just have traditional lectures, there's some type of interactive engagement. Something to make the lecture more active be the symbol. So on average if you do something that makes students in your class more active in participating in class you get improvements in their learning understanding. So that's a very general result. Question? Yes. Why does it involve possibly because they're getting more effective at the way in which they give this test or maybe there's some transfer of information because they give the same test over and over. Could be some combination of those things but the point is that there is this between here where there's no communication from prior years that suggests that there's and this is the kind of thing that this plot also is more data from hundreds of different classrooms. Tens of thousands of students involved this sort of data which suggests that making the classroom, the lecture more interactive gets students more engaged. So let me say something a little bit, something about, yes Eva and you are retrieving it in an active way that you're actually engaged. So the whole point of learning is that you're engaged with mentally engaged in the material at hand and however you accomplish that you create the job of the teachers to create an environment in which that sort of mental engagement is more effective. And so that's really the underlying sort of science cognitive science. It has not just to do with the content but it has to do with how people learn. And so this is you can connect it to neuroscience that basically there's neurons connecting as you're learning, learning processes forming these appropriate network of neurons that requires engagement and the more you can make the environment, learning environment engaging the more effective learning is. That's really the underlying idea. Education research data has it's not quantum electrodynamics. It has lots of spread. So you have to sort of look at it with a grain of salt but if there are lots of repetitive studies you see similar trends then that gives you confidence just like in any branch of science that the findings are have validity. Professors may be getting better. Yes. So this is a very famous thing that people will say, you know, we all feel this, right? Here's that textbook which gets bigger and bigger every year. They cover the textbook, right? You know, the fact of the matter is if they're not learning it, they're not you can feel like you're covering it but if they're not learning it what's the point? Yes? It's not the amount that you cover, it's the amount that you uncover or something that effect. So these sorts of points are people are still struggling with. They struggle. There's a very important questions very, you know, unresolved I would say how do we make sure that they're learning what we need them to learn in the next stage but the question becomes if they're not learning and this is an issue. This is a common complaint one hears and instructors of advanced classes, didn't you have this course before and didn't you learn this material? So I think the answer lies in you have to find and judge the right level. You have to try to make the environment in which you're teaching more effective and you know and you try to coordinate as much as possible and identify what are the most important things that they need to carry on to the next level because simply because you are talking about the subject doesn't mean the students are learning it. And so let me this actually before I go on I have a couple of slides I want to say about that in a moment but I want to say something this technique I've told you about is one of many such techniques that have been developed through education research. I want you to, so these are really important places to take a look at resources that you can use in physics. So you know I know physics literature best. This is a repository compadre. It's called compadre. Here are the websites for this. You'll have the links that you'll be able to access when you download this. This has in particular this link leads to what's called a physics education research user guide. And it gives something like 60 different techniques. Everyone is explicated about their various levels of development but they have you know what is the research backing for this? How do you apply it? What is the appropriate level of student for which this technique is useful for? It's really very useful and comprehensive. If you're in teaching in others beyond physics in you know engineering there is a super set of this. Compadre is part of what's called the National Science Digital Laboratory and whether you're in engineering or you're in chemistry there's a whole set of the different disciplinary societies, different organizations have connected or put their education research literature developed techniques there. So I really encourage you to explore these too if you want to learn more about what are the possibilities. I've given you one such technique. There are many others. So let me say a little bit about the education research right? So we think about teaching as maybe an art or you know you're a born teacher. Now it's like anything else. You learn it. There is a theory there's evidence, there are experiments connected to theory it's not the level of let's say doing physics but it does give insight into how one can improve the science of teaching and learning. And let me just say a few things. First of all teaching isn't explaining this is kind of the major theme. A second major theme of this type of research is that experts, lecturers think differently than novices and I'll say a bit about that. And there's something that's called cognitive apprenticeship. I don't know if I'll say too much about this but the idea is the way that we learn. So it's actually built on the way we learn to do research. We're doing research under an apprentice, if you're a student you're essentially an apprentice. You work in a small group under your supervisor. How do you learn from your supervisor? You watch what your supervisor does. You go out and do things under your supervisor's supervision. You make mistakes. He helps you correct. But after a while he does less and less of that as you get further advanced in your career. So that assistance fades and you become after some time after this period of apprenticeship you become an expert. And this is kind of the idea that underlies a lot of the education research innovations is to try to make the classroom more like that. Not a classroom where you're just talking and you're passively listening but to have it much more focused to create an environment where students can be mentally engaged and take on and become more expert like over time. As you mentioned I got a lot of these are from Ken Heller who's an education researcher in physics at University of Minnesota. So I want to just acknowledge that. But usually we think about I'm lecturing and the students are they're taking it in. So they're like a funnel. I'm funneling all this information in. Well we're frustrated because only 10% of the students learned it. So as a student there must be dumb or something. But students have a perspective. I don't understand what this guy is saying. He doesn't communicate very clearly. So everybody's frustrated. If you have this sort of picture of a structure speaks, students are supposed to learn and that's the nature because the lecture's giving a very clear explanation. And there's a lot of difficulties with this picture and one of the chief difficulties is that experts organize their knowledge differently than novices. So for example for doing a problem in mechanics. Block down an inclined plane. How do we think about it as physicist? Well we say what are the principles? What are the laws of physics? This is where we start. And what are sort of then with those laws what are the general conditions? Equilibrium, non-equilibrium? How do we set up the description? And then we talk about the particulars. It's a particular setting. What are the objects involved? There's a plane and a block. What are the interactions? That's the hierarchy. Starting from fundamental principles. Students don't organize their knowledge that way. They look at typically surface features. That's an inclined plane problem. There's a plane and so there's an angle in there. And it's only very late when they said okay and then there's some set of equations that I need. So there's the friction equation. So they have a very different way of organizing their knowledge. Now the job of instruction is to try to help guide students from their novice way of thinking to more expert-like thinking. And that's where making the environment more interactive. Again, not telling students but providing an environment where they are guided to engage in exercises which helps them learn to become more expert-like in their thinking. There's no other way. This comes from cognitive science research. And I'll say the picture is the following. The point of the lecture is to show to model the correct behavior, the expert-like behavior. But just like if you're learning to play golf or you're learning to play soccer, you don't become an expert at soccer by watching someone play soccer. But you have to know how the rules of the game are worked. So it is informative. It's useful. But it's the starting point. And then you have to create an environment where the students are actually engaged in doing those activities needed for that particular subject. So it can happen even in the classroom. That's why pausing and having students talk to one another, engage and respond to a question under the supervision of the expert. But it's helping the students engage rather than having the expert just continue to show. So now you're engaging them, coaching them. This happens in the classroom, in recitations, in laboratories. And then eventually fade that support away as they become more and more expert-like. So the degree to which instruction can be modeled in that particular way and redesigned. And naturally I would say the underlying objective of lots of education research is to create this kind of environment. I want to say one more thing about why is it worth spending time a common refrain that I hear from my colleagues in the department as well as some students is that I'm here to do research at this institution. And I'm doing a teaching, maybe doing what's called a teaching assistantship. And it's kind of a waste of time because I'm here to really become a researcher. And I think, and others, particularly under the leadership of folks like Ken Heller, is that this is a lost opportunity. In fact, if we think about the teaching and the time spent teaching as a way to advance your career, there are lots of skills that can be learned in that sort of environment. And so it is sensible to think about ways to try to make that teaching assistantship be something or teaching experience. And I'll think thinking about TAs being an opportunity to practice a set of skills that are helpful in the lab in doing research and beyond in your career. Now in terms of career, I mean it's good preparation for employment. So the idea that, and this is also something that not everyone who is a graduate student becomes a faculty member. In the United States this is certainly not the case of only about 30% or so of the PhDs end up being faculty, but only about half of those in physics end up being faculty at PhD institutions. So that means 85% in the US of PhDs in physics end up being in a position different from their supervisors. So this is a broad issue preparing graduate students to become effective employees. There are lots of skills that you learn as part of teaching experience which can carry over to that. So for example if you're in industry, this is a poll that was from this publication which says what are the sort of skills you need to be effective in industry? Teamwork. So you've got to work with groups. Communication. These are sorts of skills that you can get lots of practice in, in a teaching environment. And government laboratory, so there was a talk by a supervisor, director at NIST, talking about the sort of qualities that it's important for physicists to have at government labs. And some of these are very much the kind that you would involve or acquire while doing research, but also while planning and organizing, teamwork, communicating in the classroom. So what we're doing at Georgia Tech and others at other institutions are engaged in trying to make our TA, what we call TA training, but TA career development much more tightly integrated as part of a career development experience, global career development experience for our students. And so there are various ways in which we're changing the way we're teaching our TAs, but one key thing is we're getting our faculty involved with this. In the past we would rely on basically having someone who is not a faculty member just run the TAs through a Friday before training, before the laboratory and recitation. And now we're moving much more to the model where faculty are taking charge of developing the TA training and because they are providing the model for the graduate students to show that this is important and also then to help them practice techniques that are important for them to be more effective in the classroom, but also to help them practice these real soft skills that are important for their life beyond the classroom, beyond graduate school. So with that I'm going to stop, but I just want to say one summary point is to try these sorts of, there's lots of opportunities and ways to try a hands-on approach to teaching too, so I encourage you to explore it. With that I'll stop, thank you for your attention.