 Well, I'm very excited for this session and I am these guys are at the cutting edge in the art and science of teaching Which is the name of the session? This will be this is being webcast live and video recorded Which is great because then we can refer back to it share it with people So I would like to introduce our Session leaders speakers and then just let them get on get on with it. So First I would like to introduce dr. Steven Pollack from the physics department at the University of Colorado Boulder Steven was named the 2013 US professor of the year by the Carnegie Foundation for the advanced advancement of teaching He studies student learning in large-scale physics classes And the constraints and opportunities involved in replicating proven curricular practices as well as issues of gender gaps in introductory physics. He is known amongst his grad students as the human electron Stephanie dr. Stephanie Shasteen is a physicist and education researcher focusing on STEM educational reform she is the associate director of the science education initiative and Independent consultant at the University of Colorado Boulder Dr. Chastine uses her skills in science science communication and education to support educational Change programs at the University of Colorado and around the country including providing resources and support to faculty producing pedagogical videos assessing programs and Researching characteristics of effective educational practice. So with that take it away. Thanks, Val There's some instructions up front. So grab a clicker. Can everybody hear me? Okay, it's not too loud All right So take one of the clickers You probably don't need to do anything except turn on the power button and then click a through e And you'll know it's working because when you click it, it'll briefly flash green So have you ever taken or taught a class that used clicker is something like this? There's many, you know different technologies, but they're this sort of idea Or voting cards, which is a low-tech version of this where you just hold up a little index card Raise your hand if you need another clicker Yeah, and if you're finding that you're having some techno problem Somebody else at your table might be able to help you figure it out Okay, so we've got about 50 votes here. I'm gonna Sort of move on, but this is this is your voice right now in this room So so it's helpful if you can get this thing working and um, you know When I'm teaching large lecture classes, and I wouldn't call this a large lecture I I'm teaching 300 students this semester. This is sort of a typical, you know, large-state university introductory level class There's all sorts of questions that I can ask that really help me to figure out what's going on So for instance right now We are all over the map. Yeah, so that's good for me to know And not a single person voted E other it's complicated I'd like to give you the option of like I don't fit into a bin I'm a human being right and and often like I'll then say okay Who voted E? Like tell me what you want to say because that's like an opportunity for you to get your voice heard So this is all over the map. I'm not, you know selling products This is a tool like many one of many tools that we'll be telling you about to try to Allow some communication in this room This is a picture of me in the Upper division electro dynamics course that I took a couple years ago, and this is in the middle of lecture So so something different is happening in this room than just me at the blackboard Explaining to students as clearly as I can the rules of Maxwell's equations, right? So so I'm speaking to a few students if you look in the background Some students are working on their own over in the upper left-hand corner Some students are talking to each other that one happens to be smiling. It's always nice bonus There's somebody like standing up in the back That's an undergraduate who took the class the previous semester we call him a learning assistant I pay him he comes back and helps me So that you know, there's all sorts of stuff that we do in this classroom as well as me lecture You'll find you know, I'm lecturing right now. So there's there's certainly a time for that Title of this talk the art and science of teaching so I noticed already when we came in people were saying I'm looking to you forward to your talk on the art of teaching. So so you know that I I'm a physicist and so I want to include the word science in the title In part because otherwise I can't keep my job at the physics department at the University of Colorado You know, I'm gonna make some claims Which is that there is both obviously an art and there is a science to teaching and I think it's useful to be aware Of that let me ask you this How many of you think that the odds are greater than 50% that someday you'll be teaching on a undergraduate class. I Just want to get us out lots of hands. So let me invert that. How many of you are pretty sure that you're never gonna be teaching undergraduate class Nobody is willing to raise their hand for that one. Okay, good. So, you know, then then it's appropriate that we spend a little bit of time here We've already been introduced Stephanie. Do you have any additional introduction that you need to make right now? So I'm gonna be telling you a little bit about ideas from physics education research But you know, I'm aware that like this room is people in a related but separate discipline And so the physics part is not really the key element here You know, you have to acknowledge your colleagues and your funding sources whenever you give a talk Part of the point here is that the font is almost so small that you can't read it There's a lot of people at CU Boulder who are interested in and working on questions of physics education and if you expand this to blank education research like geophysics education research or Chemistry education research or engineering all of a sudden, you know, there's like lots of people at the university We have discipline based education research meetings going on so it's a very active and But I feel like this is just a little bit Loud So When you are looking at an institution, you're trying to get a job and you're thinking about your teaching statement We won't be saying too much about that, but maybe a little bit It is useful to find out whether there is a big community of education research at the place You're interested in some schools do some schools don't and it depends also on the discipline physics Kind of has a long tradition of education research It's more common to find and it's worth it's worth just learning a little bit about that because you don't want to Adversize yourself as being an education researcher if you're not but you'd like to be aware of education research so that you can use it in your teaching Okay, so a little quick lesson here. I'm raise your hand if you know about traxylene You know what traxylene okay? I'm gonna teach you It is very important to learn about traxylene traxylene is a new form of xionter It's mottled in Sarastana The Sarastanians found that they could grister late large amounts of fervon and then bracteric to quasal traxylene my little mini lesson and You know, there's an awful lot of science classes that look like this at least to the students So okay, so I've given you a lesson if this was in fact a class where I felt it was important that you Learn about traxylene. I would surely have a test or some homework problem So I'm gonna give you a test now. Okay, so better read it again Actually, I'm gonna leave it up so that you know, it's like an open book test and Here you go Just think can you answer these questions? What is track? Where is it Montille? How is so how many of you feel that you would get an A or B on this test raise your hand? Really? You can't tell me where it's Montils. Could you please read the paragraph again? You know Raise your hand if you think you'd pass this test. Yeah, I think you would all pass this test I know you could because your postdocs, right? You've passed many tests like this in your life So and and you know so part of the point here is I Don't want for students to think that this is what science is Memorizing nonsense and repeating back words without having any understanding, you know I think these questions would feel to an instructor like they were good questions And I would find out if you know about traxiline Maybe if I asked this one, I said it's important, but why is it important all of a sudden that's a little bit tougher now Actually, if I asked you this question, you'd say oh why it's important. Oh because it's a new form of xionter So, you know you'd be asked your way and you'd hope that the professor would give you an A for that answer If I asked you how could the Sarastanians improve the traxiline production process now you're totally lost, right? So so I would claim that you don't know anything about traxiline I know that because it's nonsense and yet, you know, this all feels like a science class So as we move on, you know, you're welcome to say traxiline, you know, which is sort of a coded Indication that all of a sudden I'm I'm speaking nonsense. So it's Stephanie. Let me let you start an activity Okay. Oh, is that Mike to Mike? Yeah, okay So to sort of compare the modulation of traxiline to what we know can really work in education I want you to get a chance to think about something that has really worked for you in any STEM course that you've taken Think of and STEM is science technology engineering math. So it could be within Geology, Geosciences, but it doesn't have to be think of something that was a really Transformational experience, and if you can't think of a transformational one at least one that was positive And and what about that experience was transformational or important? Why did you value that experience? What were you able to do as a result of that learning to think about how it is That you were different how it was your transformed What it is you are able to add to your life as a result of that that learning don't you to just think about that for? 20 seconds try to get get something in your head So hopefully you'll have some ideas starting to form So I want you to now turn and talk at your at your tables and share what that transformational experience was and why You value that experience so highly go I've tried that and failed Yeah, it's tough to find papers in physics that are accessible to undergrads We keep trying to like you know identify good good readable papers that because Physics doesn't have a good tradition of writing papers that are accessible So we just My mind feels too loud You could ask the AB people, but I thought it was one so are you gonna Take a few minutes to get a couple Take another few seconds to wrap up your current train of thought Turn it off when you're when I'm come back together and hear some of the ideas that came up So We're not gonna have time to hear like all of the great discussions that I hope you just had I heard some really great ones But I want to hear just a couple of things from a couple of tables some of the things that came up What what what was the experience that was valuable and and what made you value that experience so highly any? Anyone want to volunteer? Not that I'll pick a table. No watch for the wait time here So Yeah, do you want to I said at our table? We realized that people had interpreted the question very differently and someone had kind of answered with This was a course that I took and this element I liked and now my research is on that topic or that's how I got into this field or something and I thought you meant what was a Something specific about how a course was structured that was transformed how you think which is kind of very teaching-related, but and Mine was like about teaching me to distill information better through like some exercises So there are different ways that we can be transformed by a teaching experience is maybe the content actually informed our future choices Or maybe something about the way that it was taught changed the way that we think or the way that we approach a problem You know both are I think very valuable since this is a Session on the art and science of teaching. I'm thinking more of like what kinds of teaching techniques You know brought you to a place of new thinking, but they're both valid other other volunteers See in the very back Yeah Thank you to the microphone carriers at our table we kind of stuck within physics Talking about quantum physics and heliophysics, but essentially the common theme is that we thought it was very powerful for professors and lecturers to kind of Teach with examples are relatable to the class and really get at this This larger aspect of kind of teaching the practice of science to a to a larger audience So the teaching the practice of science not just the facts So not just the the modulation of tracks lane stuff, but how you actually do it how you formulate Hypothesis how you think about drawing conclusions is that is that what you mean? Okay, so those sorts of things We might call skills in addition to attitudes We've got time for one one more Or someone at this table before yep So my example was about a geophysical fluid dynamics class where we did rotating tank experiments or demonstrations to show, you know, how weather forms like what causes bariclinic instability and I Found that really powerful because we can we write all the equations down for what you know governs that But we did the tank experiment it was very clear that it was rotation plus temperature gradient and so like being able to like get that Conceptual sort of understanding, you know to complement the rigorous math. I found really powerful and you know It it made it easier than to say to anyone what causes whether you think oh, well rotation of temperature gradient There you go. There's a the focus on conceptual understanding So you really could kind of make predictions or understand how that how that worked This is great. You know it since everyone said that they would be most likely teaching undergraduates Think it's really important to reflect on what experiences you found particularly valuable for yourself and talk to tell other people about what They find valuable because those are the sorts of things that you want to emulate and one thing you might have noticed as reflecting on your experience sometimes Those experiences are rare and they're not how we were usually taught But it's much easier to teach in the way that we were usually taught Which is typically lecture at the blackboard So think about those those experiences that stand out and you know if you're thinking about applying for teaching jobs in the future Those might be the sorts of things you might want to capture for instance on a teaching statement to sort of explain your vision or philosophy of teaching so I Just wanted to say a word about education research because because one of the things I want to encourage you to do is after You know after this meeting and while you're writing your applications to think a little bit about What your disciplines education research looks like within physics so the first line is really critical This is a research studies done by physicists So it is not for instance the research literature that comes out of the educational psychology departments or the school of education That's super valuable and we use that but I think in order to talk to physicists about teaching physics It's it's been Sociologically really important and helpful that it's coming out of physics departments And I'm you know I was sort of trying to distill what PR looks like I think one element is How do students and I'm thinking how do adults learn science so mechanism and you know I sort of point at my head because that's where I kind of Think that learning happens But there's a lot of people in the in the education research community who would not point at your own head But think about the fact that learning is a social enterprise and in fact learning physics is in part Socializing you into the world of physics and therefore the learning mechanisms have to do with the interactions You're having with other people and not just you know rewiring your own brain. So all of that goes in here The next is how do we know they are learning? This is hugely important So you start to when you're thinking about this you start to articulate What it is that you would like to be able to assess and then how could we assess it? So I think some of those questions those exam questions about traxilin that I showed you oh Are not very good measures of the kind of learning that I would like my students to be able to demonstrate And it's highly non-trivial. There's been a lot of work done in trying to think about how do you know? That they're learning. It's not always tests. It's observations and conversations and there's all sorts of ways The third is the one that probably everybody in this room is like gimme right how How do we help them learn because you know you want to go and be a good teacher right from the start or convince? Somebody that you're going to be a good teacher if you're still trying to get jobs when I started in this area of Research actually before I started in this area of research I did theoretical nuclear physics and I taught because I was an assistant professor and So I would listen to P. E. R Talks and I read some of their papers and I was just interested in the bottom line and what I took me a While to appreciate is that you can't help students learn Until you start thinking about how do adults learn and how would we know that they're learning? So I think these things all tie together So you know we're not up here kind of trying to give you teaching tips in part You know there's lots of individuality about the art side of teaching that I wouldn't want to try to Suggest you teach the way I do but to start to think about these questions And you know I'm in a physics department. So we all get labeled. Are you a theorist or an experimentalist or do you do applied physics? And so you know that that's sort of how I helped come up in my own mind with these categories and Because it's a young field you can do all three You know if you're normally in the physics department you you are one or the other theorists experimentalist applied Okay, let's do another clicker question. So this is a question from the force concept inventory It's a research-based assessment instrument developed 25 years ago by physicists and you're looking down So there's the head of a person and they're swinging Maybe it's a dog toy or something with a tennis ball at the end that's swinging it around their head and at point P the rope snaps so Go ahead. Let's see the clicker a system is on so you can vote which path viewed from above by a Stationary observer would you see the ball follow after the rope snaps a bc d or e? Now there's a little bit of stress in the room because you're all postdocs and you're thinking I should know the answer to this and probably you do But but you know that the reality is that the undergrad freshman in my class also feel that stress And there's some value to that stress and it's anonymous, right? Clicker down here and Okay, did you notice how many how many clicks we were getting whenever is everybody voted I? Thought it was 50 So if you haven't voted don't forget you might have to turn the power back on make sure that it flashes when you press And you can you can change your mind right? It only it records all the votes, but only the last one counts And having trouble you can just raise your hand. We've got more clickers. Okay. Well, that's interesting the the ones those of you who waited the longest Did the worst because the score just went down Okay, so I'm gonna have you Answer another question, and I'm gonna have you talk to your neighbors this time And so in the process of talking to your neighbors, you're welcome to check in about this physics You might think about the nature of this question and whether this is one of my traxling questions or something rather different In particular, let me emphasize that the questions on the fci this inventory Are never asked during the semester So I would ask this question at the beginning of the term then we would teach For instance when an object is in circular motion. What is its velocity vector at point P? What direction is it traveling and then we would talk about Newton's laws? And I would say an object in motion remains in motion That's Newton's first law and I would have some examples, but none of them would look like this I don't want my students memorizing the answers because then I haven't figured out if they can put ideas together At the end of the semester, I'll ask the question again. So what I want you to talk about with your neighbors is At the University of Colorado up the street Big public institution for the algebra based students So that's probably not the physics course that you took probably took a calc based physics course This would be the integrated physiology majors. They want to be doctors. They want to be nurses vets Physical therapists and then you know, there's some architects, right? It's a bunch of people in this room How difficult is this question for them? So take one minute talk with your neighbors vote as soon as you you know You can vote now and then change your mind. I just want you to This is before Before they so they walked in the door I give it as a pretest and you can think in your group about how they're gonna do at the end of my course I will tell you in a minute. What do you think? Is there a consensus in your group? What do you think physics professors think the answer to this question is? Okay, folks, so I'm gonna collect this one So so vote if you haven't already voted We had about 50 people clicking away Remember it has to flash green. So if it's not flashing, you know, you can press it a bunch of time I hear students in my room like click click click click Okay, oh, that's an interesting distribution. So let me let me show you sometimes in class It's really helpful to stop a question and then show the histogram. So kind of a split between C&D with Half a dozen of you saying how could I know this? Okay, so so the lesson here So it's not a wrong answer. You're not physics professors. So so, you know, what is the answer to E? How could you know this? Just read a paper. I mean, there's all sorts of literature about student difficulties And if a concept inventory exists, there will also be papers that will tell you how do students in different populations do You know the answer to this question is it different for the algebra based in the Calc based students And so the answer to E is just read a paper or two about the teaching of the course that you're going to be teaching And you'll know the answer So I will forgive you for voting E. But you know if you're going to teach the material don't stop voting E All right, how could you know it learn about it? People do research on this stuff As for the rest of you? Yeah, I mean that it's really hard to guess if you haven't taught freshman physics in my class last year at the beginning the answer was At least very difficult. So the score on this was approximately 50% So half the students can answer it and half cannot And you know if I call a a like 90% and be a b You know like 80% then my students are failing on this question at the end of the semester Okay, now this is the part that I have to hang my head a little bit You might think oh come on Steve you as professor of the year So so the score on this particular question goes up from 50 to 75 So it's good progress. Okay, but let me put it this way They had 50 points they could have gained on average right and they gained half of that So the normalized gain is 50% they gained half of what they didn't already know by normalizing the gain I can compare my students with students at a community college who walk in the door getting 20% because that's random guessing or At Harvard where they walk in the door getting 75% as well as my students walk out the door And so you know if I just compared how much they gained without dividing by how much they could have gained I can't compare CU instruction with Harvard instruction So let me show you a histogram of how universities across the US do on a 33 question Test that's all like this. So it's all conceptual. You don't ever pardon Oh, we're not there yet. Okay, so Then then Very good one more activity before I show you the data So we wanted to give you a chance to see what these sorts of concepts inventory questions look like in a discipline That's perhaps a little more familiar to you So we have pulled some questions from the geoscience concept inventory, which is in development So it's an earlier stage than the first concept inventory. We have less data on how students actually do on these questions I'm going to give you these questions in a moment But first I want to frame let go of trying to just argue about the answer It's really hard to do when you look at the questions But what you want to be thinking about is to try and to get into your students head I mean think think about what the answer is but then focus most of your discussion on what do you think our students are going to answer? Why are they going to answer that way? What are the going to be the sort of tempting wrong answers for your students? So Yeah question You make it up, right? I mean everybody in this room has some the question was which student population Are you thinking about and until you're teaching a class? You don't know. Yeah But these are all fairly in introductory and we tried to take a smattering of different topics So face down on your tables is this sheet It says questions from the Geoscience concept inventory. You've only got a few minutes to do this So I don't get too stuck on any one question But discuss what you think the right answer is and what you think your students are going to answer and why Do we have extras so you thought we were going to be done with this activity now Yeah In the stratosphere So your group is being very quiet I might encourage you at this point to just One of you who sees something that's like triggering an idea. Let's like sort of talk a little bit about that one question Yes, and and why are they answering the wrong answers like where is that coming from? Kitching it up Later And not very long I think it's not important for them to like get get too far in this it's more that they looked at it and thought about I've never taken Another 30 seconds to finish your train of thought. Okay, let's let's come back together In the interest of time We're not going to go through every single one of these We'll see if we can get to two, but so whoever volunteers pick a meaty one Any volunteers to talk about one of the questions that you talked about and where you think students might go wrong? I do have the right answers now. Of course. I didn't write these questions Why don't we pick the question first and and then we'll Is there one that you think is particularly rich like as a non-geoscience person I don't know which one is going to be give us the richest conversation You okay, so number two Any brave souls to talk about why you think there? What what the right answer is you think and why? And a group of geoscientists Thank you brave soul. I was gonna say I don't like any of the answers So it's definitely tempting to pick to pick apart the question and I you know not not knowing this But right well the answers are part of the question I don't want to get into into picking apart the particular question I was I'll go ahead and and do the reveal in that I was told by the person Who authored the question that the answer was D? Do you guys like that answer? More than the others. Yeah, right Right raise raise your hand if you Think that D is not a good answer just from a scientific standpoint. Okay, so so there is something called Question validation where you go through and make sure that the answer choices that you've chosen are good answer choices But let's assume that this is The best answer for this particular question for the moment and we won't try to rewrite the question What do you think students might answer to this? Well, how might or how might their thinking go go astray on this question? We thought at our table Excuse me that students might get it backwards and think that earthquakes caused The plate motion because they know that the two are associated So that they connect earthquakes with plates and so they just sort of you know, don't really understand the underlying mechanism And so they just sort of regurgitate something that's not correct because they've got the wrong mechanism in mind So these are really helpful to go through and and think about Just exactly this question where your students might go wrong And and what you might do to help them With those ideas what activities you would do in class if you found on a pretest that students were were answering incorrectly To some of these questions So the the full concept inventory is at that link on the top of the page and you can go and it's actually a wiki And you can input your discussion as to why you think that there should be a different answer choice for example Because this is this one's still in development So it's really helpful in physics that we do have a lot of data on student responses to questions like the force concept inventory question Because we know just where students tend to go wrong What difficulties they tend to have and have done research on what kind of teaching helps to Impact those difficulties. There is also some some research in in the geosciences on Common student misconceptions, so I thought I'd just sort of share some of those We tend to like to call them student difficulties rather than misconceptions Just because it's a general area that students struggle They might not have a well-formed opinion yet, but some people do think that summer occurs when earth is closer to the sun That's a pretty common one. That's been documented In astronomy as well as geoscience classes Holes in the ozone layer are the reason that global warming is happening. That's a commonly held belief about climate change And then floating continents that underneath the continents. There's some layer of liquid that the continents are floating on And why else would they call it drift? So just to sort of center your thinking in in your discipline There's this is some content about the geosciences, but now we're going to go back to physics land. So go ahead Yeah, so, you know when you read literature you find websites or research that tells you information like this and you're about To teach the topic of seasons, you know the idea here isn't to tell your students now Don't think that the reason why it's hotter in the summer is because we're closer to the sun That's not going to be very effective knowing these is just the beginning of thinking about how best could you help your students to make Sense because you know, there's I would say number one is good scientific thinking. It's just wrong I mean when I'm closer to a hot thing. It's hotter. So there is some good logic that sits behind that It's not even necessarily bad science except that it's wrong because there's other reasons that are much more important So, uh, it's lost Right, so, uh, you know, we we have a few ideas for this session and we've articulated them So lesson number one know your audience is, you know, it's sort of obvious I noticed that this was also the statement that was made at the end of the previous presentation It's it's so obvious that sometimes you lose sight of it you there's a tendency and I Fall victim to this myself that you think your student is a bunch of little mini-yews Just younger right so so you're teaching to yourself You're thinking how would I have learned that this idea the best and then you know You go to the board and you give the clearest explanation you can knowing your audience is super powerful It's very difficult, especially when you're a new teacher So finding the literature is one way to do it another is to what will show you some ways that you can find out What your students think on the fly? Formative assessment is the sort of idea that all the time these clicker questions don't count Against your grade in my class as you're voting. I'm getting information. You're sharing information. It's a communication tool It is not a test So that you know that can be very powerful and so knowing what they know already So that you can bridge them from what they believe and what they think to where you want them to be And way better than just assuming that they don't know anything because that's not true either So this was part of this fourth concept inventory. I told you about my scores on this particular question Here is a histogram of the overall learning gain that normalized learning gain So I don't know if you could read this axis it starts at zero It doesn't even bother going up to a hundred percent because nobody ever gets close to that This is six thousand students at a dozen universities. It's a big meta study by Richard Hake It was published in the American Journal of Physics almost 20 years ago Many many physicists, especially 20 years ago read AJP was like one of the big journals for us And so we all kind of saw this graph and started scratching our heads This is a disaster at this peak. This is a histogram of learning gains. It's picking at 25 percent So that means that if the students start with a 50 they go a quarter of the way So what's a quarter of 50? You know, it's like 12 points So they go from a fail to a D, right? It's it's really depressing and so now you know People started asking all sorts of questions one of the things that people did was to look at the questions on the FCI and Start picking them apart. Well, if you take into account special relativity that answers wrong But you know, that's not what the issue was with the freshman another Another thing that you know, that was almost universal was and I did it myself it not my class I don't believe it right like this is education data. It's noise So, you know people tried to replicate this and it's really easily replicable data So for 20 years people have been giving the force concept inventory And this is what you will score if you stand and lecture and you can explain you can know about those Misconceptions and you can carefully explain them to your students and they'll still gain about 25% on this this kind of a test So, you know, so what's going on? What's the reason? Well, you know, we've been talking about reasons One of them is just it's like how our classes are built. So here's a picture of a classroom The good classroom at least it was it was the best classroom It's that state of the art 2,000 years ago and now I'm going to show you my classroom up at CU Which I'm going to be teaching in tomorrow Kind of the same room, isn't it right like we can teach at night now and when it's raining, but otherwise Fundamentally, it's the same idea rows of obedient students listening Scribing down what I'm saying so that the knowledge of the professor can go into the notebook of the students without passing through the mind of either right so So, you know, this is a problem and and it's a problem you will all face This is my classroom. I got to deal with it So I got to think about ways to leverage that physical space and do the best I can with it There is a you know a theoretical models of how people learn that I mentioned I'm going to show you a diagram of a Conventional model of teaching and learning called the transmissionist model the idea is I know stuff. I'm speaking now You know it so here's the here's the diagram So, you know, it's easy to make fun of there's a lot of things in this picture that are connected with the transmissionist model So students are empty. They're passive Learning is a thing. They get filled up They go off into the factory, right? There's so many elements about this that you're just thinking. Oh, that's all wrong And yet kind of this underlies how I started teaching I thought basically this is what learning is you guys are listening to me and now you know stuff about teaching that you didn't know before and You know the one of the contrasting theories to transmission is construction So you are a master learners. I know this because you're postdocs There's no way you could be here without being a master truly masterful learner Which means that you are engaged with me right now You're questioning everything that I'm saying you're connecting it to your own teaching experiences You're already starting to think about applications. You're doing all the things. I want students to do But you know, you're not really being passive. You're actually being very active Even if you're silent right now and in my freshman class I got plenty of data that suggests that almost none of the students are like you so we have to make it explicit We have to start talking about what it means to learn in the classroom. So this is the buzzword I hope it's not traxiline interactive engagement. So so There's two parts to that interactive and engagement. So, you know, I've had faculty members who say well I'm pretty engaging, you know, I got a charming British accent on my colleagues and you know So so yeah, you can be engaging but that's different than interactive So the idea here is that students need to be building ideas in your classroom all the time And they can only do that based on what they know You have to know what they know and there needs to be feedback you have to be thinking all the time when you're teaching about Local feedback what's happening right now? That's giving you an idea of whether what you're learning is correct and making sense and allowing you to move So I'm going to do another little mini lesson How many of you have had a formal lesson in refraction at some point. Oh, so almost everybody. So, okay, this is a little review I've got a light ray coming in Already there's some issues like I'm representing light as a black arrow. So you got to think about what that means It's coming from air and it's hitting a block of glass. What does it do? So, you know, if I'm in lecture mode That's what you might think if you're an introductory student that it light goes through glass So it goes in a straight line, but it doesn't here's what light does it refracts and so we can start to think about How would we describe this not explain it but describe it? So one thing I might do would be to define the geometry of the situation So I would draw I would I would identify some number that characterizes the two media index of refraction and one in the top and Two down below. There's a number that tells me something about light and then I might want to identify a normal Right that little dash line is perpendicular to the surface and now I can label angles and You know, there's an ambiguity. Should I label that one theta one or should I label? It's complement theta one. It's a convention and I decided to label this one theta one and by naming it this way I can write down a law which was discovered Experimentally empirically long long time ago n1 sine theta one Equals n2 sine theta two and if this was a traditional lecture I've got my lovely power points and I was as clear as I could be and I think we're done with this topic You can now solve homework problems And you know if the homework problem is n1 is 1 theta 1 is 20 degrees n2 is 1.4 calculate theta 2 You can all do that and you know feel like you've learned about Refraction, but you know, I would say there's lots more to refraction One thing that I would certainly do at this moment in my class and I might actually give this lecture Because this formula is going to be useful Whoops for our you know for the student understanding At Colorado and all over the country, but Colorado has this site. How many of you know about fat PHET? Oh Check it out PHET dot Colorado edu. It's like hundreds now of sims They simulate everything it started off as physics, but now it's got you know glaciers and climate change and radiation and chemistry and biology So I pulled up the there's one on bending light and so you can mess around You know, here's my little laser beam. Oh, I just discovered that there's also reflection And I can you know, I can mess around so so you get this you can do this in front of the students better Yet get the students to mess around with the sim on their homework so that they can you know see for themselves What happens, you know, everything's sort of live if I change the index of refraction of the glass Notice that the refracted ray changes angles when the two are equal and there's a little number that's showing up So I can connect this to the formula light goes through a straight line So, you know messing about at this stage of learning can be very productive and then I go back And maybe in class the next thing that I might do would be to ask This question So let me let you vote on it Talk with your neighbors right away about this. So let me read it for you array of light passes through a sheet of glass Oh, but it doesn't have parallel surfaces anymore. I've made it thick at the bottom and thin at the top So it comes in there's glass. It's got a larger index of refraction Which path will it follow when it comes back out again a B or C just qualitatively? I Claim you can answer this question with n1 sine theta 1 equals n2 sine theta 2 But it also requires some drawing and some thinking you've got paper and pencils, you know have at it for a minute A is bent towards the thin end B is un deviated C has bent towards the thick end Many of you know the answer to this from your real-life experiences or from previous classes. That's fine You can vote the correct answer, but in my class the discussion is about the reasons not the answer So that's you know, if you're voting whatever D because you know, that's the right answer You may convince everybody at your table to vote D But when I call on your table and you say cuz Steve said so Okay, take a moment. We're not going to spend very long at the point here is not for you to study refraction And in fact you can immediately go meta and think about what am I learning here? Is this a traxiline kind of question? Well, you know, what's the nature of this question? Would it be valuable? Discuss We won't spend too long on this either There's nobody. Oh, you've started drawing. You've started drawing Yeah, I would walk around my room and say if you don't draw something you can't figure this question out unless you know the answer Yeah And even drawing it there's you know, it's a non-trivial question So you're answering the question and I'm not seeing a sketch and I can't imagine oh She's sketching but she's like way over on the other side of the table I can't imagine how I could answer this question without drawing a picture So I would walk around my room and tell my students, you know, please Start drawing You could try I Find it hard. I mean, this is just complicated enough for me. I lose track of like what's happening The picture here is for me. It's worth a thousand words Obviously I think of a laser pointer It is a wave phenomenon, but you can think of it as a ray as well Yeah, very good, yeah Monochromatic makes it simpler if there's multiple colors It's you'll have a prism and they'll be like a sleigh. It's really small You know the difference so whichever one you're voting a B or C the difference for the different colors will be a really really small It's enough if you get far enough away that you can see the rainbow But it's not the dominant effect the dominant effect is just this one any ray Okay, so you know this is a fun question and I bet we could leave you for 20 minutes and everybody would have the answer and That might be what I would do in class But we've got you know other things to talk about not the physics of light I just wanted to give you a moment to experience You know what it's like to be in a classroom that is a mix of teacher lecturing But also you sitting in small groups arguing thinking I Could pull up the FET sim And I will just recommend that you know oops if you if you want to Mess with this you go to the prisms part of the FET sim and oh it just so happens that I have such a such an object and I can turn on my beam and and answer the question now This isn't explaining it, but at least I know what the answer is and I actually get some insights like as I mess around You know I sort of discover what happens when we come into different angles and There's a lot that you can actually begin to develop intuitions But it always is bending towards the thick part which begins to explain what's happening Why are glasses thick in the middle and thin at the top and bottom? Well for some of us actually my glasses are not like that my glasses are thin in the middle and thick at the top and bottom Because you know next in class what we would do after students have learned about this certainly if it was my algebra based class would be to a Yeah, I'm not gonna walk you through my explanation Would be to talk about eyes because in my algebra based class There's at least 30 people out of 300 whose career is going to be you know They want to be opticians and so all of a sudden they're at the edge of their seats And I can explain, you know far-sighted eyes versus near-sighted eyes And I will then have a clicker question where I show a close-up view of two people wearing glasses And I say which one's near-sighted which one's far-sighted just by looking at them and what their face looks like behind the Glasses if you've made sense of refraction. You can actually answer that question So this connection to people's lives is going to be part of the story Which we didn't have yet, but you know doesn't have to come right at the start sometimes it does My colleague Michael Dubson drew this cartoon about teaching I claim it's a picture of me because I used to have a beard, but he says it's not So here's me or somebody teaching the Gaussian exponential function, and I you know I worked on this lesson I may have forgotten the Gaussian exponential function until last night, but as I was preparing my lecture notes It's now so clear to me right and if only if only I could get into the heads of my students It would vastly improve my ability to help them to understand the Gaussian exponential function Now the point of this cartoon is not to make fun of students Right the idea here is that everybody's looking at this and making sense of it in their own way based on where they're Coming from and I'm not even making fun of that student because you know There's people in my class who are working two jobs, and they're exhausted And so you know part of what I'm thinking about is if I'm aware of this I can try to Figure out what's going to wake them up in my classroom. I'm not seeing that person in this room yet, but we're not done yet So I showed you the dismal learning gains in traditional lectures Here's data from that that same 1998 publication on what happens in interactive engagement classrooms And you know this was published and see you said not us But in fact, it wasn't us because we immediately Started doing interactive engagement right away. We saw this paper was just it was like of course we should be doing this It just seems so natural. Here's the CU data And so a education data it is replicable and lots of institutions have produced the same plot Typically junior faculty, that's me You know, that's why you get the Carnegie professor the year Fall into that pin. It's still only, you know, 60% learning gains. So we still, you know got a long way to go and Typically a junior faculty move up. So that's also inspiring right it means that well as you teach and you learn how to teach You could better at it like anything. I mean if it was just an art Then you might say you're saying I don't know I'm just not a natural born teacher and that would be a really destructive attitude to have everybody, you know people send their kids to a Piano lessons. So and they don't say well, you're either a pianist or you're not like there's no point in taking lessons So lesson number two interactive engagement works and you know, I got Decades of data that suggests that you can't teach students. They teach themselves And so this style of classroom is demonstrably more effective It changes learning gains by factors of two or three and so it's huge And you know this little quip teacher show you where to look not want to see Yeah, I didn't write that one And now I'm thinking about whether I agree with it or not, but anyway, it's my PowerPoint. I'm sticking with it So let me quickly just sort of mention Ways that people have developed. There's a million ways and everybody kind of figures out their own I've been showing you about clickers and peer instruction. I think of this as it's still lecturing it's a part of lecturing and You know you interest first questions They should be challenging and the element here that is really crucial is the peer peer instruction So when faculty give clicker questions and it's silent in the room and the students think and vote and then you move on The learning gains aren't nearly as much. It's the conversations that are really critical in this pedagogy Sometimes you really break your class up in a different way People get into small tables and they may have a worksheet and you can do it in a crummy classroom with You know these little benchy seats that we all have to work with Again a reconceptualization of a lecture or in this case a recitation which is our you know one hour a week with a grad student And maybe 30 people instead of 300 Some schools and the odds are pretty good that if you go to any institution and get an assistant professor position They will have at least one room like this and you should find out about that and ask if you can teach in it Because when you're teaching in that room, you're not teaching in the 2000 year old You know, it's going to change what you think you need to do You've got to come up with activities where the students are talking with themselves You're not the center of attention anymore. They are there's a lot of that this Reconceptualized classroom brings you see you has one of these rooms and it's actually ironically too small for me to use It only holds a hundred people and I'm teaching 300 so we got to do other stuff You can use tools, right? There's the FET sims. There's millions of these things and you'll want to go hunting for them Just in time teaching is a so I'm now like throwing words at you. So I'm gonna stop This comes from the Air Force. It's actually a really simple idea. It's so obvious that I you know when I heard about it It was another one of those I got to do this before lecture I have one or two conceptual questions that the students have to answer and it's do an hour before class So a they've started thinking about the upcoming material. It's not graded. It's low stakes But it allows me an hour before lecture, you know, I said know your students Well, what a simple way to know what my students think before class starts So I've got an hour to make some tweaks and changes and think about what I'm gonna do You can just Google just in time. It's just in time because right you're getting the feedback just before lecture In my class there's a lot of lecturing So you don't have to sort of feel like after you hear a talk like this that you've got to put students in small groups And like that's the way they're gonna learn There is a clicker question running in any class low level high level so that they they're they're getting into the classroom mode They're getting into the topic of the day and I might have out of 50 minutes 20 minutes of me talking But it's interspersed based on psychological research about, you know, typical 20 year old attention spans So that leaves 30 minutes for the student student conversations This balance is not, you know, a number that comes from some research paper It's my intuitions about what works with my population of students your mileage may vary You know 30 minutes on five questions gives you six minutes per question for the students to think to discuss Maybe they vote twice. They might vote on their own and then they might discuss and they might vote a second time That's a very good way of you know finding out what's going on in the room And I often will end with it with a fun Puzzly clicker question so that they have something to think about for the next two days that we'll start talking about in the next class and On top of this just so that my colleagues don't like blame me for watering down the physics curriculum We give the same old difficult homework questions mixed in with some conceptual questions I still want students to be able to compute stuff I would say my exams are more in line with the pedagogy So I have to think about that. I can't just ask compute Snell's law angles anymore I got to think about more conceptually focused questions and Next activity Actually before we go to the activity. Let me just take one quick pause for questions or comments Anybody got anything that's like bugging you or hey, thanks Okay, so so let me let me repeat in case people didn't hear the question was specifically about I would say math phobia And or lack of math skills and you were thinking about upper division courses where it becomes really problematic If you're talking about majors, but it's also true at introductory level where some fraction of your class is not Starting from the place where you kind of feel you need them to be starting from so I think we should probably talk about this offline There are people who study questions of both math and the teaching of math in the context of the sciences Contextualizing the math can actually help bring a lot of people in so that so that they begin to see, you know write the joke about People are having a conversation at a party and this person says, you know algebra what I'm wasting who cares about algebra and the other person says Oh, algebra is awesome. I use it all the time. Oh, what do you do? I'm an algebra teacher. So, right, so So, you know helping the students to see why the math will help them in the course in their life Is part of the story so I think I have to defer that one because it's it's complicated and I don't have quick and easy I typically don't have quick and easy answers for any real question that you have Okay, so actually let you know we're gonna do another activity I'm gonna be wandering around so we can actually do kind of individual questions during that period as well So this next activity I call gallery walk because it's an opportunity to get a chance to look at a variety of Questions that you might ask students now the two questions that I've chosen are not all Geoscience questions. There are some in there. There's some from chemistry physics math, etc so The the challenge I posed to you When I set you free to look at them is to try to look beyond the content and to not just again Try not to get too stuck on just figuring out the question and what the answer is but think about what the question is forcing the student to do and what what the teacher might be trying to Accomplish by getting the student to think about that question so the three questions I Posed to you as you look at the questions are number one if you look look you can see all of these questions are grouped into a Group and each group is grouped by a theme. It's a theme. I have in mind You might end up with other themes that you have in mind I want you to try to think about what the sort of unifying theme is for those questions and the the label on each Question group you can lift up and you can look underneath it and and get the reveal of what what I think the right answer Yeah, yeah, the right answer but think about it first and Then I also want you to think about why you might use that type of question You know, even though it's a physics question You might be able to use that type of question in your in your classes And whether or not there are any questions that you think there are bad and if you think that they're bad Why you think they're bad? I I did see it a couple in there that I think are not the best questions So the the questions are there's two sets of the questions All the ones along this side is one set all the questions along the brown wall and around the corner are another set So what I want is for the we've got about that's our identical, right the identical sets And within each set there are five different groups I'm a clay chair, but so what I want is for the folks on this side of the room So let's say those three tables along that wall plus the two back tables go to those questions and then These one two three four five tables Go to these questions and spread out amongst the groups to start So you're not all trying to 20 people look at one group questions And try to only spend like three or four minutes for each group of Questions if you want to try to see all of them that will let you see most of them And as usual we're gonna cut you off well before you're done with this activity Yeah, and talk to each other. This is not a silent activity Yeah Okay, we've run low on time so Finish your thought and please go find your seats Just out of curiosity quick show of hands. Who was able to get to all five groups of questions? Now who got to four four of the groups? Okay, so we got Got a fair way around First off anyone spot any stinkers in there any questions that you really didn't like like I said I seeded a couple that I wasn't crazy about questions you You didn't like They're too easy poorly worded I'll tell you one that I'm I'm not crazy about it is in group one Group one was the facts and concepts group Anyone think of the question I might be referring to in that one Tectonic plates. I don't know enough geoscience to not know know whether or not to like that one What's wrong with the tectonic plates question? Huh? Boring, okay, so maybe it's too Is it is it too easy like the students are all gonna get it, right? I didn't quite hear that could you repeat, huh? It's traxiline. Okay, so so students could might be able to answer it without actually understanding the underlying concept That's important because if if that's the case then the the student answers to that question are not going to inform you And they're not going to form the students as to where it is that they're having trouble So so that's a good thing to notice. There's another question in group one that I'm not crazy about It's the Leah and Sue and their ages Tell you every time you ask a question like that as a quicker question like you know where you're asking students to talk to each Other immediately the room goes silent as everyone turns to their paper and starts scribbling now The calculation questions have a utility like we still want students to be able to calculate But they don't tend to a question like that does not tend to lead to good rich student discussion Like you want to have student discussions around the things that are hard that students need each other in order to to get to the answer So so group one is sort of these facts and concepts questions questions Which is very similar to the the little mini lecture that Steve gave us with the refraction question And that's pretty pretty common type of question that we might ask either on Worksheets or in clicker questions. What are some of the other types of? groupings of questions that kind of surprised you or give you a new way to think of of what kinds of tasks you might give Your students any of those groupings Sort of illuminating for you the sparking discussion questions. He likes why did why do you like those? So it's so he said you're bringing in discussions around what could be around like actual physical content and so You're able to really have an argument and and people might argue for many of those different answers So this is the questions like which superpower would you rather have the ability to change the mass of things? ability to change the charge of things when I give that question in workshops usually people say that they want to change the mass because They want to lose weight whatever But but those those questions where there's not like a right answer It's all about the argument is also really helpful for framing for your students What it means to learn science in your classes. It's not about the right answer It's about being able to articulate your reasoning and are you a point to defend a point? That's a much higher level skill Than being able to just regurgitate traxiling Let's see and then The other groups we had facts and concepts. What's what was group two remind me? Which one is group two is that? prediction or is that image that one's images images, so I I put in an example of a worksheet a lot of the worksheets come from the science education resource center that I linked in the online resources for for this group one thing that I Really envy of of you guys in your discipline is that there are a lot of nice ready-made Worksheets clicker questions, etc. That you can download. I had one question during the activity as to what is the? cost the sort of startup cost to doing some of these being able to beg borrow and steal resources To run clicker questions or worksheets or whatever in your class is really helpful So I think I'll end the discussion there on the gallery walk. Did you have anything to to add? and Turn to our next any We're doing good So I want to just Pull together some of these ideas of like what did these questions help your students to do? By talking about something called backwards design, okay? I know that's traxiling again, but it's helpful traxiling because if you Google backwards design you'll get a really good Framework for organizing your your classroom design So backwards design is the idea that is that you first identify where it is you want your students to be where are you going? Figure out where you're at so your student ideas and then figure out how you're going to bridge the two How are you going to get there so when you're designing a class using backwards design first you figure out What do you want your students to learn those are the learning goals that you've got for your students then? Look at those student ideas. So remember know your audience Was our first lesson and so we want to know where it is that our students already are at and once we have those Two things where we want our students to go and where our students are already at then we can design our instruction That helps us to get them there and then At the end oops at the end you want to make sure to assess whether or not students got where you want them to be So you have lots of different goals that you might have for your students Conceptual learning so group one was that that sort of conceptual learning you've got for your students Skills the ability to perhaps make predictions. So there was that set of questions on experiments and procedures Learning to learn you might want your students to be able to Become expert learners or reflect on their own learning Or attitudes and beliefs about the subject itself. So an Example conceptual learning goal would be students should be able to identify the geologic Processes that generate water resources. So depending on your class that might be a goal that you have being able to articulate a learning goal That's that specific really helps direct to your instruction and then helps you write questions like what you just looked at For attitudes and beliefs. Here's an example question Students should be able to recognize that newsworthy events related to geology occur every day. So that is the kind of So attitude or idea about the topic that we might want students to come away with So When you start with your goals this way You can then use this to design the actual course once you've developed all of these learning goals it can be very helpful and Then it also makes your assessment really easy because once you've identified Exactly what it is you want your students to get out of your class. It's much easier to write the exam So sort of a tip is that this kind of kind of thing Written on to a teaching statement can be very helpful What kinds of goals do you have for your students? Do you recognize that there are a variety of goals? And can you use backwards design this idea of setting goals for your students? In order to talk about how you'll be teaching courses so I want to turn it over to Steve to talk about I have only one small comment to make about that which is When I'm thinking about learning goals It's easy to to be sort of glib about this So what are my learning goals in physics one? I want my students to understand Newton's laws I want my right. That's a terrible learning goal. What does it mean? Understand Newton's law. Do I understand Newton's laws? I mean people are still writing papers in which they're applying these ideas and developing further understanding So I think the articulation of the learning goals is often best framed as something they can do Rather than understanding topic X because that's ill-defined. So that was just like a little quick comment And actually yeah, that just reminded me of a couple of other things I would mention just to connect it back to the gallery walk. So If ideas about learning or something that you want your students to get there are all of those those questions About you're asking for your students Ideas and attitudes the survey questions I'm including like what's difficult for you or which of these procedures is most easy for you to follow Those kinds of things would explicitly help Bring students along in that kind of goal and then The images and graphical thinking perhaps I know I know in many of the geosciences It could be really important for students to have graphical literacy, but if you don't teach it They're not going to come away with it So thinking about what kinds of activities or questions to get students to that kind of skill So that's the sort of match between instruction and goal So let's I think I'd like to sort of wrap up There is this Lesson that if students, you know, I think about this in physics a lot if they have learned Newton's laws But you know go to a party and say oh you're a physicist. I hated physics, right? This is very common and very depressing to me and and to me I think you know morale in the classroom is vital and it's going to be true in a geosciences course in a physics course in Any science course so listening to the students There's lots of ways to support this, but it's worth being kind of thinking about it explicitly So here's here's my my summary slides This is a real book. I think it's a great philosophy of medicine I think it's an equally great philosophy of teaching kill as few patients as possible So, you know, we've said these things you have to know your audience Attitudes and beliefs are really important if all you're doing is talking about concepts all the time You may not be addressing this and attitudes and beliefs doesn't just mean I like the class attitudes and beliefs might mean Learning science means that exam questions are going to be things. I've never seen before That's a belief that I want my students to have that they don't have when they walk into my algebra based class They think that science tests are regurgitation tests And so they're shocked when they actually have to do a lot of work to get them ready for my first exam So that they understand that that science is not about remembering it's about it's about figuring stuff out Active learning works. This is one of our themes And you know, I'm not sure if we've emphasized this but conceptual understanding doesn't come along for free knowing Snell's law doesn't mean that you understand eyeglasses So so starting to pay attention to that explicitly and distinctly Will help you as a teacher because it focuses your attention on things that maybe you didn't have your attention focused to when you Were learning that subject and the natural thing to do when you teach is to do what was done to you The easiest first thing that you that you want to do and here's my last point. I think it's super critical We're not here trying to give teaching tips. I'm not really I don't research teaching a research student learning And I think that's like a shift in focus that will completely change your teaching because it's not it's not about really what you're doing It's about what's happening to them and if you keep reminding yourself of that Then you start paying so much attention to your own activities and stressing about it so much And you start thinking about assessments and learning goals and activities And so thinking about a classroom environment as about learning rather than as about teaching is I think it's super productive and So I don't disagree the teaching is an art But I also want to argue that teaching has scientific aspect to it. You can collect data You know, I like to joke. I used to do nuclear theory. It's it's a it's a complex nonlinear many body interacting system just like teaching so right you know, I there's a lot of science that you can do and You all are right now Appropriately focused on research in your lives, but you're also thinking about getting jobs where teaching may be an element And I would like to argue that there's no reason why these have to live in separate silos. So this is my little political pitch I argue in the physics department sometimes vainly about this, but I'm teaching can be improved by using your science skills and Applying them not to questions of geoscience or physics or atmosphere But to questions of student learning and when you start to do that, you know your classrooms change. So I think we'll stop here Yeah, we'll leave these up as walkways and I think we've just used our time Thank you so much and I Think you'll agree that that was pretty great So and if you can come away from it remembering how did you feel as a student in this session and you know Remembering the techniques that they use that they were modeling, you know getting us to talk and walk around and Those are those are things also, you know transfer. So thanks again Thank you