 A lot of students think that there's a real benefit to small classes over large classes. I mean, we all have experience in sitting in lecture halls, massive, you know, 500 people in a lecture hall versus an intimate sort of 20 people in a lecture hall. Is there any evidence that, I mean, we like to think that small classes are substantially better than large classes? It seems intuitive that they would be right because you get more attention potentially from your teachers and so forth, and there's quite a bit of evidence, actually, enough evidence to do meta-analysis after meta-analysis. It turns out that the relationship between class size and student achievement really is really small. In fact, it's almost non-significant in many cases, which is startling that it would be that way. But in fact, smaller classes aren't necessarily better. So I suppose the reason might be that small classes aren't as good, I suppose, is that you can do more in small classes. Just great, yeah. But is it because they're doing the same things in the small class, in the large class, or is it...? I mean, if you think about it from a teacher's perspective potentially too, a lot of the education reform, like, let's get smaller and smaller classes, well, if I originally had a large class, like, say, 40 students in biology in high school or something like that, and then, lo and behold, we're all going to, like, spend a lot of money and just have these really, really small classes. But if I don't do anything to change my teaching style to take advantage of the small class, the student achievement really isn't going to change. So that I think is probably part of the equation, that just because you have a small class, if you don't really utilize it and take advantage of the fact that you now only have to interact with a handful of students, achievement is just not going to get any better. Sure. So I suppose it's what you make of it. Absolutely what you make of it. I mean, 40 kids, you're constrained, right? You can't do a lot of really good things for the students that you could potentially do with only 10, 5 students. But then if you don't take advantage of the small class, it's just not really going to help the students out. They might feel better about their interaction with the teacher. They might like the teacher more because they get interaction. That doesn't mean they're learning more, per se. One of the other ones, in Australia at the moment, our politicians are talking about producing really good teachers. Love it. And so a lot of it, they seem to be basing on just training. So we need teacher training. And they haven't really done much to kind of articulate exactly what that means. Yes. Is there any evidence that training teachers is effective? Unfortunately, just like with the smaller class sizes, teacher training isn't always that highly related to student achievement either, which just doesn't make much sense, right? Because we train individuals to become teachers because hopefully they can effectively then train their students specific content and so forth. It just doesn't seem to be a strong link between the amount of teacher training, how well students do. But many professional development courses for teachers don't necessarily train them effective techniques to use with their students. I mean, they feel good. They're intuitive. But there's not a lot of really evidence to show that any particular professional development program will necessarily lead to better outcomes. So in general, on average, many teacher training programs really don't translate into higher achievement. They can if they're done well, but on average they just don't seem to have a major impact on student achievement. Right. So you've done recently an enormous meta-analysis, an analysis of analyses. You've looked at a whole bunch of different types of learning techniques. So things that students can use or do use to try to improve their learning. Right. Could you tell us a bit about what you've done and what you found? Sure. With a large group of collaborators who actually worked very hard on this project for almost three years, we just reviewed lots of literatures, as you said, about the effectiveness of a variety of strategies. What we didn't look at were strategies that involved technology because we wanted to focus on just those things that any student could use. So just maybe with a penciled piece of paper or something like that. And we chose those strategies for two reasons. Some of them that we wanted to evaluate, we thought they probably did work, but why not check out the evidence? A couple other strategies, however, we knew students used a lot and we wanted to know, are these really effective strategies or should they be doing something else instead? So something that all students use, I still do it myself as a highlighter. We like to highlight things when we're reading. It's like a security blanket or something for learning. But it turns out highlighting itself doesn't really improve student learning, right? It doesn't increase achievement in any way. I would never take a highlighter away from a student. Again, it's like a security blanket. But it's just the beginning of the learning journey. It's not the end of it. So after you highlight all the important stuff, it turns out most textbooks highlighted for you anyway. You need to go back and use effective techniques to learn that material. So at least some of the things that students do like highlighting and rereading really don't have a big bag for the time box, so to speak. So we'll spend time rereading, highlighting. They're really not learning a whole lot. One, in fact, they can replace those strategies with other ones that really do boost their learning, which is exciting. So now we just have to retrain students, build a better student right to use better strategies. Interesting. So highlighting, rereading doesn't have much of an effect? No, it doesn't. Isn't that strange? You'd think everybody rereads, right? You go back to the two. Unfortunately, when you go back to reread, your eyes are moving across the page. It's probably late at night, the night before an exam, and your mind is somewhere else. So basically, mind-watering is a rereading. Students need to do things that are more, to engage them more actively. So in this analysis, you looked at some things also that had a massive effect. Yes. What, obviously, it makes sense. I mean, a lot of people, when they are studying or cramming for an exam, these are the things we do, but they might not be the most effective. What are the most effective? What are the things that actually work? Certainly cramming is not that effective, right? Students think it's effective, partly because they can squeak by potentially in the exam, and then they'll just forget everything. I'm not suggesting students don't study the night before a test. It relieves anxiety and all this sort of thing. But there are better things they could do. First, instead of just cramming, begin studying two or three weeks before an exam. So you're distributing your practice out across time, which is very important. So you're studying the same material over and over again. Now, that does take a little planning, maybe a calendar to remind yourself. Geez, it's two weeks before the exam. Now I really need to hit the books versus just the night before when you're panicking and you try to cram. So distributed practice is really good. But distributed practice just tells you kind of a schedule of how you should be studying, kind of like the when of studying. But there are lots of what's, the things you can do instead of just rereading the material passively. And one that we find is really effective and many others. There's about 100 years of research, very exciting showing how effective this is. Just retrieval practice. So after you mark up your book, right, and about all the important things, and you go back, instead of just rereading everything you marked up, cover it up with your hand and just try to recall from memory the content. And if the student gets that answer right from memory, that has a really potent effect on subsequent performance. They really learn it that much better. Of course, if they can't recall it from memory, then they can restudy at that point, but then they should come back and keep trying. So just retrieval practice, a really effective way to boost performance, especially if students use it distributed across time. Sure. So it's kind of like flashcards, I guess. Absolutely like flashcards. And most students at least say they use flashcards for simple, paired associate learning, things like foreign language vocabulary. But you can use flashcards for complex materials as well, concepts. So you write the key term on one side, definition on the other side, and then use that to basically test yourself and then to restudy. Even important concepts in textbooks, how they take notes, they can take notes in better ways to actually support the use of retrieval practice, which is an effective strategy that many students don't use. They under-utilize it, yeah. So retrieval practice, so then it's, I suppose, really pretty effortful by comparison to rereading or highlighting. So it probably takes a fair bit of desire on the part of the student to want to learn that. It absolutely does. I mean, it's very easy to sit there and reread, right? Especially when your mind is somewhere else thinking about how fun it's going to be after you take your exam type of thing, where it does take a little bit more effort that engages you to try to retrieve the information. The nice thing is out of all these studies that are done to compare rereading to retrieval practice, the time on task is always equated. So the students who are just rereading spend the same amount of time that the students are who are practicing retrieval, okay? So yeah, it might be a little bit more effortful. Effortful, you know, it's like a little bit more painful to try to retrieve stuff from memory. But even the same amount of time used in one strategy versus the other, the students who are practicing retrieval using a little bit of extra effort are getting a major, basically, increase in their performance. Interesting. Right, yeah, it's. And so the other aspect of that, the distributed learning, the spaced learning, what does that entail? I mean, so does that entail studying a little bit each day? Or is it, how many days do you need? That's a $100 question, or maybe even $10,000 question. This is how I think about it. Let's say you're a student getting ready for an exam. And maybe you decide, okay, I can give this four hours of my time. Most students, because they think cramming is good, they'll spend that four hours the night before the exam. Just study, study, study. They're going to do a lot better off if they take the same four hours and just basically segment it into four one-hour study sessions that are spread across, say, two weeks prior to the exam. Again, the same amount of time, but now that just spread across time where you're coming back to the same material versus just kind of going over that material over and over again during one four-hour block. How much they need to do for long-term retention, a whole lot. Okay, this won't be a great strategy to learn all your course content. You've got to decide what's most important, what you think you're going to be tested on, and focus on that material. But the more is the better, the more often you come back and re-study, of course, and use retrieval practice, the longer you're going to retain that material. So for stuff that students really need to know, this is an essential strategy. Else they just forget stuff. I mean, you don't have to tell a student, right, that after they take a test that they cram for, the next day they pretty much don't remember anything that they studied. Distributed practice ensures that you're going to remember that for a much longer period of time. Interesting. It's essential, yeah. Now it seems to me that the students who rely on highlighting material or rereading material... Which, by the way, is almost all students. All students, exactly. Yeah, when they're surveyed, about 80, 90% of the students say they use a highlighter. In fact, I think almost everyone uses a highlighter. Everyone says they just reread. Yeah. Yeah. How did they get it so wrong? How is it that students who use these very ineffective techniques at the expense of techniques like retrieval practice and space learning, certainly they would have stumbled across something that would have been a fit. Maybe not. I mean, are people very good at monitoring how well they do or their own performance? In many cases, they're not. So for instance, just rereading, for instance. Most students, when they're rereading, if they're reading fluently, so it's just easy to read, they think, oh, I'm understanding this. I'm getting it right. But it turns out that the ease of reading isn't really that highly related to how well they're understanding it. In fact, some really neat research that I've done with one of my collaborators, Katherine Rossin, we scrambled sentences within a text, scrambled them all up. And then we rewrote those sentences so that they flowed well, read beautifully. But it didn't make a whole lot of sense. Then we had students read that and tell us how well they thought they understood it. Unbelievably, the students who read this paragraph felt they understood it when, in fact, there was not much to understand there. Students can get a really good sense about how well they know something by using retrieval practice, right? Because if you can't produce it from memory, you probably don't know it well enough. So often this kind of these ease of studying leads students down this really unfortunate pathway. So they feel they understand it when, in fact, when it comes down to the test time, they find out that they don't understand it. I mean, all teachers have this experience, right, where they've hand back the exam grades, and then the students will come up and say, I thought I really understood this well. Why did I get an F? It's like, well, you probably did think you understood it well. It's just how you were studying wasn't really going to provide a lot of information on how well you comprehend the material. So students need to use special techniques to help them monitor their learning in this context. That seems to be really problematic, I suppose. So teachers are kind of rewarded for making students student satisfaction. It plays a big role in high schools and in universities. Of course it does. And so I suppose there's a real push for teachers to make the material go down as easily as possible, right? So to be as fluent, to make it as easy as possible to process the content. Absolutely. Isn't that kind of working at cross purposes? Well, unfortunately, the teachers fall prey to the same difficulties of monitoring that students often do. So if they see students picking up something very fluently, very quickly, they'll think they must really understand that. But quite often, the ease of picking something up, of understanding it, and initially, is not at all related to subsequent performance. So for instance, if you're just going to study something over and over and over again, boy, it'll seem fluent. You'll feel like you understand it. If teachers are watching it, you're not making mistakes when you're solving problems. So everyone thinks that this is a great technique when in fact, come back two or three days later and the students haven't retained anything. So often, unfortunately, those strategies that are going to work the best are effortful, right? And yet, understandably, many teachers would think, well, they're not really making fast progress there, right? Because it's effortful. So maybe it's not a really good way to proceed when in fact, no, it might be. I think we just need to inform teachers and students more about these kind of foibles of evaluating the quality of strategies because sometimes something that looks terrible in the moment because it's effortful is really beneficial in the long term. Yeah, interesting. The name of the course, Think 101 is the science of everyday thinking. And so we're really interested in the sort of strategies, the sort of things that people can do to think better. And that means we're covering a lot of material in the course from the sort of shortcuts that people tend to use to make judgments and decisions. With respect to the research that you've conducted over your career, what do you think might be a good sort of take home message as far as improving people's everyday thinking? Well, especially if the everyday thinking has to do with performing in the future, okay? Whether that be performing an exam, performing in workplace, just doing your hobbies better. The best way to think about will I be able to do something well in the future is to simulate it in the present, okay? So the more people can rely on simulations which sometimes can be effortful, right? If I'm gonna wonder how well will I do in this context in the future, the only way to really know is to try it out now. And then if you make, basically, stumble or you mess up or you don't understand, you know, oh, Jesus, this is gonna be problematic. So sometimes here, at least in thinking about thinking, this is what I'm talking about in this context, really can be effortful in its own way. So to evaluate how well you're basically picking something up, the best way you can do it is simulate it. The strange thing is in any other context, besides learning and education, students and people of all sorts simulate their goal before they get there. So if you're a young child and you have a dance recital, right, the way you're gonna figure out whether am I ready to go, here's just something I can do. It's basically simulated. You're gonna do your dance routine many, many times until you feel you have it right. You take that same student and you put him into, say, a mathematics class. For some reason those lessons learned in dance don't translate. So we can think a lot better about how we can perform in the future by simulate it in the present. Without a good simulation, we're really gonna make a lot of mistakes about our own abilities and how we'll perform in the future, so. Interesting. Earlier you mentioned the basketball analogy. I suppose it works the same way. Yeah, absolutely. So it seems very odd that people on one hand will go to basketball practice, practice day in, day out, once a week or once every day. Yeah, but they fail to recognize that the skills that they're learning there, the same sort of practice that they're using in that context, they fail to practice in the educational context. So you're suggesting that if they did what they learned in basketball and applied it to mathematics or science or something, they'd be great. Right, because most students, like if you're a basketball player or wanna be, so to speak, you don't stop practicing after you've just missed 20 free throws in a row. It's like, no, I'm not getting it. You keep practicing until you get it. And then you come back several days later and you do it again. And really what practicing a sport, dance, music or everything is really a lot about retrieval practice in a distributed fashion, right? Why that doesn't translate to studies skills or at least for most students, I don't know. You'd think it's like, oh, why don't I just use these same techniques now to get ready for my math test? Yet what students do is, of course, wait to the last moment and they cram. Maybe mathematics isn't quite as fun as playing basketball. I don't know. Some of us like math. But yet it just doesn't seem to translate and we're trying to get the word out, let it translate. Start many weeks early, right? Keep practicing, getting ready for that exam so you can really do well, so to speak. So I think the opposite would hold true as well that cramming for a basketball. Probably not gonna do well, right? Yeah, wouldn't exactly do, wouldn't get you very far. No, not at all. You don't wanna pull an all-nighter that the evening before a basketball game, you'll be tired and you won't play well. Practicing free throws. Yeah, it's like it's just not gonna work. Interesting. My name is John. I think about reflection.