 That distinction between performance and learning that Bob mentioned I think is really important I mean we like to think we know how to learn how we learn best in what Situations we learn best and that we know when we've learned something, but as we saw in episode 3 We have no real insight into into the Into the basis of our behavior and we it's really difficult to predict What we'll learn and whether we'll be able to retrieve this stuff in the future I mean, that's that's exactly right, but who better to provide An indication of how much we learned than ourselves right and again as you said we saw this in episode 3 We're not the best people to determine whether we understand something or not And it's we keep coming back to this idea of fluency or ease of processing System 1 is responding and system 2 is picking up on how easy it was for system 1 to do that and That's exactly what's happening here with learning if if something is easy if something goes down easily we misinterpret that ease that simplicity with knowledge with comprehension with understanding and that's That's a real problem, and it's it's paradoxical in a sense because people use that information To gauge how much they know about a particular concept and I have a really good example of this It was back quite a few years ago I was teaching an advanced statistics course for honors students, and it was compulsory all the students required to take this course a fourth-year honors level course called advanced multivariate statistics and students were understandably very worried very anxious about taking this kind of difficult topic and it was difficult and I worked hard I really worked hard to try to do everything I could to make it easier for the students to understand this material But at the time I didn't really I hadn't really read much in this area And so I didn't understand the importance of retrieval and the importance of desirable difficulties So at the time I gave I prepared this lecture. It was a very difficult lecture on eigenvalues and eigenvectors, and I Did everything I could to make it easier on the students for example? I represented it in a whole bunch of different ways, so you have you can represent this in matrix notation and algebraically geometrically, I did I had diagrams I had I was essentially Pre-processing digesting all of this stuff for the students So all they had to do was just sit back and kind of watch it unfold Now they did that and I had one student who came up to me after the lecture And she said Jason, you know, I I completely understand I get it It feels so much better now that you've presented it to me I understand everything about eigenvectors and eigenvalues and I thought to myself well There's no way there's no way that you could I struggled he waits to Stuff to get out exactly and in preparing it you have to sit down and work at it do a few problems and and try to explain it to yourself and She didn't do that and sure enough when the exam came around She didn't do that. Well, she clearly didn't understand it as well nearly as well as she thought that she did Yep The same thing happens to me when I'm watching a documentary on a topic that I don't know much about so when I'm watching a Beautiful BBC documentary by Brian Cox on the wonders of the universe, right? So he's talking about a complex topic like entropy and he goes on and he's using these beautiful examples with you know Sand and talking about time and how it relates to basic physics and how we perceive time It just it looks fantastic and I'm like wow. I've never understood this so well I guarantee if somebody paused that TV show and asked me one thing about entropy I would not be able to tell them at all that I was fooled by That that feeling of fluency it felt really good But I actually didn't understand the material as well as I thought that's really important to have these checks to have and that's exactly what this sort of retrieval Does when you're actually forced to generate Entropy so if you pause the film and said what is entropy and forced you to say well, it's Yeah, I really have no idea exactly what it is But it went down easily then then I think we're getting closer But there are a bunch of strategies that we can use a bunch of these sort of desirable difficulties. Yes, they are hard, but One is retrieval retrieval practice another is spacing putting it time and time again another one that we didn't talk about is this idea of Interleaving and so what you can do if you're studying study statistics for a bit followed by biology do a bit of English and Present these things in kind of an interleaved format. So you're not just studying Statistics in a five-hour block. You're kind of spreading it across So you're kind of getting spacing in there as well, but it's this idea of interleaving Of doing one thing for a while and switching to another task. That is also a desirable difficulty There are a few others that you can do to make it just a little restructuring your notes in a particular way organizing it by theme instead of by topic which and textbook Producers do this all of the time, right? They present it in exactly the the most logical format They highlight the important points for you. They're doing exactly what I did in my stats course. They're pre-digesting all of this Information for the students so they don't have to so they can just let it wash over them and they mistake that fluency With understanding and that's a real problem. So students in this course should probably be careful as well because I mean We're not professionals, but we've tried as hard as we can to make these videos look as good as they can We've tried really hard to make the online experience as best as it can possibly be But be careful not to be fooled by but by that that feeling of fluency instead Pause the video spend some time working actively working Recalling this information and going through that struggle to make sure that you really Understand it and that you're really going to be able to retrieve this stuff accurately for months and years to come Well, we've done a few things in think 101. We're not just Producing this stuff for people to kind of Absorb and then have them wash over it exactly We're actually practicing what it is that we're preaching here And so in the next segment We're gonna talk about exactly some of the things that we've done to take advantage of these desirable difficulties To to try to help the learners in think 101 Retain the content for a much longer period of time