 We are at the top of the hour and this is the Future Transform welcome everybody. My name is Brian Alexander I'll be your host and guide to the next hour I'm the creator of the Future Transform and I'll be happy to introduce it I'm absolutely delighted to welcome Justin Reich Justin is a professor at MIT Where and he is the author of a really really important new book failure to disrupt a very very powerful book about Education and technology and that's gonna be the subject of our conversation today If you'd like to learn more about the book if you don't have a copy if you like to grab one directly Look at the bottom left corner of the screen. You'll see a kind of tan colored button That says failure to disrupt press that take you to the page and you can grab a copy And now let me bring Justin up on stage Welcome sir Very well, it's good to see you Thanks for having me Brian. Well a pleasure and I'm glad to see your beard is growing every day. Yep That's right. It's a key part of the forum. We're Openly biased towards beards. That's an important thing Where are you coming to us from today? Are you in Cambridge? No, I'm in Barnard, Vermont everybody who does MIT tried to de-densify very early on like everywhere else and so the people who are on campus do things with fume hoods and nuclear Reactors and wind towels and those of us who just do stuff with computers were told to bug off when we did So I'm actually in Barnard, Vermont, which is a town of about 800 people right in the middle of the state. That's a big town How how much snow is on the ground? finally zero We had our We had our thesis day maybe a week ago Friday and it was the last big storm of the season with about a foot of snow So I just turned zoom on my phone and hopped on in the mountain and skied down and listened to graduate student thesis presentations Everyone's in a while. I have to ski to the side to safely have to ask a question and then That is Vermont story On the internet, no one knows you're a skier Did you get any good selfies? I didn't maintain a very low profile see that'd be your next book check at cover the You there are a lot of ways that people in academia can introduce themselves and the way we do it here in the forum is We ask you to say what you're going to be working on for the next year So what are you going to be writing? What are you going to be researching? What are you going to be teaching? And what are you thinking about? There are a lot of answers to those questions because I'm lucky to work with the big team interdisciplinary team of cool folks But the main thing we're working on Is that as we've looked at the sort of instructional design palette that online educators have We feel like there's a real missing piece around simulation and practice This is particularly acute in teacher education, which is my main area of interest When teachers learn they listen to people talk about teaching and they talk with each other about teaching But they very rarely do teaching Um, so we built a tool called teacher moments Which you can look at at teacher moments dot mit dot edu And uh, it's a digital clinical simulation platform that immerses people in vignettes of classroom life Through text audio and video And then periodically a microphone icon pops up or a text box pops up And you have to say what you would say to a person in that moment and make some difficult decision teaching And there's all kinds of data capture that lets us, you know Learn about how people are authoring scenarios how they're participating in the platform We just added some functionality out of sort of intelligent coaching agents come in So it was sort of our effort to say okay Here was here was what we think was one missing building block in the instructional design palette that we have Let's see if we can try to to build something that fills that in And make it available, you know under an open source license So a big part of what we're doing now is that we have a community of practice It may be a hundred people in various places that have started to to adopt this tool And we're trying to figure out how to how to support them and have that to grow in scale It's fantastic. I hope everyone gets a chance to play with this And rocks and thank you for uh, I'll put that in the link. Do you uh, what's behind the All the content. Do you have an algorithm that just randomly generates this or do you have a kind of imperative structure It's all authored. Um, so the so the back end of teacher moments. It's like a combination between google slides and qualtrics um, in in fact You know, we might not have built it Except neither of those platforms has really good audio input That turns out to be like the killer app or the killer feature You know what what we want is you're you're in some scenario And a kid calls you a racist and we want the microphone icon to pop up And we want you to say not we don't want you to scribe what you would do in that circumstance We want you to actually use the words that you would say and have them recorded and hear them back We have we have some colleagues um in neuroscience medicine the a yell professor who runs the national neuroscience education center and uh, apparently there are some Drugs that can be prescribed for opioid abuse which are dramatically under prescribed It's the same kind of thing like don't tell us what you would say the exact words that you would say to a patient suffering from opioid Disorder when you're going to suggest to this patient that you're going to prescribe this particular medication that has the stigmas associated with it and stuff like that but anyway, the I I got the rail from your question, which is You know the way the content appears Is that we make partnerships with folks? One of our partnerships is with 15 computer science teacher educators across the country people who are starting new programs to help Middle and high school teachers be licensed as computer science teachers And they think to themselves like what are the situations that I want my students to be able to practice in a simulation before they practice them in real life And then the tools are very very simple to author And so You know there's a bunch of there are a bunch of these scenarios that we make but mostly people make scenarios for their own local context And you know a big part of some of this will connect into maybe some stuff we talk about later There are a whole bunch of people who who've built simulation tools And they're like, oh, we should build it with a vr headset and or in augmented reality or in all these sort of complicated ways And we want people to be fully immersed in everything that's happening and teaching And there's potentially some good reasons to do that, but there are two big problems One is most institutions that train teachers Don't have tons of extra resources and don't have huge technical it capacities So our thought is why don't we build things that like people can author on a laptop and do on a phone? You know, let's let's let's You know, basically bring the technology demands of the system down as low as possible As long as people still feel like they're having some kind of authentic experience And then the second is that when people make simulations, they often make things that try to get folks to practice the whole thing But it turns out that there's there's good cognitive science that suggests that when you ask novices to practice the whole thing It's actually very difficult for them to get better at individual components If you want people to get better at things have them practice parts of things This is why athletes don't do scrimmages all the time. They do drills This is why musicians don't just play the whole piece all the time. They do scales and repeat pieces and they do other kinds of skills So we're trying to build like low tech totally seamless front end experience totally simple front end experience that does these sort of Drill like simulations that might have tons of complexity on the back end that you're not exposed to And all kinds of interesting research capacity and things like that But the initial experience, you know is simple as straightforward is authorable is adaptable to different contexts And you know it does just as much as it needs to do Nice Nice, and this is just one project you're working on. I know. Yeah We have a big team That's fantastic. And are you working on the on your next book yet? You know, I'm not It was a lot to write one book. I I think I agreed to write the book in 2015. I told them I would do it in 18 months And I did the final proofs of march of 2020 The one the next book that I'm thinking the two books I'm thinking two kinds of projects. I'm thinking about One is in my lab. We make a bunch of online courses I don't think it would be very hard to turn some of them to books because we put a lot of work into them already like with peter sangly I wrote this we made this course called Launching innovation in schools There's a lot of good stuff that's embedded in there and I bet people would read it as a book You know, I sort of take a bunch of transcripts and sticks them together the second project There's a little bit of the second project sort of branches out the book a little bit, which is that When I teach I teach a bunch of classes No one is an education student that I teach People often don't know much about education and learning and usually the whole course is not just a primer of education and learning So it's learning technique, you know learning media technology introduction of media studies You know technology design workshop. So so most courses have something that that starts with like a brief introduction to learning science And they're basically just two ideas like to a first approximation. There are two ideas in learning science There are people who think that you learn by pouring stuff in their head And there are people who think that you learn through apprenticeship like practice experiences, you know in in the 19th century in the United States these sort of You know positions were epitomized by Edward Thorndike and John Dewey But they're also captured like when Plutarch says something along the lines of Education is not the filling of a pale but a lighting of a plane There are a bunch of people who like no we have good science. It is also like it is filling of a pale Or it is also like filling of a pale So I think sort of explicating those two positions is interesting But I also think those two positions are are not just grounded in cognitive science But they have a sort of moral philosophical dimension to them You know, so Rousseau who had these ideas about You know natural liberties and the noble savage and the dignity of primitivism and things like that You know was a big fan of apprenticeship kinds of models and You know folks folks who like Sort of pale filling learning they tend to really celebrate how learning is difficult And folks who like sort of natural Apprenticeship learning like to celebrate how learning is easy And of course, I'm sure everyone who's listening can think of a moment in which they learn something effortlessly and naturally You know the way they they they sat next to their grandmother and learned her recipe for making tamales And it was it was pleasant and it was enjoyable and it was seamless and they could think of other things They could learn, you know oftentimes in math class or something like that where it required practice and revision and drafts and was painful and uh, you know, there's there's there's those those kind of like Those affective dimensions get kind of woven into what are nominally sort of You know arguments about the structure of the brain and cognitive science and things like that So that's another project that i'm thinking of trying to trying to weave together, you know How these two perspectives are not just pedagogical perspectives, but they're they're philosophical they're effective they're Um, they're multi-dimensional Quite extensive. Yeah. Yeah. Yeah, I mean, I don't know them as well in other kinds of cultures but uh, you know, they uh, um, you know, particularly in the west these two ideas run very very deep I'd go back to uh frankenstein and take a look at the education of the monster as well as the education of um, but friends, I've I've got a couple of questions, uh for an extremely energetic guest To ask him about about his book and where he's headed But then I'd love to hear more from uh from you and we already have a couple of questions, but I just want to begin what That it's the uh the technology But not transform it Or that it's just it never scales up Uh, and you have a whole series the second half of your book is about all the all the reasons why the curse of data The familiar the problem the familiar and so on Um, I'm curious when you started working on this, uh, was that where you began or did your view get Darker and darker as you again Um, well, I I don't I don't my view isn't dark to me. Um, it's uh That technology has a role to play in improving schools Um There are two ways of thinking about that improvement there may be more but there are at least two one And my colleague morgan aims is a great book called um the charisma machine about one laptop per shot And she defines this view of technology Um as a sort of charismatic technology and charismatic technologists which says new technologies will come and they will sweep away the future And they will they will break down the errors and failings of the past and they will lead us into uh into a very different kind of place And then There are also people who are more like tinkers who say no, no, no, that's very silly Um, educational institutions are incredibly complex They balance this incredible diversity of competing incentives and stakeholders and needs they're in a constant Delicate political balancing act, you know our school systems k through 20 They teach people to factor polynomials and to tie their shoes and pay their taxes and to get a job and to throw a baseball And to criticize the government and to love your country and to not have sex But if you do have sex to do in certain kinds of ways and on and on and on and all these kinds of things that we ask schools to do um And so it's silly to think that you're going to build something That's going to make a huge improvement across all of that because when you tug too hard on one part of that system All the other parts of the system go wait, wait, wait, you're screwing up our parts of the system This was all delicately balanced to meet all these competing needs But that's not to say that the system can't be improved It's very unlikely to be improved simply through the introductions of new technologies Because every technology solution is a human capital problem Every time you introduce some new tool or resource It's not really useful for reasoning there into until people have the time to develop new routines and pedagogies new supports and not just teachers Or faculty instructors, but students and families and community members and it staff and custodians and you know The whole range of of people that make these systems work So to me, it's not dark in the sense of I just think our Our celebration and our optimism is misplaced. I I think I think the charismatic have just been wrong over and over and over again and It's not it's not neutral They architect these boom and bust cycles that have institutions over invest in new ideas And then they get you know, then the charismatic skip board once they're not the saviors anymore and for the most part they leave We're not all in do Some of them turn into tankers And you know those systems are left behind my my mit students right now Our members of the smart board generation every year students come into my class I asked them tell me tell me the story of the technology that you remember from your schools And we've sort of finally hit the wave of students where their districts But smart boards in middle school and high school And nobody ever used them and they sat in closets or they just use as whiteboards And there's millions and millions of dollars It could have been done set productively to all kinds of other things sort of flushed away Which you know remain only as a memory in our students minds But I'm quite optimistic that when people bring a kind of shoulder to the wheel mentality you say look human development is hard Um the systems we work in are complex. They can be improved. They they get better technology Can be one thing that helps them get better of almost never technology alone But this is good work to be doing so so for me It's not so it's not so much painting a dark future. It's painting a cautionary tale About charismatic technologists who seem to be wrong over and over and over again And trying to really explain in detailed ways why And then give readers, you know both the motivation to be tinkers And then a set of tools that I think are useful for tinkers and doing their work of making change Well, thank you. That's an excellent excellent answer And I hope well already we're getting some responses and I'd like to Put these out for everybody Just really quickly Fritz van Dover says that charismatic technology seems to be brittle You get some older folks with older technologies mentioned here ditto machines and microfiche for example But let me put up a couple of the first questions that we've got This is one directly into a certain part of your book This is from Jeff Alderson at mathworks. Hello, Jeff. Jeff asks I have a question for you about auto graders and computer programming What do you recommend in lieu of auto graders to assess students? Sure, so our graders are pretty important to a bunch of models of learning at scale because scaling Scaling the dissemination of knowledge is not that hard We could record this talk anybody could watch it And they do and they do but if people wanted to learn they would probably want some feedback on their comprehension and understanding They want some way of knowing that what they've taken from from the discussion Is being interpreted correctly or being reinterpreted in meaningful ways and so you gotta give learners some feedback There are some domains in which we're pretty good at giving learners feedback The problem is that those domains are things that computers are already good at so When we ask humans to act like computers when we ask them to do sort of routine Highly structured kinds of tasks computers can evaluate them pretty well They can evaluate computation and math They can evaluate the conjugation of verbs the pronunciation of words in early language acquisition We don't have good auto graders that can assess whether or not You understand complex texts in a foreign language We can tell whether or not you've pronounced poor for work correctly We can't tell whether or not you've had some meaningful interaction with Cervantes d'Antio de So And also, you know one of the I call this the trap of routine assessment Which is that technologies are good at assessing things that we don't need human beings to do anymore And technologies are not particularly good at assessing the things where humans have a comparative advantage where our education systems should be focused on So You know I I hope I you know I hope that sort of argument does two things One is I hope splashes a whole bunch of cold water on the notion that like right around the corner Are these extremely powerful AI driven Auto graders that are going to be able to reliably evaluate human performance in all kinds of different domains Very very smart people at wonderful institutions with millions and millions of dollars and the support of foundations And governments and the departments have worked on this problem for many years and have not made much progress The essay auto graders that you know Sort of I first encountered 10 years ago 20 years ago The ones we have right now are like a little bit better Like you know in the way that Grammarly is better than you know the 1995 microsoft word grammar check But not better in the sense that we still don't have machines that can tell whether or not a sentence that you've written You know conveys meaning in a powerful way Now you know however all that said we have auto graders that are good at some stuff You know lots and lots of young kids and probably lots of College students too though, I know a little bit less about this particular application Have used some kind of math tutoring software this year They haven't been able to go to physical buildings. They've had less access to their human teachers And fortunately people have made stuff like you know alex and dreambox and st math and all these other kinds of things Um, and we can ask students to do computation kinds of things. Um, and the computers can go Yep, you've got that right or no, you've got that wrong sometimes they can even go No, you got that wrong because you don't understand this thing go look here There's lots of things though in math that they can't evaluate really well You know one of the things one of the things we most want people to learn as they study mathematics throughout their career Is to be able to articulate why they chose to solve a problem in a certain way to be able to explain What the solution to a computation means in the context of the real world, you know Was it 37 kittens or 37 board feet or you know 37 hot dogs We have much less good tools for that. But if you think about again, what mathematicians actually do Um, you know, I I do this much less, but I still periodically, you know, conduct statistical analyses and write papers and things like that The computer does all the computation for me. I figure out what problem is interesting I figure out how to set up and frame the problem. I explain what the computations mean and what their significance is Um, but you know, I let the computers do the computer stuff um So there's you know, there's going to be a role for auto graders The thing to avoid is two things to avoid are being seduced into believing that the things that we can auto grade are the important things for people to learn Um, in many cases, they're good building blocks, but they're not the whole field I'll give you a second example, which some of you may enjoy We have really probably the best auto graders that we have are in computer programming So I could ask you brian to write a computer program that performs a task And then I could write a computer program that could evaluate whether or not you've met engineering requirements Whether or not the the syntax and formatting is right. Um, certain kinds of errors there's a professor at um At mit of computer science named howl abelson sure who wrote a beautiful line, which is something like computer programs are communications between humans about methodology that only incidentally can be run by computers um If you believe that to be the case about computer programs, then our auto graders don't measure that at all You know What what howl is talking about is like art You know, are the structures that you're choosing the way that you're naming variables the way that you're commenting code Um, are you really communicating something about methodology to another person? Um, and so, you know, even the domains which are auto graders are best We're still not teaching people the thing We're still not evaluating the things that people need to be a really great software engineer a really great collaborator on a software project um But Like these things are helpful You know, like the technologies that we build are useful for Certain parts of certain subjects for certain students in certain contexts And we should use them wherever they're useful and we should not be seduced into believing that they're you know sweeping solutions Hi brian Hi Jeff Hi, Justin. So first off, I don't know if you can see this. I've got your book. It's literally in my hand. So I'm not bluffing Um, but but I know but as brian knows I've been I've been asked to join the stage a couple of times Um, actually I'm looking at us now. I think I'm the covet after six months. Justin, you're the covet after one year brian You're the covet after 10 years god forbid um, so So I'm happy to join I've been joined the beard panel today. So let me give you the context for why I asked the question I did so I'm I'm currently the product manager for matlab grader, which is an auto grading solution for matlab code at mathworks MIT is our one of our largest users of that auto grading solution your colleagues swear by this product in their in their classes and I'm also enrolled as a graduate student full-time at university of illinois for my masters of education And i'm focusing on learning technologies and when I was in a discussion in my class someone says, hey, justin You know right has this great book That's challenging some of the assumptions that you're making about your product management And I said, uh-oh And so I read the book And I'm in the chapter and I get to this heading and where you're literally put me and my work in the crosshairs but the good news is I actually Want to when I give you an open invitation the next iteration you do on this book. I would love to invite you to Talk with the folks at MIT. They're using our auto grading solution Because I think we actually are on to something and I want to specifically focus on What you said, which was the other stuff that auto graders aren't Evaluating but have to do with the intent have to do with the learning objective What was the behavioral things you're trying to check? The the skill the mastery of skills about computer programming and software engineering That can't be just gleaned from the syntax But they could maybe be implied by the syntax and need follow-up and discussion And and so I guess the way I was the reason I asked you the question is in on top of auto grading Is there another potential for automation or something at scale that could get at the heart of that? What's what's missing and as a product manager? What should I be focused on to? Take the next step beyond auto grading and look to Automatically assess the behavioral intent and the learning objectives Yeah, um, that that's a great question Um, you know, and I and I hope that one thing that people take away from the from the book You know two things one is that they're all kinds of good uses of auto graders um, there are you know, one thing I focus on a bunch is uh Computational assessment of pronunciation for language learning, you know I mean I had this sort of vision in my head of when you know, a Spanish teacher would go around the room And we would all say por favor You know in the Spanish teacher would like tap me on the head and be like, I don't think so But would move on, you know, because she doesn't want to spend Five or ten minutes just having you pronounce it slightly more right over and over again But computers will do that We'll just sit there and listen to you over and over again mispronounce por favor, you know coach you again kind of infinitely And I also think that instructional designers can do what I would sound or project managers can do what it sounds like You're doing which is continuing to think about How are we going to expand the domains of what it's possible to assess? And you know the example that I use for that in the book is some colleagues in mit Who are getting ready to teach a calculus class and they said we're not going to teach this class until students can draw curves It's just too important to be able to you know to to do the sort of conceptual work that's involved in You know in drawing curves and where they cross the excesses and y-axis and things like that And we could do with multiple choice questions, but that's not good enough. We have to be able to draw curves When they built that tool that drew curves, which took a lot of time and was was complex But worked very well like they sort of slightly expanded They sort of you know, that's kind of a way through the trap of routine assessment They kind of slightly expanded What was possible, you know, you've got a big team full of smart engineers at math works I'm sure day after day or sort of slightly expanding the domain Of what's possible And and pushing, you know, every every assessment You by definition is sampling from a domain Um, and you're making that sample sort of brought you know My argument is that the things that we're sampling are often not the most important things And you're continuing to try to expand that sample, you know For someone who's sort of looking at the field of of education technology broadly Those expansions are unlikely to help students You know who are who are in In a writing class or in a law class or you know in a poetry class or in a biology class They're just very particularly like your team is going to you know, spend gazillions of dollars and do tons of great work And you're going to keep sort of slowly expanding what's possible You know, it's unlikely that you're going to develop the sort of generalized ai that solves all these problems for everything All of that is great and worthy work You know, I think of that as sort of great tinkering work And the important thing for institutions to do is to keep auditing where that Where that where that sample from the domain is and to not be lulled into thinking that the sample is the whole domain Is to say look if these are things that that the math works on a greater doesn't evaluate right now Where in our program are we teaching these really important things to students? And you know, how are we you know paying people fairly at reasonable wages to be able to help other students learn that stuff You know, I I I am optimistic about the notion that There is more time for educators to focus on the things that they do best if you're building us some auto graders that That captures some things that computers happen to be good at doing and even You know, I mean MIT has also adopted. I think it's grade scope or something like that which Pass an animal to each other through the instrument. We can do this through that I just I just saw Brian holding at one cat and it disappeared. That's how Jeff pick up another cat. Um, it'll look very much the same so, uh You know, it's a it's a tool that it was built for huge classes And it just lets you sort of stand in test answers so people can grade them a little bit more easily It's like those are all good things to do and we just have to we can't confuse those like pretty good tinkering incremental improvements For the disruption of higher education Yeah, I I appreciate that context and it's one thing we actually ask our customers to help us with is to Prove that our auto grading solution is as good or better than what they used to do to assess the same learning objectives and outcomes and if we're able to In a statistically valid way show that our auto grading solution is doing the same things they used to do but at scale Then and only then will we move on to the next thing? Right, like we we want to take those incremental improvements But we don't want to like jump into such new territory that they've never tried to assess before that's not You know, we can't auto grade some new concept that they've never tried to even understand Behaviorally or from a learning objective or a pre-work as a skills perspective We don't we don't want to go into those territories. We want to show that we can grade things Faster more officially than that and improve that it works as good as what they used to do Um, you know, I appreciate I appreciate that context and thank you for the the answers in the time No, yeah, thanks and if folks at math works when I talk about this stuff more I'd be happy to Thank you, Jeff for the really good problem And uh, Justin, thank you for the very very rich answer Friends if you're new to the forum, that's an example of a video question. It really is that easy to do We just now share some of the test questions as well And let's see one comes from another close reader This is from Kate Montgomery at SMU and Kate asks You say that institutions and investors often favor programs to scale up quickly but at the expense of true innovation How is true innovation defined? I'll put that up on the screen again so people can see it I don't If you're going directly to the book, I don't remember that sentence anymore Um, here's here's what I hope it means but then someone can go back to the text and tell me if I did it wrong. Um you know, one of the phenomena I talk about is uh Is I call the cursive familiar? um and the uh The you know one way to frame the curse of familiars If you look at the history of education technology The thing that people do most commonly is they use new technologies to extend existing practices So they use new technology to do whatever it is They were doing before like a little bit faster maybe a little bit more efficiently If you look at the technologies and most widely adopted, you know, one example I use is in pay 12 One of the most widely used technologies by like half of all high school students I'm sure it's used by lots of higher education students too Is this tool called quizlet, which was made by a by a mit dropout or an mit You know industrial graduate or whatever you want to call that You know, and if I most of you probably know what quizlet is if you don't it makes digital flashcards It has you know, as soon as you glance at it, you'll know exactly what it is the flashcard app You'll have questions on one side answers on the ever terms on one side definitions on the other you practice Adopted very very widely very very quickly Um Because it let people do a kind of thing they were doing before If we grabbed a whole bunch of educational experts and pulled them together around a table and said like What do we really need in k12 and higher education? What do we need for the future? What do we really need to move things forward if you if you had not if you had 60 years of Future trends forum I'm pretty sure that not a single expert would raise their hand and go. You know what we have a real flashcard deficit in our schools You know the thing that we're up against is there there are not enough index cards We can't write definitions on them fast enough. This is what's really holding us back So I I think what I mean By true innovation in that context Is that the tools that most readily spread and scale are the things that let us do what it is that we're already doing But the but the trap of that is if we simply do flashcards a little faster and memorization a little faster That's not going to reduce the yawning gaps and equity that we have that's not going to you know help our six-year graduation rates that much That's not going to you know rearrange the iron triangle of cost access and quality By contrast When we build technologies that seem like wow, this could be a really different way of teaching and learning Like there could be some real advantages that would prepare people for the world in the ways that's you know Universities anchored in the medieval times might not be able to most people find them confusing You know when you looked at like the original massive open online courses that came out in 2008 that sort of borrowed from these network learning environments Super cool super interesting Really exciting really neat technologies and most people look them and like what am I supposed to do here? This is weird And we're really hard for people to persist in the k12 space You know a similar sort of phenomenon happens around the scratch programming language Which resnick and his colleagues at mit Build this incredible new community this incredible new programming language for creative computing That asks for really different relationships between teachers and students really different use of time really different ways of evaluation And so most schools if they do adopt straps, they're like all right We're not going to do that part actually we're just going to give students a recipe and we're going to have them all make the same computer program So You know that that's that's the curse of the familiar It seems it seems to be difficult to navigate between what kate reference You know just generating things that do what we're already doing a little bit faster That probably won't make that much of a difference or generating really new things that are hard for people to adopt The pathway through by the way, there seem to be two parts of it as far as I can tell Building things that start in the familiar and then lead you to really new kinds of practices Desmos is an online graphing calculator that I think has done that really well When you first encounter it you're like, oh, this does everything my ti 82 does but for free And then as you get into it more you're like this is a completely different way for students to learn modeling and math There's sort of a pathway for the familiar to something More ambitious. Um, and then it's not about scaling distribution. It's about scaling community If you have a thing that's going to help people do teaching and learning really differently They're not going to learn those strategies just from your technology They're going to learn those strategies from being in community with other people Now everyone's going to be graphing way And and the questions will drop. So I have to be careful here That's a That's a really good distinction between the strange and the familiar here We have more questions come in that build on that's uh, one I think comes from keeldeutsch It's most colleges just Reused classroom lecture model for online learning and they ignore the real value of the internet It's rich trove of books articles videos podcasts That's not a question. That's a comment Yeah, I think that I think the challenge of that is that um If you treat the internet just as a rich trove Even in some of the way that like mooks and other things have done like look, here's a course come and take it It's you know, the same thing that we offered at mit or harvard or whatever other great university. You think it is Most people are really struggle with self-directed learning Maybe one way to qualify that is when people learn things that have an Immediately that are immediate concern to them and are of high interest. They're actually incredibly good at learning online so if you want to learn new ways to style your hair to do your makeup to Do a trick on a skateboard to beat a level in video game People from all kinds of different backgrounds all kinds of walks of life All kinds of ages are quite proficient at that kind of learning When we ask people to do things that look like school Where you sort of have to like grind through some stuff that seems kind of abstract for a while for a longer payoff Many folks are not nearly as good at doing that so You know And the people who are really good at that tend to be people who have had a great apprenticeship in the formal education system You know, I don't I won't I won't say I notice for sure from science But if you want my current best bet the best way to become a really good autodidact Is to have a really good apprenticeship in the formal education system Which means that if we treat the internet as a rich trove of resources The people who will benefit from it most are already educated already affluent folks So we do need models That recognize that people don't just need a rich trove. They need supports They need peers and teachers and structure They need guidance some of which traditional lecture kind of courses offer But certainly I think more of our higher education institutions should be Should be deliberate about building pathways, you know By the time you leave this program you shouldn't have just learned some stuff You should feel like you've had some practice with some some mentorship and support at learning more about this stuff on your own That's a good answer And it makes me think of quite a few things involving MOOCs, especially the x MOOCs that you address so well We're building on the autodidact aspect Let's bring up a question from adib sayid Who asked what are your thoughts on alternative education such as homeschooling Unschooling and we just heard self-directed education, etc Well, you know an amazing thing that we learned during the pandemic Is there are a lot of there are a lot more students who might be interested in some of these approaches than maybe we thought In particular Amongst our most vulnerable students, you know, there there are lots of reports But you know, this hasn't gotten the quantitative science I don't think but but there's lots of really strong journalistic and qualitative reports that say You know in k12 There's a bunch of students who went to schools every day where they experienced a bunch of racism And then they didn't have to go into those buildings anymore and they really liked learning better You know in the loving arms of their homes and their families but the you know These homeschooling independent learning self-directed like all you know all that stuff is good Our our public education systems are a gem and a treasure and you know And maybe we should supplement them with these more distributed options But there's something enormously powerful about having places where people go to build community and learn together You know, even if a lot of that building, you know, even for higher education institutions that are moving more online You know, I do I do think we want a kind of federated system that has different options for different people because different folks are different Um, but but I'm but I'm not at all enthusiastic to some extent. I've never been enthusiastic about these sort of like highly These sort of models of highly personalized learning where we're going to algorithmically optimize each student's individual pathway through learning experiences In part because I got my career started as a ninth grade history teacher And there, you know, what I wanted more than anything else was kids With as different brains as possible and you have there are lots of ways of getting different brains But having different backgrounds and different life experiences is one great way of having different brains Having different brains read the same stuff At the same time and be in the same space with each other to talk about them You know, I want them to read the mayflower compact or the letter from a birmingham jail And just sit with each other and wrestle with what it might have meant to the people at that time What it means to them now what it means differently to each other because we have different life experiences And there's no algorithmic way to optimize through that faster It's uh, you know, one way to think about it is that there's some parts of Learning which might feel like, you know, the joy of learning is maybe like a jazz musician or something like that Where you're kind of doing your own thing Maybe doing your own thing with a couple of other people But a lot of the learning experiences we value most are like an orchestra Um, and even if the second violin has mastered their part You still just sit there and you play your part over and over again So the tuba can get it the trombone can get it and whoever else needs it can get it Because you know, the transcendent experience is when you all play that together And uh, you know, um, so that that's that's where some of my concerns about Some of you know, I did has a great point in the chat that like many homeschoolers and unschoolers create vibrant local communities That are more natural than the weird things that we do. I think that's right I think there's a lot that we can learn from those institutions. There's sort of cross pollination But I also think that some of the most vociferous advocates Of these kinds of things are also sort of advocates of free market approaches to learning Who you see online learning who see homeschooling who see unschooling You see virtual schooling as ways of breaking the social contract that we've had for you know, in various ways higher education for 16 years and Um 150 years and k-12 education of communities Educating people together Well speaking of time we're coming close to the end of our hour and I want to make sure that everyone gets a chance to Way in with their questions and comments So now if you have one that's burning up in your mind friends, this is a great time to put it in We have one from Michael fried at the ethical s and r and they just bring this up Is there a risk of veering the curriculum or course content to other technology is good at supporting its scale Instead of what is important to learn If so, how can that risk be mitigated? Yes, that is that is an excellent summary if you turn the first turn the first question into a statement. Yes We over anchor on what we can assess on what we can deliver online rather than what's really important The answer to that is to have as our ongoing mantra over and over and over again That technology needs to be in the service of learning that we think about Learning first and where we want people to end up and then we look at the technologies that we have And if we're educators, we say which of these helps meet our learning goals If we're technology developers, we say what are really important learning goals That whose needs aren't being met by technologies and let's try to build those kinds of things Financially that's actually quite difficult to do because if there's something for which You know it's not happening as well as it should have it probably doesn't have a particularly big market Because that needs to be sort of generated at the same time but You know I um Yeah, the way we do it is by is by having Learning be in the service by having technology being the service of learning the one of the challenges with that is for whatever reason Technology like takes up a ton of logistical brain space in people's minds You can start it You can start a conversation where learning and technology are in balance and all of a sudden you're finding yourself talking about like Software licenses and logins and single sign-on and where the chargers go And all these other kind of logistical questions You just found sort of further and further down this path and then you kind of look around and be like How do we stop talking about learning? How do we you know institutionally? How do we totally lose track of where we are? And it's very very common in all kind of schools. They're extremely well resourced schools. They're poorly resourced Elementary schools higher education institutions. It's a very very common pattern And so part of it is a discipline Of you know constantly asking ourselves the questions like you know, what are the learning goals? We have the students. What are the students goals that they have for themselves? What's the technology infrastructure that we're buying that we're building? You know, is it aligned with those goals and how do we prevent our so when I ever I think of this I think of the old farside cartoon of Um where the neanderthals in class and raises his hand and said excuse me, ma'am I'm going to go home. My brain is full Technology somehow does the brain is full in the chat carolin coward at jpl says that Chief finance technology takes up a lot of emotional brain space And if we furiously agree about that then we approve the point Thank you for that. It's a really good question. Um, and thank you again for that, just We have another question that comes off of the technology needle. Uh, this is from carolin How is the ed tech space combating hidden but systematic bias? And the algorithms drive many of our automated systems It's not it's not um, you know, one of the things that I don't know my students and I have just been reading The age of surveillance capitalism by uh, And one of the things that um that she says You know over and over again is that You know, it's it's it's in the interest of technology companies to make what they're doing is Tooth and abstract as possible Because they're trying to do things in places in which there aren't cultural norms or laws or regulations And so the more quietly they do it. Um, the deeper their incursions can go I'll tell you what we think the solution is. So as I mentioned to you before We have uh You know, we have a project to build this digital clinical simulation tool and we're actually building an artificially intelligent coaching agents in it You know things that say like recognize when you sound confused or hesitant and say, hey Brian, you sound a little hesitant there. Do you want to try that one again? Here are three sentence stems that might help Um Part of what we want to do we haven't gotten there yet But part of what we wanted to do in the build is also just like expose that whole process to people To be like, hey, remember those little nudges you got like, you know If you want to find out more about where they came from here's how we program them Here's how we tested them on them. How did it feel to be Supported by an intelligent agent. What concerns do you have? A lot, you know in a lot of consumer technologies. We want the technology experience to be seamless I just got a new electric car when I sit down in the electric car. Like I don't want I don't want to confront it You know starting to use my phone as a sound system. I just want it to work and go away I think in education we do want to trip over and encounter our learning technology systems I do think we want to have rougher experiences there where we go. Oh We are using ai How did it get trained? Who are these people? Why is it doing it to me? How do I have an opportunity to make input on my choices and things like that? So that I know that doesn't give you all the answers I saw a couple of great colleagues at the mit who are you know, not to not to be an advertisement for my institution today But um, who are trying to do more stuff in ai and education and help people especially young folks start understanding better how it works There's a there's a thing in uh in german called uh, I believe stolperstein them Which means stumble stones And this is when you have a cobblestone road and you take one of the cobbles You rotate it 90 degrees so that when you're walking along you hit it and you immediately look down And people have used it for ads eat at a hansel But also for history This happened in a place. No wonder about technology like that. But we have a metaphor. It's a great metaphor brian It's a it's a wonderful idea What does lead to bruise toes, which is also part of We have a one question from michelle denise miller who was a great guest. We also have a wonderful book Michelle says even as a fan of ed tech generally, I'm still constantly struck with a gap between a promise and the reality. What's actually That's one of the major reasons why this gap persists Uh, you know, I mean people fund promise and reality is hard um, you know, I think one of the things that I articulate in the book is that Though many of the promise reality gaps that we experience are not for lack of effort You know in in in mooks. I made this one of the promises of mooks was that we were going to generate a new data driven science of education that just like the genome project revolutionized medicine That uh, that mooks would be a kind of genome project You know that would that would dramatically change teaching and learning and has it and it's But not because the promise was totally crazy. It's kind of to me It's still on its face seems like a reasonable analogy and might have been possible And not because we didn't try I mean, I feel like I've worked with like literally hundreds of colleagues at dozens of institutions of millions and millions of dollars Trying to tackle this problem. Um, and I don't think we made a lot of progress because the challenge is hard Um, uh, that's what that's the thing I come back to over and over again You know, you think about the complexity of our institutions You think about the granularity of the work that we do like today Somewhere in this country. There's a seventh grade earth science teacher who is starting a unit on plate tectonics Let's say you want to make a technology that makes formative assessment a little better Well, what formative assessment on the first day of a unit on plate tectonics in the seventh grade looks like Is different than what formative assessment looks like in a spanish four class to a A graduating senior at a four year institution Is different than what it looks like to a computer science student who's doing their first lab practicum And it's different, you know with Like within each of our units within each of the days of our units that the granularity of the work that we're doing in Teaching and learning is so fine Um, then it's very hard to build things that are useful when you're trying to help people Learn how to tie their shoes and to have sex safely And to criticize the government and to conjugate arabic verbs and everything else that we ask people to do That's why it's hard, but each of those things that i've just described is just so immensely Valuable that they're all worth doing it too. You know, it's worth keeping our shoulders or we'll try to work on this stuff But we have to be realistic about what can be accomplished and we have to recognize that there's just a bunch of Financial incentives in the system for people to promise us the moon and it's not helpful when they do that It's not helpful when we believe them And so, you know, I I joined the ongoing Quixotic effort Trying to get my colleagues to You know, my colleagues in the tech development side to stop doing that my colleagues on the education side to Bring a healthy skepticism combined with the fakes that like if we tinker together we can totally make these systems better If we tinker together I I hate to pause you on this, but we just shot past the end of our hour. That's a fantastic note to end on sounds great. We're kind of like a tinkering and uh and a Marriage counselor at the same time What are the best ways for people to keep up with you? Is it uh, uh, you're on twitter or should we uh, I'm at bjfr on twitter if you go to the our website tsl.mit.edu. We send out a newsletter Um, okay, and uh to sign up there Those those those are those are the places that we that we publish stuff these I don't know you could sort of a google alert for my name, but That's not what I'll do. That's what I do. I wouldn't say what you all would do Well, that that works very well and some people have been tweeting at you during during this event and you probably get more More twitter questions and comments as we go. Thank you for an incredibly nuanced Deep and very inspiring discussion. I really thought it was a pleasure But don't go away from that as well. We're going to bring you back next time But let me tell you what's happening for the next few weeks. Um looking ahead We're going to continue our exploration of technology at academia again We're going to be looking at sparking emerging ed tech conversations How to improve equity and education for black students federal policy changes and a lot more so just go to forum the future of education dot us Now if you'd like to continue discussing these issues Honestly, I think right now twitter is the best one. Uh, we haven't seen a lot of people using our facebook or linkedin pages Um, so I may just sunset those but please use the hashtag If you'd like to look back into the past and to including some of the sessions that we've already evoked today, uh, please go to Our archive on youtube. You can find that at tiny url.com slash ftf archive And uh, as you proceed as you wrap up this spring semester as you think about the summer as you think about the fall I hope all of these discussions have been of use to you all I really appreciate all your comments all your questions today It's been a pleasure to think through all this together with you And in the meantime, stay safe. Take care and we'll see you online Bye