 the work model that we have that was in through the Harvard class, and so how we have come to develop that. And we, many of you, as you know, we've been teaching the Harvard class online since 2016. I've been teaching that course since 2014. But in 2016, they generously suggested, offered, slash insisted that we move our course 100% online because the demand of the students was actually to do that, and that we would be able to have a broader international audience if we were to do that. And so since 2016, they've pretty much handheld and just helped us figure out everything, you know, as far as the design is concerned. And when we did that, we asked if we could then begin teaching classes in Spanish, and they said, not for another 10 years. And so we basically said, shall we move this instructional design into a Moodle platform from campus? And they said, do whatever you want because basically you guys have come up with this design yourself, but they had helped us so much. So I wanted to share a worked model because, as many of you in education know, one of the best and fastest ways to learn is to have a worked model. If somebody says, hey, if it looks something like this, you're going to be OK. If you can see a model of what it's supposed to look like or what potentially it could look like, then it's much easier to get your head around what you have to do. So I wanted to share a worked model. But then I thought this morning, and this is where I got overly ambitious with our time frame today, I thought what would be of a lot of interest to many of you is what has really changed fundamentally about education due to COVID and what will never go back to the same again. And there are some things that, thankfully, have been changed related to the tools we have, the way we evaluate individuals, what we think is important or worth teaching, and also the way we manage time and space. And so I wanted to go into all of those different things. Now, I confess, totally, this is an overly ambitious agenda, but I will be stopping along the way to take questions at each one of these junctures. And if there is a space where you guys feel it's most important, no, I'm really only here for part one or no, I really hope you would get into part two or part three. Can you just put that into the chat? And Cynthia and Michelle, I've got their finger on the pulse here trying to make sure that I manage my time correctly where the interest is on your part. So thank you all for being here. And if you did want to say hello in the chat and let us know where your background is and why you particularly care about this, some of you are teaching online. Others of you are doing conferences as Anne was just pointing out to us. And other people just want to see what is trending as far as where all of education is now headed. And so if you do have a specific interest area, also share that, because that will also help us focus, okay? So basically throughout, most of you know our system here, just go ahead and raise your hand. If you have a question, we'll put it in the chat if you don't feel like talking out loud, but we will be attending to all of your questions as they come up, okay? And we want that to be a big part of our exchange, all right? Okay, so great. Anybody wanted to do something else? Anybody here for a different kind of a conference? Is this okay with you, Joe? Is this what you were expecting? You're all right with that? Okay, all right. In case you have anything else, go ahead and put it in the chat and we'll see if we can all fit that in as well, all right? The short answer, for those of you who say, okay, I just want to get the quick and dirty on this, basically my take on great instructional design, course design is a logical plan that has a clear trajectory, which is basically a hierarchy and a constructivist order that is aesthetically pleasing and intuitive to the learner. They can find stuff really easily, that offers a variety of resources that allow students to begin where they need to begin, not where the course says they should begin, but from their own starting points and facilitates their reaching the same objective. So that's the short answer. And that is why I broke this up into design elements, because basically, if you look at it, if it's a logical plan and it's a good trajectory and aesthetically pleasing way and intuitive order, that is basically figuring out backward design that is also integrating mastery learning concepts here with an idea of instructional design that is then leveraging the whole concept behind universal design for learning, okay? So that's why I wanted to begin by talking to you about design concepts, okay? So that's kind of the short vision of this as far as why design. I also just wanna give a heads up to anybody who's here who thinks that this is good for teaching MOOCs, it is not. What I'm gonna be talking about is a very big and different idea from just emergency remote learning or doing a MOOC or running a webinar. This is actually talking about high quality online courses that we feel have to last at least eight to 10 weeks in order to get a big shift in attitudes. We know that you can teach somebody really quickly, you can teach them knowledge, you might be able to teach them skills, but you're not gonna change attitudes in a short period of time. And so we do take into consideration different formats and some of them, MOOCs are great. Do things at your own time pace, but as one of my students pointed out in her paper, only about 10% of people actually finished them because you're not doing them with other people and you're not learning as a community. And so there's a lot of distractions there and it's not a webinar, a webinar is just to point out information in a short amount of time with no revisiting and no time for reflection. So what we're gonna talk about today has to do with the design of a high quality course, okay? A long version of this is in my book, the neuroscience of learning, it's sort of bringing the neuroscience of learning to online teaching. I do reveal that I have a great bias towards the brain. None of this is possible and none of this is justified in my mind if we cannot say why it's working in the brain. Why does the brain need to learn in this way? So we've made some choices here very much based on that core information that we had to know a little bit more about how the brain works in order to be able to create an instructional design. It is not only logical, but we understand why it actually is effective, okay? So here's the idea of design frameworks and there's a lot of different ways to think of design thinking. This is very much kind of a popular way to approach teaching and learning these days is through design thinking, but I'm gonna use these three design concepts that we mentioned before that has to do with instructional design, basically using backward design in a format that leverages universal design for learning because we know there's some really concrete problems people have. I know people are just sick of being in Zoom. Well, you have to create engaging environments or that we have other people say, oh no, I'm so tired of having this generic kind of instruction, one size fits all. Well, you have to learn how to personalize leveraging certain kinds of tools. There's new tools out there, right? Or, and people say that no, I'm a slave to the curriculum standards. Instead of understanding, if you teach for mastery, you will actually reach those standards and more in a much more individualized way, right? Or just thinking, you know, I'm so tired. Some of these courses that are online and one of my students pointed this out to me, it's just awful Tracy that people who are entertaining but have nothing to say and spread myths are much more popular than the people who have high quality information and are giving us significant learning tools. And that is so true. So how do we get around that? How do we actually get to significant learning? So all of these are questions of design. So I'm gonna stop right there. So just the basic framework then is a design framework. Does anybody have any questions about that up to the stage about why we're thinking about this through design? Does it sound pretty natural, like a natural way to go about it or just somebody having a hard time getting their head around it? Is that all right? We're good here. Okay, anything in the chat I need to see Cynthia? Nope, nothing yet. Okay, great, all right. So let's go to the first element of design and that is understanding by design. And this is a concept pretty much created by a really good friend of mine, Jay McTide who is he and Grant Wiggins stole this concept from business and basically saying, look at guys, we have to first figure out what is the objective? What do we want at the end of the day? Once you know what your objective is, second, you need to figure out how you're gonna measure whether or not you're achieving it. Once you figured out how to evaluate, then, and only then do you think about what to do. What are my activities? What resources do I need to leverage to do that? Okay, so understanding by design is only this very simple concept that you have to have very clear before anything else, the objectives. If you do not know what objectives you're aiming for, it's impossible to have a good instructional design. If you do not know what knowledge, skills and attitudes you want, you will not be able to get there. And that's the hard part. To get a clear objective, we insist that people disaggregate that into knowledge, skills, and attitudes. If you think about it, there is nothing, nothing you teach and nothing you learn that is not either knowledge, dates, facts, figures, Google-able stuff, right? Things you have to memorize, the structure of the quadratic formula, things like that, knowledge, skills, which is the ability to use the knowledge in meaningful contexts or attitudes, valuing things like the idea that, you know, you have a good idea, I have a good idea, but together we can have an even better idea, valuing teamwork, valuing perseverance, those kinds of things. Values are also a part of the things that we teach. And so the big idea of objectives is that you have to disaggregate your learning objectives into knowledge, skills, and attitudes in order to be able to then design around that, okay? So I'm gonna stop there for a second because this is a huge idea, sounds very simple, only three steps, but it's very hard for a lot of teachers to get their heads around this because you walk into a class and they're often doing something really interesting as an activity, but you ask them, why are you doing that? And they say, well, they just do the activity and they evaluate the activity. And I said, no, but what's the objective behind the activity? And if you cannot justify why you're doing something, you should really do a double think on that. There's a lot more to deciding how to teach by based on what we choose to teach, okay? So knowledge, skills, and attitudes. So once you've got that, this is the key here, design is shaped by those objectives depending on your objective. If your objective is simple factual knowledge, then go ahead and do a webinar or a lecture, okay? That's fine, that's a great format. If all you wanna do is transmit knowledge or information, okay? But if it's conceptual knowledge to get your head wrapped around multiple concepts, you have to have some kind of exchange there. There has to be some kind of live exchange. It's not just a matter of watching somebody else's video. You have to have the opportunity to have that given take in order to own that information eventually, that concept. If it's procedural knowledge, how to learn to do something, then you actually have to have that reflective practice. And that usually has to happen asynchronously when somebody is on their own at home trying to rehearse or practice or do homework to get better at something, okay? And if it's a metacognitive knowledge, if you really want people to get deep into their own thinking and to figure out how they are thinking about their thinking, I highly recommend you do something that's flipped, a flipped classroom which would give you much more time to be in the real classroom setting to get queries clarified, to get clarification, to get the actual application of the information rehearsed in a group setting so that you can get feedback in that sense, okay? So depending on your objective, you can use different formats of online learning, okay? So the key here is to figure out what is your real objective? And if your real objective, for example, in our course and in the Harvard course, it's the neuroscience of learning, it's to help people figure out how they learn better. You could never teach that course as a webinar or as a MOOC, you just could never do that because so much of what is gained is through the learning community and that exchange that people have with one another and the opportunity to see the same information in multiple lights because different people are bringing a different perspective to the information, okay? So based on that, the idea is if you have a clear objective, and I'm gonna make this slightly bigger for a second, if you have a clear objective, then you would be able to decide whether or not that should be met through an asynchronous or synchronous activity. Should we have people rehearse things on their own or should we do things together as a large group, okay? Then you have to decide, should that be done online or offline? And is this something that we call out when we say, hey, we're gonna talk about metacognition today or do we simply do activities implicitly reaching the same goal of teaching about metacognition but not calling it out, right? And then what kind of tools do we use? Do we use digital educational artifacts or do we use something analog or something that's more traditional? So by breaking it down like that, we're able to come up with better selection of the activities that we do in our class, okay? So I'm gonna stop at that first juncture there to talk about understanding by design, how it is that deciding objectives really is the key to great design. Is there anybody that would like to get any clarification or did I speak to, I know I always speak too fast but is there any clarification or things that we need to go into a little bit deeper? Cynthia, is there something there? No, people are actually being quiet in the chat. Oh, no, that's just terrible. We need you to get active in the chat or to tell me if there's anything that you'd like to clarify as far as the conceptual knowledge. Because if we don't have this foundation set, it's gonna be hard to move on. Are we all right with where we're at right now? Jennifer, are you okay with that? I got a thumbs up, okay. I'm feeling good. Yeah, doing good. Okay. Well, please do jump in and let us know. Thanks, Tanya, for the confirmation. If you do have questions though, do speak up, okay? Go ahead, Nancy. I have noticed that when you mentioned about objectives, yes, it's very hard when you are teaching because we have this curriculum with the objectives and then you have these activities that at the same time, sometimes because you cannot reach the whole curriculum during the year, like sometimes you have to incorporate certain objectives in just one activity, just to make sure that they have kind of like that understanding that we need. So I just, sometimes perhaps teachers may think that they need to do it like separately and cover everything in the curriculum and it's very hard doing it all the time. It's a very smart idea. Several objectives, as you say, can be achieved in a single encounter, right? Depending on your design, how you've laid out the structure of that. So you're absolutely right. You can have multiple objectives in a single lesson. This also gets to the last point and I hope you can drag the slide deck into the chat so everybody has a chance to look. If you fast forward, Nancy, to the very end of the slide deck, you'll see something about what is worth teaching. And one of the big lessons that we got out of COVID is that covering the curriculum was pretty much not the thing, not the main goal. Actually, mental health, number one, and then critical thinking, number two, if you could help them learn how to think through things, they would cover the curriculum fine. Our problem has been the focus as being the eye and the prize of covering a curriculum as opposed to teaching people how to think. And so we'll get there in just a bit but thanks for that observation, okay? So we moved to the second design element which is universal design for learning. And universal design for learning is a really cool idea that David Rose had several years ago but the idea was basically if you have universal design for learning, what do we do? We take a lot of the different resources we have and we create a variety of things available to students to create safety nets. So we basically think I am not gonna have a homogeneous group of students. I'm gonna have people who are from all over the place who need different things at different times. And so I have to think about that. I have to think about the students who are gonna have gaps. They're gonna come to my class and do not know certain things. And I'm gonna have students my class who are super high fliers and they are just beyond the information and are gonna be bored. So given my topic, what is the range of learners I have? So I have to take that into consideration and think about those students. You can't teach them if you don't know them, right? Then you have to create universal design for learning is basically creating that safety net of options where you can differentiate to meet each individual needs. And I say differentiate homework, not differentiate instruction. I don't have to change the way I teach. I have to change the resources and the safety nets I provide to my students so they are able to then self-serve. They will have enough range of things here that when I say go to the bundles and look at at least one resource before you come to class next week and let me know what questions you have. If they've done the Khan Academy video and they keep getting stuck on the same kind of questions they can come and say, here is where my problem is. They are able to self-assess their own needs. And that is a huge, huge skill set to have in education these days. Self-assessment is huge. So the core idea of universal design for learning is to appreciate that we're gonna have a variety of learners in the class and to prepare for that, to design safety nets. The concept is really funny. It comes based off of the tail end of Todd Rose not David Rose. Todd Rose wrote a book called The End of Average basically making fun of education these days that seems to shoot for an average or to be above average is really our goal. When the whole point is that there are very few people who are really average and that preparing an educational system for average is not really what we're all after. So trying to get beyond that. So how would you work in that? Basically what David Rose and his colleagues have come up with, and I think this is a free downloadable looking at the research and universal design for learning. It's basically a concept taken from architecture. And in architecture, really interesting idea. Pretty much everybody can go up a ramp. Everybody can do ramps. Most people can do stairs, but everybody can do ramps. So here's the idea of design. Why don't you design ramps? Anybody who can do steps can do ramps, but the people who can do ramps don't necessarily aren't able to do steps. So the core idea of universal design is to create this kind of a global safety net where people will be able to get what they need to thrive. If I were to ask you a big question here, how many times, Peter, do you need to see a new piece of information before you know it? Before you just know, you know this new concept or you know this new formula or you know this new idea or this historical fact, how much time does it take to know that? Well, basically the answer in all of education, when anybody ever asks you a question in education, the answer always is, does anybody know the answer? It depends. It depends. Oh, good. Oh, these are my students. I know. It depends, totally. And it depends on their prior knowledge. You can show a new concept to a whole class of people and there are some people who already know it. So they don't need any practice, right? Some of them, an average, may take about 10 rehearsals of that concept or that piece of knowledge before they get it. But before, if you take the whole class into consideration, Marcellano shows that it can take 20, 30, 40, 50 repetitions of an idea before some people really get it. Why? Because they don't have the prior knowledge upon which to scaffold the new learning. They have to first fill in these gaps of prior knowledge, okay? So this is the whole idea behind universal design for learning, accepting that there's gonna be people who have less prior knowledge or have gaps in their prerequisite understanding and have you designed your class where they can fill in those gaps? That is the key idea of universal design for learning, is that you have designed it in such a way that there's a safety net. People cannot fail if they take the initiative. Now, you can do all you can, but it's obviously the learning is ultimately up to the individual. But have you created the variety of resources necessary so that everybody could fill in their gaps? Even if they had a lot of gaps, will they be able to rehearse enough so that they are able to achieve the same thing? So that's basically universal design for learning, okay? This gets to a very old concept, whereas Todd Rose's work was starting around the 2000s, Benjamin Bloom back in 1968 already said, you know what, guys, most kids in our classroom, most everyone who decides to take our class can learn, they can definitely do it. And he basically presumed this because it has to do with mastery, mastery concepts of learning. If you would just, if you look at his original paper, what I love about this is that he says, there's just five ideas about learning. And that has to do with how much rehearsal they've already have so that it's easy for them to attain the task. If you've got a good teacher, good instruction, okay? The ability to understand the instruction. So does a kid have a language deficit, for example, that might prevent him from understanding? He's a smart guy, but maybe he doesn't know enough English to be able to do this, right? And so does he have the ability to understand what is being asked of him? Can they persevere? And that also has to do with rehearsed practice. How do you get kids to stick with it or students to persevere and not just give up right away? So that attitudinal shift, right? But basically he said, it's the time. Time is the key to mastery. 90% of the people in our class can get it if we give them enough time, but we do not design education with that in mind. So the next best thing is a design. If you don't have the time, can you design around that? And that would be the key idea here in universal design for learning. And so basically this 10% idea has been confirmed. He thought that 10% that we're probably still not gonna be able to learn because they need other things. So if you look at learning disabilities around the world, the world average is about 10%. So if he was not far off, even in 1968, he sort of had his head around this idea that there's gonna be some people who will need even more than just what we are offering them through a mastery structure, okay? So universal design for learning. You have to have really clear objectives. What is it that you're after? You have to then be able to disaggregate those objectives into this hierarchy of understanding. Well, I want them to be able to multiply. Well, to be able to multiply, they have to understand how to add. To be able to add, they have to have conceptual understanding of magnitude. So they have to have a lot of prerequisite knowledge before they can just multiply, right? So you have to be able to disaggregate that into a clear understanding of what it is that builds up to that ultimate goal. Once you've done that, can you differentiate by clarifying different options and understanding that students are gonna come with different needs and create that scaffolding through new tools or resources that you're offering? This would then, you need to also let them know it's there. You need to tell the students, this is being offered to you. You cannot just presume they will intuitively understand what you're offering them. You have to call that out. And so you have a transparent design, okay? So that's universal design for learning in a nutshell. And I know that this is very, very popular, especially right now in Latin America. So I'm presuming there's questions about universal design for learning. Does anybody have a question they wanna ask right now? Jacy, maybe before you move on to the questions about the actual like universal design for learning, Jennifer had a question about objectives, which does match a bit this, because you just mentioned objectives, so I don't know, Jennifer, if you want to ask that out loud, it is a bit of a- Yeah, just real quick, how do you kind of facilitate a universal buy-in? If objectives are really important, my question is essentially, is it always the expert or the teacher trying to facilitate buy-in towards that objective, or is it kind of like a mutually collaborative process where teachers also kind of adjust to what they feel like might be important in the eyes of the students? How do you align that objective process? I think that gets totally to the whole point of motivation for learning. And it has to do with, there are a lot, it also depends very much on the age group and the focus, doesn't it? So you have a lot of self-motivated adult learners who are here right now. Nobody's telling them that they have to be here, right? But so I can share objectives. There's almost no debate about it because they had a buy-in based on what they thought they were gonna get, okay, so that's already there. But what do you do? How do you get buy-in with younger kids or with other people who have not chosen what they are about to do? And that relies heavily on your ability to show the importance of it, which is why what we came out of COVID understanding is that just imposing kind of random curriculum, which to them literally sounds random, why do we have to read that particular novel as opposed to life is happening? There's so many other things going on. How do you get them motivated for that? So actually co-constructing those goals is really clear. One of our teachers here from Hawaii was reminding me that during COVID, one of the best tools at our disposal was to have a clear objective. What is it that you want them to do? I want them to be able to understand these different genres of literature, analyze the information and write a good essay. And I said, okay, is that really tied to those particular novels that you had on your plate or whatever it was? And basically when they laid out the objectives with the students, they were able to then co-construct. This is what I need you guys to be able to do by the end of the year. How are we gonna get there? We have a lot less time. What are the other tools we can use? How else can we get there? So co-construction of understanding is you're suggesting Jennifer is fantastic in that sense, okay? Sergei, you had a question? Yes, can you elaborate more on what you mean when you say that instructional design has to be transparent? Transparent to whom? What are the attributes of transparent design? Okay, can I do that? When I talk about instructional design, one in the next group of slides, because I think that'll become clear there. Thank you for that question now. Joe, go ahead. I'm just feeding into what you were just saying about this clear objective. I was just gonna share a quick little thing. I'm just coming back from my daughter's first grade classroom, where I was helping them build these bird seeds and it included them molding suet, which these kids at, you know, by the time they're in their third grade, they're like, ew, this is disgusting. But then as soon as they understood why we were doing this, why are we doing this? Is because coming out of the winter months, we want these mama birds to be able to lay healthy eggs in the spring. That focus went right into the activity, and they understood to learn beyond the disgusting element of them molding, but understood why we're doing this and what the outcome will be. So because as you were talking about this so far, I'm kind of like, okay, what grade level is this? These are these classes fit for, but it's very fluid, right? What you're saying is it can be molded to any age level, you know, all the way up to a university level, but all the way down to the classroom. So I just wanted to kind of... Excellent, thanks for helping me extend that for sure. And that means, and that's also getting to one of these lasting points of changes in education due to COVID is that really, it's not so much the what we teach, it's the how and why we teach that makes such a big difference in those thinking skills across the lifespan. So thank you for pointing that out. So now let's move into instructional design. And if you look back, and this is gonna be, here's a paid close attention. I'm gonna show you, you know, 10 different models. I'd like you to tell me what is the key or cumulative understanding of instructional design. If we look back 75 years and where it started and where it is now, what are the big takeaways that you're getting about instructional design? How do you design instructional encounter? So for optimal learning, okay? So if we start from there, you know, way back, Dale with his little pyramid of schemes, which by the way, for Dale, he did not say any of these mythical ideas that you remember, you know, 90% of teaching and 5% of what you read, that's just wrong, that's a myth. He did though, call attention to alternate forms of learning. So not just direct instruction from teacher to student, but what could there be in terms of a visual influence on learning? That was the only thing he wanted to say, but people twisted his words and then it got twisted again and then it got cited and cited in the wrong way and became a myth. But back in the day, he wanted to say, instruction does not have to just be somebody talking at somebody else, okay? So that was pretty interesting. And then good old Bloom back in 1956 with his taxonomy of cognitive abilities said, you know, I think there's another way that we have to design teaching and learning. We have to remember that there are some fundamental things that have to basically be remembered. Students have to understand that and then be able to apply it before they can ever get to a stage of analyzing it, which is really deep thinking, right? Or, you know, higher yet to evaluate, to judge it and to create something even better, they have to have these basics, right? So nowadays we think about this more on a wheel as opposed to this hierarchy, you know, stacked up on one another. But the basic idea is that there are fundamental ideas that have to be learned before the higher order ideas. Okay, so all of them are needed to get the higher order cognition, but you have to figure out what you are asking in your objectives at what level of looms you're thinking of. So basically that's another way to think of instructional design. For example, flipping. In flipping, the recommendation, my recommendation is that anything that is basic core knowledge, dates, facts, formulas, names of people, flip it. And then come to the live classroom to do the higher order thinking together. So basically that's another way to have a filter for choice of how you design classrooms is to take the lower level types of thinking, the things that are necessary, but they depend on a lot of different repetition for different people. Some people will know a lot about the information and can leapfrog through the flipped video. Other people will need to watch it 10 or 12 times because they don't understand the core concepts. But everybody can then come to class and then we can analyze and go deeper with the information. So Bloom is another way to think about instructional design. Gagne was another one who really was key in looking at instructional design and he said there's other conditions. And he's the first one who started to hint at, you know, your brain and your mind have something to do here. You pay attention to certain things over other things. You must rely on prior memories to get that information going. You have had life experiences that will change the way that you will understand the new information. If you don't have guidance, you will never get to higher order thinking. Your brain cannot not learn, but it can never get to higher order cognition without guidance. You need to have good feedback and you need to be assessed so that you get that kick in the pants to try to do better later. And you have to be able to transfer that information or there's no proof you really learned anything at all. So Gagne had this other type of perspective on learning. Abby was a structure designed for when we started to think about computers and computer learning. You have to analyze, design, develop, implement, and evaluate. That's the ad-e-model that was attached to computerized kind of thinking back in the 70s. And then we had this other idea of, you know, with a whole space raising, getting ahead and trying to see how could we teach science and steam and STEM? There was an original 5E model which basically said you have to engage students. Let them explore the information, force them to explain what they mean, and then the teacher steps in and says, wait a second, let me help you elaborate here. The right vocabulary word for that idea is whatever, right? The teacher only steps in and in this elaboration stage and then you evaluate. I added on this idea that you also have to, I think, get prior experience. What do you already know about this information? So elicit prior experience and then how can you use it in a new context? If you can't show that, you really haven't proved that you have learned. So this is another way to organize the design of classroom encounters, okay? Briggs's idea, he was a student of Ganyu, is to, you know, you have to assess what that person actually needs. So here we got the first chance of people starting to think about personalizing education. You have to match that with the objective. So what do they already know? What do they need to know? Okay, now what are the right strategies for them to get to that? How do we deliver that content? And then how was that assessed? And then they start all over again, okay? So Briggs was also very keen on this evaluative loop. You have to know what they already know. So this sort of pre-assessment diagnostic before you can actually teach and then evaluate whether or not what you taught made any difference, right? Have a baseline first. Merrill was the first one to really dive into this idea of authentic learning. So this gets to your point, Jennifer, what does this mean in your life? You know, not just, you know, we have to cover these books for AP English and you have to read this, but what does this mean to you? How is this authentic in your real context? And that was, believe it or not, not on our radar until around the 90s to think of authentic learning and the applicability in real classroom settings. And Keller, going back more to, you were like solely on this idea, everything is based on motivation, but the main problem with motivation and this gets to your point also, Jennifer, is it relevant? You know, do you have confidence that you can actually learn it? Are you satisfied with experience you just had? Therefore you pay attention. That's all great until you get to neuroscience which points out that different people are motivated by different things at different times and at different stages of the learning process. So while motivation is absolutely beneficial and it's worth taking into consideration this idea, you have to know your students or you cannot motivate them, right? So if you don't know what motivates them, you can't do this. Now, Sweller is probably the God of instructional design. Why? Because he married the concepts of the brain and mind to computer design. How do you lay things out? How do you make this aesthetically pleasing? How do you reduce cognitive load which is the energy your brain needs to be able to pay attention and to memorize things in order to make sure that they become real learning. How do you reduce cognitive load through design? How do you create a webpage where the person can find things intuitively so they don't kill themselves trying to figure out where do I download the homework as opposed to doing the homework, right? So you have an instructional design that becomes intuitive and more aesthetically pleasing thanks to Sweller's work which has to do with cognitive load. That's a huge idea that really changed the whole face of design thinking about education. Then radical neuroconstructivism based on Von Glasserfield's idea, this idea that learning doesn't happen in a vacuum. It's just not Megan all by herself sitting there. It's Megan talking to Alyssa who talks to Amy who hears an idea from Giselle then bounces that back and forth. So there's this dynamic iterative process which makes it radical in that sense in which we have a construction of knowledge based on the individual's exchange with the whole world, right? And since all new learning passes to the filter of prior experience this means what you already know influences what you can now. And this is precisely why if you were to read a book if you were to read Anne Frank when you're 12 years old and then you read it again when you're in first year school and high school and then you read it again as an adult the book is different. But the book isn't different, you're different. You're different because your life has changed you which means that you are now reading that book differently, okay? So that dynamic exchange is basically what we consider radical neuroconstructivism which means that learning takes place in a social context. This is also very much heavily relying on Bandura's idea of social cognitive development, right? You have to have exchanges with others by knowing the other, you know yourself a lot better. So that all works into this whole design element have you created that? Then more recently we start to get the idea that social emotional learning is part of education and that had never been a part of it. And this has been the home problem until boom 2000s and then all of a sudden COVID which prioritized it even further. So that understanding equity, choice, flexibility does everybody come in on an even plan full? No, they don't. Kids are coming in with all kinds of baggage and so how do you actually leverage that to teach better? And the most recent thing in instructional design comes from some friends on Glenn Whitman at NeuroTeach where they're trying to do an overlay of diversity, equity, belonging, inclusion with mind, brain and education. They basically think that if you do not have an integration and there's a publication of this I can drop in the chat if you're interested in it. If you're not able to take into consideration that social emotional element, cognitive learning is out the window. You can't really get there. So we all know from Mary Helen and Mordini Yang's work there is no cognition without emotion. So how do we now have a buy-in as teachers that our instructional design has to care for that? Not a matter of content, content, content. It's relationships, it's feedback, it's building community, which end up having a huge, a bigger weight than just transmitting content. So those are the basic ideas of instructional design. So there's a lot of different summary titles, things that you can look at if you're interested in that or you can just read my book which has all of those ideas in there already. I'm very happy if you can get to pull some other ideas about instructional design from that. But what do the collective models say? And I think Sergei, this will get to your main question then about what do we mean by making this transparent? How do you make instructional design the expectations of my design clear to the students? And so if you look at all of the models together, if you look at all of these instructional models together, you'll see that it has clear objectives, that documents the learning before and after, that the evaluation criteria is shared and transparent, that there are a variety and a balance of synchronous and asynchronous activities that are purposefully selected depending on the objectives, right? That they leverage prayer knowledge of the individual and that students can consider them authentic. This is something that means something to myself, okay, to me, myself and I, right? And that there's a variety of pedagogies that are used that can lead to a leveraging of the technology tools that we actually have out there. So that's kind of the summary idea, but I'll break it down into 10 steps that are suggested or 10 criteria that are on the association for educational communications and technology, they're standards. They basically say, here's 10 key standards to good instructional design. The first is that the objective is purposeful and clear. It's visible. You've clearly articulated what is the goal? What is the objective? That is clear. The students don't have to guess. What are we doing that for, right? Just like Joe was saying, this is so they can have babies in this nest in the future and have food, okay? So state the objective upfront, okay? The goals and objectives are clear. Second, it makes sure that the assumptions are clear. This means I can presume prior knowledge of X. This is why in all your classes, does this have a prerequisite? Does this course have a prerequisite? No, okay, then I expect nothing of you. Oh, it does have a prerequisite? Oh, you should have really done some more, you know, academic writing before you get to this class or whether, if we'd called these things out, different people would or would not join the class, right? So the assumptions are made clear about what we think about your prior knowledge and to be able to learn better, okay? The sequence, this generally has to do with how you lay out your syllabus. Is it clear to the individual? Do they know what's coming next? Is it clear what order things happen in on a weekly basis throughout the semester? Also, how it fits into any certificate programs or any larger structure, a degree program. That the activities are clear, that they have passive activities and active activities. They have synchronous activities as well as asynchronous activities, not just all the same thing, okay? And that they have a range of resources. This is why in bundles, when we work with K-12, we have them develop bundles that are not only readings and videos and podcasts, but also gamification and apps and other things that can create bundle opportunities for choice so that people have a variety of different resources that are available to them, okay? That there is a clear application of the information. They can use, it's usable knowledge. The opportunities for use are actually clear and authentic to the individual. That there is visible assessment. People know the criteria. There is a rubric there. There is explanations. There is feedback so that they don't feel that it's a random kind of an assessment, but that they know what they are being measured against, okay? But there's time for reflection. There's opportunities to take a little breather and say, wait, what am I actually understanding by this information, okay? The reflection time. And then also independent learning as well as the active group learning, but is there an off ramp for people to follow up and pursue their own areas of interest or do deeper dives into what they care about most from that lesson. And then finally, oops, has the evaluation criteria been laid out in multiple ways? A single evaluation tool. One of the things that has definitely gone since COVID is a single standardized test determines whether or not you're smart or not wrong. It's always been wrong, but it's extra wrong now because it's very, very clear that that is not using one kind of a tool is not the way to measure human worth, okay? Human understanding or knowledge, okay? So based on all of those things, the ideas are that the standards, there are standards for instructional design. There's lots of different ways of laying this out there, but part of the idea is to make that visible. Cynthia made this graphic for our class showing how in our class we do things before class, there's things after class and there's things during class, there's things that you do on your own, there's things you do in small groups, there's things you do in large groups. There's a different variety of activities that occur in our class for a reason. It's purposely thought out that it has this variety of different entry points, okay? So Sergei, let's go back to your question. Can you answer your own question for yourself now? Yes, I think so. I'll need to go through the presentation once again to digest every single item, but it appears clear now. Thank you. Great, so the whole idea is that these things are shared. It shouldn't be an invisible target. Kids at any age should know what's expected of them, should feel accomplishment when they achieve. They should understand what was the goal of the learning and then they should know when they've reached the goal. It should not be something that's invisible or only the teacher has that criteria, okay? That's the bottom line in that sense. Okay, so that is instructional design. That's a lot to throw at you. Most people spend about two years getting a degree in this and I've just tried to do it in 20 minutes. So I'm sure that there's a lot of points that can be confusing. Are there some things you want to latch onto here about instructional design? How about you instructional designers out there? I know that there's several of you in the audience. Did you wanna add anything? I cannot be that clear, you guys. I'm sure there's questions. Amy, anything you wanna add to this? Not yet. Okay, anything in the chat, Cynthia? So, yeah, but Nancy raised her hand. So maybe Nancy first and then I can take it. Okay, go ahead, Nancy. I just want to, I also put it in the chat just now, but I want to thank you for explaining because I love times, it's very hard for the students to understand what is in the teacher's mind, like how they display a class and kind of the methods and from teacher to teacher, it varies. So I think that like, is there any challenging time for the students, especially to identify the objectives, identify why they are doing the things that they are doing and explaining it in this way is much easier to understand it. So thank you, that's very nice, thank you. I'm glad that's helpful, thanks. Alyssa, go ahead. I just wanted to add a small thing that we talked a lot right now about instructional design, but another term here that is often uses learning experience design, which is very closely related, but it still kind of can be somewhat different. I think sometimes thinking about designing an experience helps to better understand because you are not talking only about the actual kind of material and how you communicated to students, but you also think about everything that surrounds it, which I think sometimes instructional design does not include. And I feel that when designing, of course, designing an interaction, it's good to switch between the two and kind of to help you understand, okay, if I think about it as an experience, what else would be around the students that can affect their learning, that can influence their learning? Excellent. And kind of, well, yeah, balancing in between two, I think it's an important thing. I agree completely. And in fact, Jay McTye has actually switched out backward design. The last one used to be objectives, evaluation activities. He has now changed that for learning experiences. So I do agree that idea of the conceptual idea of getting the experiences that you have that actually help you reach that level of learning or meaning making as Mary Helen will put it, is actually vital. So thank you for that, excellent contribution. Cynthia, did you wanna say something about the chat or anything there? So for example, there was a question from Joe about how to bring together how backward design, universal design for learning and instructional design all kind of mesh together and come together. And I think Lailia, you were asking about flipping strategies too, but there's a few people responding to that in the chat. So maybe the first question about how they come together. So how they come together is actually the next slide, Joe, because we're gonna give you a worked example of how that all comes together. So I'll do that in just a second. And Sergey, did you wanna say something else about instructional design? Yes, a reflection and a suggestion. My middle son has an extremely talented math teacher here in North Carolina. Everything is perfect, except now I understand the design is not clear to students because what happens from time to time, around two times per week, my son complains about his class, his teacher, the subject and so on, even though he's an A student. And every time I hear that, I come with my explanations of the goals of the instructional design and remind him why I think the teacher is extremely exceptional and very talented. Then he comes back to norm and performs well. So my suggestion is that I think a lot of parents observe the same issue. I think the idea of transparent design to students has to be purposefully taught to teachers. They have to have a specific instruction on what makes instructional design transparent to their students. Excellent, and that answers your own question, right? What does it mean to make things transparent? And that is, there are usually excellent intents. There is clear expectations that are expressed by a teacher, but none of that matters. What matters is the student perception. Believe it or not, it does not matter what the teacher meant to do. It matters what the student perceives was done. And that is why calling certain things out more explicitly is important. This is why part of some instructional design elements are to say, they basically say, make the implicit explicit. But they should all know that they have to turn it in on whatever. Nope, make it explicit. Don't presume knowledge, because that does take away, that pulls a rug right out from underneath your understanding. You think that everybody understands the same objective because you presume. While it was said at one point or whatever, well, maybe they missed that. So calling it out again is really important, okay? Cynthia, was there anything else that I need to look at right now? Because I need to move on to the example, because I wanna get to Joe's question about the worked model. So, Joe, there is a longer video if you'd like to watch 49 minutes of how we created psychology 16 on nine and all of the steps, the 12 steps, one by one. But we actually teach a course and connections on how to do each of those steps one by one. And it's a very successful process. But I just wanna go through some of those highlights right now so that you have it in mind. And you can just link to the YouTube video right here if you wanted to watch this in more detail and more commonly because I'm only gonna hit some highlights here, okay? How did we actually lay out this class taking into consideration backward design as well as a universal design for learning and instructional design? What we basically did is we thought our class is 100% online, right? You guys know that it's done in Canvas and in Zoom, it's flipped, we meet sync princely. All of these elements to have discussion boards, quizzes, reflection papers and a semester project were very, very well thought out, right? We have students 11, 12 different countries. Basically we have this semester 22 different countries which is kind of awesome and wonderful because it adds so much to our class to have different perspectives, right? There's normally one instructor to teaching fellows this year, we have three because we have a very class and we've been teaching it since 2014 but perfecting it since 2016 and 100% online context. And we love showing off this course because the feedback is always positive about the design. They tell us that this assessment structure work, there's 100 things but none of them will kill you. There's low stakes, testing all the time. The design format allows for any individual to be successful even all the way through. So long as you're learning, you're actually getting better. Flipping is something that they also love a lot. They're not used to getting flipped classrooms and have bundles with differentiated choice materials that students get to choose their homework is a very different structure, okay? The ability that you have to resubmit anything that you want, if you're not happy with your grade you can resubmit it and get regraded. The fact that people can learn from their mistakes is something that doesn't happen a lot. And we'll talk a lot about that at the end when we talk about some things that have changed in education as far as time is concerned but also the creation of the community the learning community that we have just because people have the chance to hear perspectives from people from so many different countries it can blow your mind because you thought, well, everybody thinks like where I came from and then all of a sudden you see something very, very different and that is the future of education international classrooms is the future of education I feel so that just changes everything on perspective taking, okay? So we have lots of these I'm not gonna show you all of them because there's tons of things I love to we get the nice strokes at the end but I'm gonna explain 12 steps that we use to design this course and they have to do with everything we just talked about. First backward design is decide the course objectives what are the objectives of this class? And we basically lay that out in the syllabus and we tell you, at the end of class you should be able to do X, Y and Z there are five or six main goals of our course that are laid out in the syllabus. So once that's clear, then we have to decide what kind of topics will help us reach those objectives. And so we basically go from there and we work backwards, we have 15 weeks to work within what are the topics gonna be? Well, we know where to place certain things and other things and what has to be prerequisite knowledge for other things and we try to time it in a way that each of these different units will build off of another and once we've done that then we have to decide the weekly objectives once we have those unit objectives we decide, well, what is the objective for each one of those? So for the week 12 on the mind-body connection we have to decide what are the sub-objectives of that particular week that feed into the larger objectives? What are the knowledge skills and attitudes at that level? Okay, then from there we're able to build out the bundles that are necessary and to create the pre-class slides. Once we have the pre-class slide deck we can make the pre-class video. Once we've made the pre-class video then we can make the quiz because the quiz is based off of the pre-class video. Once we've done that we can decide what the right discussion board prompt would be to get people thinking about that before they come to the live class. Once they've done all of that then we have them look at the discussion board replies and we have them then give feedback in that sense so we have the whole loop here where the students begin to talk to each other not just us and they use the bundles those resources to be able to rely on that for their information. Then we have to create the live-class slide deck which is based off of your discussion boards based on what the students created in their discussion boards we then create a new slide deck that highlights certain points that we wanna make sure are emphasized and we clarify any errors we saw coming up in the discussion board about the topic, okay? Then we come to the live-class and when we come to the live-class but many of you probably have realized is that classroom management is very different online than it is in face to face. Normally online, Megan walks right up to that kid puts the hand on the shoulder and says so and gets their attention, right? Well, you can't put your hand on anybody's shoulder in an online environment, right? So how do you do these things? So basically here are some examples some of the things we do is we're always in the classroom slightly earlier. We greet them, we try to greet people we try to use their names and we try to get them to land from their outside world to being in the inside class with us so that maybe we can use some of the information they share with us to bridge into where they're coming from with the information, okay? So we take advantage of those shorter conversations before class to try to integrate that into the live-class with what's going on in the real class. Then we, oops, sorry. And when this class starts, we say we're gonna be recording, okay? Just so you know that. And we also try to thank people. It's hard to come to these things. It's a big dip in your life and your adults, you can decide what you do with your day but you decided to be with us which means we feel very special and thank you for doing that. Where possible we try to call on people by their names. We also try to get exchanges going straight away at least and encourage the chat which is not happening in this group. I don't know why, what's going on you guys but we love it when you chat. Now, some of you guys have been conditioned. Some of you who are outside of my class don't use the chat because it's a distraction. Well, we try to use it to give immediate clarifications because your brain needs attention and memory to learn. You cannot pay attention if your mind is divided. I have a question about something and it's not been answered. You cannot pay attention. So we use the chat to get immediate feedback, okay? So we encourage this back channel of information or deeper knowledge or sharing. Some people don't feel comfortable talking out loud but they're happy on the chat, okay? If you don't have an assistant or TA you can ask one of the students. This is an incredible skill to be able to synthesize what goes on in the chat and pay attention to what's going on in the rest of the class. It's a great skill set to have. So maybe you can get students to do that, right? We try to begin the class by clarifying the objectives and asking if there's anything that needs to be modified and then we begin by diving into their own questions. We use the students responses to the discussion board as the content of the class. That's the jumping off point for the information. So we start with their ideas. We begin with one or two students in the way that they responded to the chat and get them to see the own patterns in how they have thought about that issue. And we try to see how they differ in different ways and call out their attention to their reaction to the topic in that way. Then we try to do breakout rooms since they can dig deeper into one or two of the ideas. It's much better to go deep than broad. Much better to go deep than broad. You can go broad in the pre-class flipped video but in class go deeper into one or two ideas and the big broad overview painting in those broad strokes, okay? Then after that we share, we come back from the small groups and hopefully we call on one or two people. You know, what happened in your small group? And we make them say the names of the people they were with so that they begin to know each other. And they can say, well, you know, I really liked what so-and-so shared and that also gives that other person confidence that what they have is a valuable piece of information to share, okay? Then we add on to those ideas and then we also encourage the depth of a breadth. It's much more important to be able to dig in to an idea and get that pattern of understanding going in their heads than to globally just gloss over stuff that they could Google, they could easily Google. We want things that are unique to the class to come out, okay? If we have time, we go into digging to another idea and do another breakout room but make sure at the end that we try to summarize where we came with the information and try to give it context within the global scheme of our entire course and also into their own lives, hopefully. And then we preview what's gonna be coming up the following week, okay? Then we give time for them to do a reflection so that they can consolidate. What did I really learn today? Did I learn anything new? Did I, am I curious? I wanna research other things. Is there anything I might change to get that reflection going is very important as far as consolidating information. And then afterwards we stick around to tie up loose ends. We end on time, but we give time for people to stick around and clarify anything. Believe it or not, you think this takes too much time, it actually saves you time because you do not have to have tons of office hours afterwards because you stick around and clarify things and others hear it. And so there's a global sharing that's much better instead of having individual office hours where you have to make another hour for other things, okay? Then we make sure that we have different types of evaluation tools in our class. So we have asynchronous activities, so this gets into the instructional design part. We have quizzes, you have to watch the pre-class video, you have to respond to the discussion boards and you explore the bundles. Those are all asynchronous and done in your time, okay? Whenever you want to do it, okay? Then we have an asynchronous live class meeting time, okay? And we also have sections or the workshops that we have, right? That's when we're all together and that's when we share perspective taking on information, right? And you do that three, two, one reflection. And then you step away from that and asynchronously you do your own things again or you dig deeper or you reflect on how you did on something else and you have the opportunity to also do over, repeat anything that you think you could improve upon. So all of the things that are graded in our class can always be redone. That is good instructional design, I think, I hope because I think that that allows for a balance of asynchronous and asynchronous work synchronous and asynchronous work in a variety of formats. So you don't just nosedive. If you're awful at taking quizzes you don't do bad in the class. If you're awful at academic writing you don't do bad in the class. You can pull out your strengths and do well in this class with a variety of different strengths but you have to work on all of them. We also insist on really we try to have good communication. I don't know, you guys will have to tell me how well we succeeded that but we do try very much to leave our lines of communication open with students and they know how to get in contact with different things for different reasons. If they're sick, they have one way to communicate. If they have a question about logistics they have a way to communicate. So they should know the different ways to communicate. The 12th step has to do with the change in our priorities. We spend far more time preparing our class evaluating student work than we do on teaching. We teach for two hours a week but we spend three times that amount of time getting the class ready and evaluating because it's been shown that one additional hour of feedback to a student is worth much more than one additional hour of instruction to a student. I don't know if you guys know that but the OECD found that just teaching a lot of new stuff doesn't help people as much as understanding what they need to know making meaning out of what they've just experienced. That's much more important. So the evaluation element is much more important, okay? So this means that this has changed priorities in time. We have a lot more time teachers and the teachers in Hawaii know this. They are working 12 months a year now. It is not like you have this, oh, I work a nine month job because I'm a teacher. No, you spend a lot more time preparing. Teachers should never have ever have more than four or five preps. It's impossible to do quality teaching in this format if you have tons of classes to do. So these 12 steps are a basic guide. If you wanna watch the detailed explanation of it, this is kind of how we got to where we are and I would encourage you to do that but it's using backward design. So at every level, at a macro level, we try to figure out what is the objective? At a meso level of the weeks, we try to figure out what is the objective and at a micro level of every single class, what is the objective? So we are always using backward design in our thinking of how to present information or structure information, okay? So that's the understanding by design element that's in there, okay? So pros and cons of doing this, you really have to have a different mindset to take on this kind of a class, okay? But I think it's much more humane. It lets technology do what technology does best and it lets people do it, people do best. When we get in the class, I don't panic about, oh my gosh, what should I be teaching? I get to think about what does Heidi need today? I get to think about you guys. I don't have to think about the materials because it's already been pre-designed. It's done beforehand, okay? It also means that we can focus on individual student needs. We can personalize a bit more. There's a lot more differentiation that's offered but there's a ton more front end planning. To flip takes a lot more effort and I acknowledge that, right? But it documents the learning far better. We know by having you do your projects, for example, in stages, we can document how far you have come. So our documentation evaluation is far better in our class. We can see how many times each of you has taken the quiz and we see how you go from getting 76% to 80% to 95% to getting 100%. We can see that progress. So it's much better documentation, right? And we use a lot more resources. We try to have bundles instead of a textbook. So that is our design and that is the way our course is structured. And I would invite you to look at the longer video. And for those of you who are living it right now, I hope you now see some logic in the way that we were trying to set this thing up. Does anybody have any questions about that before we talk about these long lasting changes in education as the last element? Anything? Isn't it funny how our group can't stop talking Cynthia when they're just together? But now there's a couple of people they don't know and so they're not gonna talk as much. I think that's kind of funny. That just tells you how building a community of trust is very important. Go ahead, Ann. Okay, so I do wonder, this totally makes sense. I love it, love it, love it. But how do you keep the wheels turning forward? How do you keep things from getting bogged down in review and backfilling? And when you say, hey, remember when we talked about this and they go, nope, we don't remember that at all. So you have to review that concept before you can move forward. And the timeline keeps getting pushed out, out, out till finally you're like a whole year and you've only covered maybe a fourth of the content. No, no, no, no, this is the key. So this is the idea of the universal design. So let's go back to the universal design, okay? The design of your class should be such that you can pinpoint when somebody has missing prerequisite knowledge and needs to fill in their gaps. You allow the students to differentiate for their homework, depending on their age and their needs, you can let them select. You guys are all adults, so I say choose what you wanna pick from the bundle. You guys choose what you think you need. But believe it or not, everybody looks at more than one, two or three things because they're trying to find the right things. So I know they're doing more homework than any assigned chapter, right? But for younger kids, you have three options with the bundles, right? If you've created the safety net of options of resources and you see that one kid really needs to shore up some prerequisite knowledge before we move on to this other concept, you can either tell him, I can go ahead and choose your own homework. I wouldn't do that, that's one option. The other option though is to tell him. Okay, so for you, Johnny, I want you to begin here. I want you to look at this particular Khan Academy video and if you can get 10 of the questions right, go to the second one. If you can get 10 of them right, go to the third one. And if you can do all that, great. Then I want you to do this next thing. So you can prescribe the homework but because you've got those options already laid out in your bundles, you are directing him to do the work. It's not that you have to wait for him, the whole process to wait for him. It's that he, no, you need to tell him though. You know what, we can't get to subtraction unless you're really sure about addition. And that's a really hard thing to do and it's very important skill to have in life. Okay, so in order to make sure you know addition, I want you to either do this gamification, play this game. I want you to either watch this other video or do this exercise or whatever it is. You can prescribe what you know he needs. But the beauty of this design is because you've thought about this hierarchy of understanding that has to occur for everybody to reach those objectives and try to offer the resources that would help fill in those gaps. Now you are able to simply dedicate it to you, you are spending a lot more time being a doctor, diagnosing what he needs and getting him what he needs, then generically saying, you don't have what it takes to move on. Now you can diagnose better and treat better because you have a better, a more refined, you know, focus on what they need. Does that make sense? It's a lot of pre-planning, but it definitely is worth it. Because in the end, what you're doing then is you're actually using the tools that you've set up. Okay. Go ahead, Nancy. I just wonder about your idea about multitasking. In a way, I got a little bit confused when like concentrating the, for example, presentation and in the chat, as you mentioned that it was like very good to have that skill in a way. But at the same time, I was thinking about the fact about multitasking and focus because for our students, most of the time is nice to have the chat for those who do not like to participate actively, like speaking, but at the same time, they just, the people that I speak, do not wanna miss anything in the chat. And vice versa, it's kind of hard to, I don't know, maybe some people can do that, but I understand that some students have that. And I love the fact that we have the recording because you can go back. That's amazing. But if you don't have that in your institution. You do have that. Zoom has that. Everybody can have that. That's the whole idea is that the whole format, we're gonna get to that right now when we're gonna talk about what has changed in education forever, but precisely what you're getting at, offering these multi, these different ways of taking in the information and interacting with the information. You can either be on the chat or you can be present with a group or you can contribute by talking or you can write or you can do nothing, right? All of those give, it create a greater potential that people will be participating. However, you need to have that safety net of also having this as recorded. As I mentioned to you before, about 50% of our students rewatch the entire class after it's been done, either because they missed something in the chat or they wanted to see another point again or and that's fine because they have the opportunity to do that because the materials allow for that. So creating that space where they can go deeper in that way is great. I do agree, you're right. Your brain can only do one heavy cognitive load task at a time, which is why instructional design is so important. Having these variety of entry points does not mean you have to use all of them. It does, it serves as a protective factor for people who need one or the other, but it can be overwhelming to be a negative thing, a risk factor for somebody who's trying to pay attention to everything. Just ignore it. This is what we tell you, don't use everything. Use what serves you. But again, your brain adapts to what it does most. So whether or not by the time you end one class with me, you'll be able to do both. But you have to get used to it. It's not something that's intuitive to be able to do that, right? And you even see oftentimes the TA's, I'll ask them something and they'll say, wait, wait, wait, I was responding to something on the chat. I wasn't paying attention to that. Your brain can't do that and we accept that. But what we do want is to pre-communication lines in the event you need it. Your brain cannot learn if it doesn't pay attention and you cannot pay attention if your mind is thinking, wait, what did she just say? And if you can't get clarification, you can't let it go. You stay stuck in that space. And so you need to get clarification almost immediately, which is why the chat is beneficial in that sense. So you're right, there's pros and cons to it. I agree with that. Let's look at some of the things that are changing in education forever, that basically will impact all of us in education. Before you go on. Oh, yes, sure. There was a question from Jennifer about learning skills and things like that about students in online learning. Jennifer, I don't know if you wanna ask a question. Go ahead. Sure, thank you. Yeah, just real quick. There's online learning cater more towards students that are able to be kind of more self-accountable without a teacher looking over them, as well as being able to just kind of cultivate their own learning environment. And is that a skill set that could be taught? So does a cater more towards those types of individuals, if that's a thing, and can it be taught to kind of elevate those students that don't need to have so much accountability with a physical classroom? Before COVID, we thought we knew what online learners were. They were people who were older, independent, self-motivated, that you have this kind of boxed-in version of what you thought an online learner was or a successful online learner. That just went out the window with COVID because we found, for example, about a third of the people who self-report is saying, you know, I'm a total introvert, absolutely thrived and loved being in the online environment. Nobody had ever studied them before. Nobody ever bothered to take these people into consideration because they were looking at these kind of superficial traits that they thought, this is how you are successful when you, you know, and there's a lot of research on that. The key idea I think to take away is that anybody can be, the modality of teaching online, it's only that, it's modality. It's not good or bad. It is, it can be better in one situation and other people do have preferences, but your preferences are highly influenced by your past. It doesn't mean that you can't get better at them. Remember we said there's a myth of learning styles, right? You are not, Jennifer, an auditory learner, you're not. It's just that maybe if you were told you were that, when you were little and you spend a long time rehearsing that and getting better at that. So we've all become very good at being in a classroom, which is why we were desperate to get back to a classroom. However, with the greater habituation caused by COVID, many people have become better online learners and they see that, yeah, it's kind of cool to take a class, you know, when you're halfway in pajamas and have a cool beverage next to you or whatever it is. And, you know, you feel better. Who knows? I mean, there's other benefits and drawbacks. So it's not that it's good or bad for saying, there is a ton of research going on on characteristics of successful online learning. And basically it's identical to what is a successful in-class learner. And all boils down, believe it or not, to the successful nature of the teacher and teaching. Basically, your experience is influenced highly by the teacher and teaching more than the modality, believe it or not. And that's some of the latest research, really 2022 articles pointing that out, which is no surprise because that's true for face to face as well, right? So go ahead, Megan. I'm sick guys, so I'm sound really scary. But I wanted to share, I was late because I was going through everyone's amazing research questions and responding and I got caught up in that. And I'm just so grateful for such a variety of interesting topics and perspectives. But I wanted to say, just looking at this question, what has changed forever, thanks to COVID, is there's been a disruption of parental attitudes and just general attitudes regarding screen time. And it's very interesting because I feel like there was an overwhelmingly negative, at least in my Montessori school, attitude about the use of screens for learning in children. And I feel like that's been subverted by remote and online learning becoming the necessary in the everyday. And so I think it's a very important and interesting time for discussions like this to be happening. So yeah, that's what I wanted to say. That's great and you're right. I think it's creepy voice. Very sexy, it's okay. The whole thing about attitude change, we have advanced more in understanding the tools that we have in online learning in the past two years than we did in the past 20 before that, which has been amazing and a positive for all the horrible, awful things of COVID, something good had to come out of that. And one of the things that brilliantly came to mind was looking at far younger children and integration with technology because as you say, err on the side of caution. So the National Pediatric Society had basically said, no, kids under three should never see screens or whatever. They started walking that back in 2017 because they realized kind of silly to blanketly say, X is bad, it's better to consider everything in moderation number one and number two, what is the objective? Because a series of studies began coming out which showed depending on the objective, technology could actually be far superior. For example, literacy, with they gave bunch of three and four year old kids iPads and basically said walk around your house, use this app, it's called My Story. Take pictures of everything, drag the pictures into an order and then tell me a story, okay? Tell me a story about your house. So then the story would be transcribed underneath the pictures and the kid and they could play it and it could play back in a British accent or an American accent, a male or a female and it would tell the story. So all of a sudden a three year old realized what I can think and say can become the printed word. And it was this huge aha moment. And then they sent the, they were asked the second part of the experiment, send that video to a relative who doesn't know your house. So the kid in Scotland would send this to their aunt in Australia and then they would videotape the kid as he talked to the aunt. And the aunt said, Johnny, I just loved your video that was so wonderful, except for I have one question. Was this picture the side of a door or a table? And in that moment, you saw this kid's brain going hot and electric where he figured out what I see and understand is not what you see and understand. Empathy for understanding, right? Ah, so therefore I either have to take a better picture or describe it better for the other to understand what I see. So what they realized is after a year of playing around with this, that three and four year olds had literacies of eight year old kids. They understood language a lot different. So that tipped the scales and the American Pediatric Society totally walked back and said, okay, okay, not all technology is bad. The good technology is good. Well, obviously, right? But the problem is people were blaming sort of the medium as opposed to the usage. But once you had good, smart, intelligent objectives, depending on the objective, different tech tools were good. So you're right, attitudes have begun to shift very much. So looking specifically at tools and technology, one of the things that's very interesting is technology has always been here, guys. It's like it's always been here. We just always freak out every time something new happens in technology. So ever since the 1900s, we've had other kinds of technology and we thought when we have movies, people will stop reading books or when we have Netflix, people will stop going to the movies or theater or whatever, it's just not true. But the updated version of this is now our ability to integrate apps and have real time, for example, tutorials or we have personalized learning which is adjusted algorithms that adjust to an individual learner is really changing the face of learning now. People don't have to have a teacher who has to talk to the kids individually. They can have an app that actually says, here's my photo math app. I took a photo of the problem and step by step it explains to me what was right or wrong. I can now go back to school and ask the teacher questions about, well, I don't understand this part to this part. I don't understand these other things. We can now personalize learning in a way we haven't been able to before. So teachers now had a big choice during COVID. They had to figure out, am I gonna use multi-course platforms? Sorry, this is a very small graphic here. Should I use these multi-course platforms that exist to supplement my learning or should I use the software that's out there that helps me do plagiarism checks? So I don't have to do this myself, right? Or that I can do blogs with students so that I can have checks, grammar checks done on the writing before they submitted. Or should I have them listen to podcasts or use other websites that have access to information or use apps? Have them do three levels of dual-ingual before they come to class in French or whatever? Or should I use gamification? All of these new tools, all of a sudden changed our toolbox. It changed a lot of what we thought about as far as what do we use? Do we use a textbook? Textbooks are out the door, sorry. They are already old before they're even printed. So now what are we using? We use bundles. We have to update our information, okay? A second of four things that have changed a lot is evaluation. Before we used to think you would assess a kid, give them feedback, teach them something, evaluate them and therefore here's your label, here's your worth as a student. But we don't do that anymore. We don't do that anymore. We have rethought everything in education thanks to COVID. And basically all these standardized tests that were brought to a halt, that made us realize, hi, there is life without a standardized test. So maybe that's not what we need to use. The United States kids are some of the most tested in the world. They take an average of 112 tests between kindergarten and 12th grade. That's absolutely crazy, right? The main problem is when we think about the objectives, your tools have to align with your objectives. So what is your evaluation tool? If you say that my objective is that they are free thinkers and have good attitudes about learning, that they know how to work together or that we imposter a level of learning and we want them to be innovative and open to life and lifelong learners about it. And then we give them a multiple-trace test. We have got the wrong tool. So we have to align the objectives with the tools. And that is something that began to happen. We started realizing if you wanna do knowledge, go ahead and give a quiz. But if you wanna measure skills, you're gonna have to do other things. Our measure attitudes, you're gonna have to use other tools as well. So what did end up coming? I'm bubbling into the surface for a lot of groups, especially those going to college were e-portfolios. So basically using a different kind of system to evaluate. Since they couldn't use tests anymore, kids couldn't show up to take the SATs. Well, that was the best excuse ever to get rid of them. And we stopped giving them or stopped requiring them. And so what is replaced that e-portfolios? Basically having digital record of a person's growth over time is a far better indicator of what they know and are able to do than a four and a half hour test. And everybody agreed on that. And so they've sort of now moved, people are now moving into this idea of looking at e-portfolios and 3D transcripts as a new way of looking at education. Okay, so our idea is to just rank or judge kids, go ahead and use the standardized tests, that's fine. But if your objective is to do a different kind of education system that values other kinds of things, we need other kinds of tools, okay? So lots of different ideas here, feed forward. And I'm just gonna end on that note just because of assessment ideas. And I'm gonna share the last two things that have to do with curriculum. What is really worth teaching? And for those of you who had to teach through COVID, you know that the curriculum in some cases got tossed. How could you possibly fit in a whole curriculum when you lost six, seven, eight, 12 weeks of school? So we rethought that and what were the priorities? Most people came to the idea that we needed to try to think about the big questions again. Why are we educating them? What is school good for? Why do we do this, right? And we came to an understanding that basically if we prioritize things that were mental health and critical thinking, then we'd be okay. But trying to cover curriculum was not the way to go. And many teachers nowadays don't wanna go back. They do not want to be strapped to that textbook again because they realize that by teaching mental health and critical thinking, they got a lot more out of that exchange with people, okay? So curriculum questions. And the last one is time and space. And then I will open it up to anybody who wants to talk because I've talked way too long. Time and space have changed forever in education. And I think some of you are really happy about that. And some of you are sad about it, but let's see what the benefits could possibly be. We have changed from thinking about, before we used to think of education as the classroom and home work, the space. Education was space. Now we think of education as time. What do we do synchronously and asynchronously? That's a huge idea. And I hope it rocks your will because it's a really big thought here. We no longer are organizing education around our spaces, but we're organizing about what is to be done at home on your own asynchronously with the amount of repetition you need versus what is done together synchronously, okay? So the use of time and space has changed on multiple levels. Four of the ways it changed most is that the school calendar got so messed up that we were able to sort of rethink that. Should we have, what was the point of having summer vacation? Does anybody know, Heidi, do you know why we had summer vacation? Does anybody know? So you could go work the land in the summer. There you go. We needed all hands on deck to help with the harvest, right? Nelia, we had to have the students out of school so they could help us gather the crops. Well, guess what? Nobody does that anymore. Less than 3% of Americans live on farms and almost no kids have to work on the farm, believe it or not, because of labor laws and all the rest of it. And so maybe we need to rethink how our calendar goes, but that was one of the bigger things here. And if you see, there are several states that are already doing year-round learning. And it's gonna make this bigger, but I can't really see. Year-round learning basically means you have the exact same number of school days, but instead of having this massive summer break where people forget things, it's too far of a distance between learning moments, you have the same number of days off, but they're more evenly distributed. And this means that you have like a long weekend, every three weeks or something like that. So it's a basically it's a different kind of structure of time and a way to think about education in the future, which is very popular in some states already do this, okay? A second way that learning has changed is by thinking about synchronous versus asynchronous activities. And there's different things you do synchronously and different things that you do asynchronously. They have different benefits and different limitations, but basically what we do together should be to understand the knowledge of the group, right? And what you do on your own is just what is sent out in a video beforehand, right? So technically speaking, this particular encounter should have been flipped. So all of these ideas would share with you in a video so that when you come, we can pull them apart. But what we basically see is synchronously, it's much better for social interaction and group exchange. And if for content learning, we see a lot of good asynchronous activity. So basically trying to get that your head around how to devise different types of activities for different learning objectives is the key here, okay? And then finally, this idea that, remember we mentioned before, choosing what is synchronous asynchronous, what is online offline and what is basically focused on as far as explicit or implicit learning, okay? Last idea here that I wanna share with you is that our student profile is learning. And I was debating out whether to say this because I don't wanna be a Debbie Downer and make everybody sad, but this is not the last epidemic and it's not even the worst thing that can be happening to us. We're getting a lot of red flags that climate change is gonna have an impact on schools that is even bigger than the pandemic. And we haven't thought about that yet. What are we gonna do? And I would say that the key idea here is to prepare for online and be happy when you don't have to. But I would say go for the online preparation in order not to disrupt education. So many people have disrupted their education and unfortunately, UN is telling us most of those are girls in poor rural areas and they're not going back. So if we can avoid disrupting education, that would be the way to go. So another reason to do this is even bigger than what we've experienced, right? Aside from that, there are traditional populations who have been excluded from education. People who go to prison, girls who get pregnant, students who have illness and long-term illness or people who have to work to help their families and they live in states where they can stop going to school when they're 15 and so they don't go back. All of these traditionally excluded populations were gifted COVID because it then restructured the face of education and allowed many of them to go back. They were able because so many things became online, they were able to do things that they weren't able to do before. We also know that there's a lot of non-traditional learners. The average age at the Harvard Extension School is 32 years old and we get older and older students and that's normal, okay? And it's becoming a necessity. As many will show you and some of your student papers are showing, retraining and upskilling will be a part of normal life forever. The world is changing so fast that getting a single degree in one area is not enough. You're gonna have to be able to go back and get a micro certificate and something to be able to be relevant. And that's gonna happen mainly in online context. So think about changing it for nothing else for that. Finally, the potential of the Global Schoolhouse of this new reality of having a group of people from so many different perspectives is absolutely brilliant. Traditional education, at least in the United States meant you went to school in the same neighborhood you grew up in and you fought like the same people who were in your neighborhood and everybody just has the same echo chamber of reality. Whereas now you can be in a classroom with people from all over the world and that rocks your world as far as perspective taking is concerned and it changes. This also means that best teachers can teach anywhere in the world. They do not have to pick up and leave and move places. They can now be online. So there's a lot of reasons to do this. So I hope you guys will definitely think of doing that. I do want to remind you there's still tons of inequities in education. This is not the panacea. I mean, just having online does not solve everything but it helps. It helps a lot. And it has pushed a very different policy agenda and a very different research agenda than we've had two years ago. So a lot has changed and for the better. So I would just say it's a positive thing. So I will stop talking. I will ask you guys to think about this design thinking the examples of the Harvard class, these long-term changes and think if you have any reflections on this. And as you're doing that, if you have specific questions I would like to talk more globally about your projects. Many of you are on the cutting edge. You are the designers of new types of educational opportunities and I'd love to hear what you'd like to add to this because I think that our global audience would also like to hear your ideas as well. So I'll stop there. Thoughts, ideas, reflections. Go ahead, Joe. My last comment on the chat was when you are referring to the we, who is that we? And how many people nationally and globally are on your boat? And my hope is, yes, we will learn from this experience to move into a new path. But I suspect there's a lot of old habits die hard, kind of an attitude with teachers and processes and habits and institutions. So as you're saying, referring to the we, who are we talking about? And what are you hoping? I mean, what is our role in kind of helping carve out this new path, I guess? I would have to respond to that with two and two different visions. One has to do with people who imagine better education in a different way, leveraging the tools that are currently available to us right now. That's one angle to it. The other has to do with when we talk about access. So one is attitude and the other is access. So attitudinal shifts are huge and necessary. The bigger challenge we saw during COVID was access, access to technology, access to internet and all the rest of that. That also drove many other inventions that we didn't have the time or space to talk about here, but for example, Raspberry Pi. Raspberry Pi is a kind of external memory drive that basically, I watched two people in London explain how they use this in Sub-Saharan Africa and also in, not Cambodia, not Khmer, in, sorry, I can't remember the country in Southeast Asia, but basically they would put on this external drive all of the national curriculum, all of Wikipedia and all of the Khan Academy videos that existed in the target language. And this person puts it on their backpack and they walk to a village that doesn't have internet and basically puts it on to the communal, they either stand there and pass it onto phones or to a communal computer, which then serves for the next months of education. Then they walk to the next village and pass this whole thing on. But basically it drove, COVID has driven a lot of new inventions that basically say we can be independent free, internet free instruction and or access to internet. So a lot of these things, I don't know if you saw during the Ukraine war how they said Elon Musk help us get access to internet, that's what we really need. Well, boom. And then, okay, so you can't move satellites but you can definitely increase access in some ways. So one thing is attitudinal, the other is access to the types of technology that we're talking about, both are important. And I think they're both, both are huge turtles. And I just think you should feel good about being probably in the company of early adopters. You guys are people who are already, it's like preaching the converted. The people who are here probably believe that this is a good thing and that this is something you can contribute to. So I think that there's a lot of positive in that. One thing though is to have high quality positive, you know, and to use this for good. I think there's a lot of ways that this can happen. So then, so then, so coming out of this class, it's lovely to talk about these ideas and ideals, you know, with fellow lovers of this topic. But then, you know, as I come out of the screen into the community to the school and let's say we go to, you know, what do we do? What are your hopes of how we can productively help, you know, guide these new learnings? Because I suspect all the tools that you're sharing with us today of how has it changed and how is that positive is not knowledge that my, you know, neighborhood school teachers know about. So where is that productive productivity gap? How is that, how are you seeing that gap being filled? I see multiple things happening. One is a generational level, which is pretty interesting. There's an expectation by a lot of the younger teachers that this is the norm. And so there's no questioning going on with some early younger teachers. The older teachers, it's been very interesting seeing how some of them, look, we adjusted when this happened, we'll adjust with this. You know, they, good attitude and they just, you know, quick buy in and as well. I think that the first stage of all of this, Joe, and I think this is where all of us can help is at a level of awareness. Just helping people understand that education is not going to go back to the way it was and that's okay. And that some of the things that we've been hoping to get rid of for a long time, at least I have, I have a personal, you know, vendetta against, you know, standardized testing because I just think it just boxes people in in just the wrong way and doesn't celebrate all those different talents. We got to toss it during my home state of California basically says, hi, we don't need SATs anymore. It's optional. Love it. I love it. And I think that all of a sudden we were at a tipping point with a lot of decisions in education, which now we need to become aware that there are changes in education that will last forever and they can be good things and that we should be aware of the opportunities that exist and not to just rush back to the old classroom setting. I think most teachers have realized, hey, well, it was actually better when we had our weekly staff meetings online and I didn't have to stay after school. I mean, a lot of teachers are figuring out there's good stuff anyways. And so helping them, you know, categorize all the good things, all the tools, all the evaluation changes, all of the deeper learning that can happen, you know, occur, being liberated from maybe a curriculum that sometimes keeps getting bigger and bigger without actually getting deeper and deeper also became part of that. So hopefully becoming aware is the first step and we can all help with that. Yeah, continuing that conversation. Exactly. We'll send you the link to this video and I'd like you to encourage your school to look at it. Yeah. Okay, Alyssa, go ahead. I wanted to ask you to elaborate a little bit on the things that are worth teaching. You were saying mental health and critical thinking. I kind of like, I understand the idea. I totally kind of get it. I'm curious as to how can it be implemented in a specific subject? Kind of just a couple of examples as I'm not a school teacher, it's harder to understand but also I'm very curious how it can be done across different subjects. So it's like, if what's worth teaching is mental health and critical thinking, how do we incorporate it in teaching math, history, some social sciences, biology, but also how do teachers of all these different subjects on a school level also do it so that it doesn't repeat itself but enhances itself? So some of the best lessons that came out of this kind of interesting to see, there's two different focus, the focus that we could have. One is on the how instruction occurs as opposed to the what, right? So the what of which subject matter is less important than how it was approached. So basically teachers getting in on the ground floor of putting the pulse on the group and how is the group feeling and doing before launching into the class. Basically, I do care about you. I care about you mentally, physically, okay. And then getting in that good space mentally before launching into higher order thinking. Prioritizing, basically understanding that there is no cognition without emotion. Understanding that some kids might be vulnerable was a huge thing. The other, the key word I would say to the second part of your question is in co-construction. Co-construction of the curriculum. And this is basically a teacher who knows clearly enough what their objectives are, that they are open enough to be able to say, by the end of the semester, I really need you guys to be able to look at a passage, synthesize it well and produce on that AP test, okay. And I don't know how to get there because normally in the past, I'd have you read, you know, these seven novels and we dissect them together and we write short essays and you practice but we've lost eight weeks of school and we can't do that. So what can we do now? And you see how they launched into that, how they actually were able to do that. Others used leveraged technology and tools. Science teachers basically discovered ways that you could have, they did invest in things like lab story. You can have a, every kid gets a million dollar lab. So kids, and this was actually fascinating because kids who didn't have good lab space in their regular schools who got virtual simulations of laboratory work actually end up learning even more because it was better than going to the Junkie lab that they had, but they had these simulations that they could actually do or doing things or creating the structure where they could use multiple entry points to the same piece of information. So for example, having the math objective be whatever trigonometry or whatever, but then you have a student play do gamification to learn certain basic concepts, right? Or you have them watch the Khan Academy video to get other things or you have them take a picture in photo math to get another explanation or you see the multiple ways that something could be resolved as opposed to one teacher telling you one way to resolve it could actually have a benefit there. Now there was a very strange finding and we'll have to still parse this out because I have my own personal theory here but for the first time, very, very strangely, the IV basically everybody had better scores despite the fact that they couldn't, they basically said, focus on mental health. And for some reason they had better scores than they've ever had in the history of the IV. I have my own theory that maybe people were being more easy on the scoring. I don't know, but it could very well be though that focusing on mental health and then talking about how to think, how to approach problems writ large as opposed to the very narrow curriculum that they'd been receiving was beneficial. So those are some examples. I don't know, there's dozens more that we can talk to you about with Cynthia. We looked at kids from kindergarten. We worked with a whole group doing kindergarten through, how old were those kids? Is it kindergarten through eight years old or kindergarten through six years old? Cynthia that we're in Florida and in Spain and yeah. Eight maybe, no, yeah, eight tops. So we've seen them the entire age span of things. I think one, so one idea is co-construction of curriculum. The other is leveraging your tools. We just became aware of so many additional ways to do knowledge construction. So it ended up being much more enjoyable to the kids, greater variety of tools, but also what resonated with one kid, didn't resonate with another, but there were so many tools out there that they were able to then approach the information from multiple angles. So I would have to say there's definitely good examples out there of how you could approach it. Yeah, okay. Alipa, did you wanna ask something or say something? Yeah, I was wondering how, why do you use this format where we can watch ourselves on your screen? Actually, I do that for the benefit of the post video because then we can see who's there and we see the content of what was shared and we also see who was there and active and all the rest of that. But, so it's, we turn it around for that. It makes the initial class. You have to drag one of the sets of people off to the side, don't you, right? Yeah, yeah, yeah. It's a strange thing, but I just wanted to, people ask this a lot at the beginning of the pandemic because I don't know other classes have figured this out, but if you do your slides, if you, when you're setting up your slideshow, if you go to set up slideshow, right? So you go, oops, you go to the slideshow and then you go to set up slideshow. Okay. And then you do the second button which is browse by individual window. That will allow you, so it doesn't just eat up your whole screen when you make the slide. So you can resize the slide deck the way you want to. Oh, okay. Yeah. And then that allows you to also see everybody, you don't waste time taking roll or, well, Harvard does that automatically for us and gather, but I mean, all the silly things that some people used to care about a lot, you know, it's automated care, okay? Mine's a real bushy sleeping. Yeah, I guess from the student's perspective, it's kind of cool watching myself in class. So like, I don't know if I thought you had a, you know, there was a method to that like here, this is what you're doing. So I don't know. If you want to hear the long backstory of all of this, my dog has done crazy because she has a timer in her belly, I think. Social contagion is the key and this is something we only case. So that's what I was looking for, yeah. But really fascinating thing is you see Ann who's nodding and smiling and that makes Amy nod and smile. So Megan is smiling. So there's a social contagion that actually connects. And technically speaking, if the teacher understands that, they drive the social contagion. So the teacher, if the teacher is there and absolutely dead boring and hates what they're doing, you know, poor class, but if the teacher loves what they're doing and they're jazzed by it and then their passion for their subject can be passed on, that also continues this virtuous cycle of learning. And so definitely there's a benefit there. Yeah, I'm gonna steal this too. Do it, it's not stealing. I'm just giving away, it's giving away. We wish everybody, when we asked Harvard if we could convert this to another platform, they said, please, we wish everybody would learn like this and that was wonderful. And so we're trying as best we can to give this all away. If you like, you can talk to Cynthia and Michelle afterwards. We actually created a whole template in Moodle because it's free and that can be given away to people. So people can construct their own classes with the basic format of our class in Harvard, which we have the flipped part, the quiz part, the bundle part, all of those things are already set up so that people don't have to go through the hassle of doing that. And what we do in the course with them is help them populate, that help them come up with good bundles, help them come up with good prompts and things like that on questions. Okay. Thank you. Good. So we're getting closer to the hour and I just look with the respectful of your time but I will stay as long as you guys need. But I just wanna hear, is there somebody who I haven't, Peter, I don't know if we know each other. Do I know you very well? You have a question, Peter? Or anything to say about this? Peter DeCannon? No, thank, I have no question. I'm just happy to hear these two hours and quite nice to have a lot of information already some of them heard last year in one compact package. It is very nice to have it in one compact and actually it is nice to look at the pupils holistically. This is the main message from this for me. Very, very thank you. Good. I think, and where are you in the world? Are you, is that CERN in the back of you or am I looking at Switzerland? Where are you, Peter? I'm guessing. No, no, I mean Slovakia, Bratislava Slovakia. Ah, okay. Well, I'm very happy that you're here with us. Thank you. So I think that in this sense, I think that there's so many different things that we hope, Peter, Alyssa, maybe we'll talk to you more if you wanna hear about the role of boot camps and upskilling and retraining. If there's anything that I've learned from looking at my students papers is that the format of education is changing drastically. And if the objective is to share information and perspectives, I think this is a good format. If it's actually to really digest it, we need to step back and have a longer exchange. But I think for beginning a conversation, I think this is a good start. So thank you for being here. And is it Dasha? Yes, hi, thank you. I'm also from Slovakia and... Oh! Done. So, but I'm writing my national thesis and it's about five pillars applied to this curriculum. And I was just wondering, because I saw this picture where there were like a knowledge that was in the biggest cycle, then there was like processes and then attitudes. And I was just wondering about that picture because when I look at it from us, it's like the knowledge is like the prior and it's the most important. And as the cycle goes slower and slower, it seems like attitudes are not that important. So I just wanted to hear your opinion about the debt and to explain it a little bit deeper because that doesn't resonate with me. Very good. The whole idea of knowledge, skills and attitudes is a very fundamental concept. What I would actually join to this idea, it's important throughout the lifespan. And in fact, I think attitudinal shifts are probably, this is really the heart of all learning to me. I can get dates, facts, formulas, knowledge off of a telephone. I don't need a person. But what's so interesting is you cannot devise a change or shift in values or attitudes without another person. So I do think that that's definitely important throughout the lifespan. I would put the five pillars concept closer to the radical neuroconstructivism, mainly because when we talk about how you construct knowledge, for those of you who don't know the theory of the five pillars, when I was sorting out a thousand studies on math and language trying to understand how the brain was working, I said, I made five piles. I said, okay, this is about symbols in the brain. This is about order, patterns, categories, and relationships and everything fit. Every single study I could find fit into that. And so basically, that's a theoretical basis that I have here, but basically we have disaggregated, Basha, the all of math and all of language into the five pillars in this neuroconstructivist way. So basically, if you were to teach to that, you would never have a missing gap in knowledge. You know how kids are missing core notions about math or about language, you couldn't have it if you used neuroconstructivist design using the pillars. If you'd like to talk about that, we should meet up a little bit later. And also congratulations to your very brave president to go to Kiev. I'm not sure he and the Czech president, I believe, are going to actually make a stand and physically go there. So thank you for that. Are you here because of Andrea, the ambassador in the United Nations? No, she's also from Pennsylvania. I'm here because of my professor. Oh, thank you. Okay, okay, how interesting. Okay, well, I'm happy that you found us and please stay in touch. Definitely would love to support you in any way we can. Okay, so Amy, I'm sure as a designer yourself, Amy is creating a gamification to actually teach and try to see if you can remediate skills of a person who might have had difficulties growing up in low socioeconomic background. Could you remediate skills through gamification later in life? And so I know that you know this very well, but is there anything you'd like to add to the information we had today, Amy, that people can take away from? Nothing to add, but I was wondering what would you consider the most challenging time during your teaching online that you did not encounter teaching in person? That's a great question. I don't know, I have to say when they asked us if we would take our course 100% online, I immediately said yes, I loved it. I thought this was the best way to have a bigger student body from many other places around the world. And I thought that it was absolutely positive. I love my face-to-face encounters, but I have to confess that there's so many things we do face-to-face that aren't necessarily the most efficient use of time. And I guess I always walk into a situation and say, what is the objective? And Cynthia knows this from our days working in other universities. I would say, you know, why are we here? Why do we need to be here? I mean, what is the objective then we should have another format. So I do think that there's some real big benefits to being online. I don't know if I've had, I'd say the biggest challenge is the best opportunity every year. And that's that all of you guys are different, which means every single year we have to pick our class. It has to be different because the group is different. And that's hard, but it's also the most interesting part of the whole class. It's the most fascinating. This is why we can teach it year after year and never get bored because this is just, it's so interesting because you guys all bring very different combination of perspectives. So I don't know, Cynthia, do you have anything that's been hard? Remind me of all the things that I think are, that I should remember are hard and that I've blocked out of my mind and don't think they're hard anymore. I mean, more than hard. Yeah, it's always in a challenge, but it's, I don't know, it's so much fun. And you learn so much that I wouldn't see really anything that's hard. I guess the first time I was working on HTML codes and Canvas and Moodle, that was hard. I know, I know one, I know one, I know one. My hardest point is, and it happens every year, Amy, is that I will inevitably in week one or two of the class have a student that writes me that says, okay, but just tell me, how am I going to get the A? And it drives me absolutely. I mean, Cynthia can hear me scream. I just, I say they have missed the entire point of this class, which is to learn how to learn. And if the good grade, the good grade will come along if you have done that. But when they only care about the grade, it drives me absolutely bananas because I don't think that they get the whole idea. So that would be the hardest thing I think on line is that every year, inevitably, I have one or two students at the first, or a couple of weeks that will always write me a side email and just say, so I'm just a little concerned here. How am I sure I'm gonna get on A? Just tell me, basically saying, how can I bribe you? I was like, no, you can't. It's all on you. You get to, you earn your grade. I don't give it to you, you earn it. And so that's a challenge to me because it just is a different objective. They just want to get a grade and I want them to learn to learn. So that's the hardest thing I think every year or so. But thanks for the question. I will keep thinking about that. Dasha, did you want to say something else? Yeah, also maybe just a little bit ground. I'm studying in faculty of physics, mathematics and informatics in Andrade Slava. And we use kind of a variety of digital technologies and we learn how to apply them in education and everything. But I keep wondering, can it all be transferred to online? Can we teach kids how to measure things like in physics? Can all processes be learned just online? That's what I've been wondering for the whole time. Yes. I mean, I can't think of anything you can't do online. In fact, the hardest thing came from a student I had two years ago from Russia who wanted to know if you could teach growth mindsets to adults through gamification in an online setting. And we discovered through the research it's already been done, but it had never been done in Russia. So she actually changed, she created a whole design of how you could do that in her context and could show that something even like a growth mindset I can believe that I can learn. I am capable if I just try harder, I can do this. You can change adults, which we thought, okay adults are fixed, they don't change very much. You can change adults mindsets through gamification in an online setting. If you can do that, you can teach physics. I mean, I don't know that there's any limits to what you can do online. But I'm sure that there must be some things people always ask me if you can have this closer emotional bond. What I found during COVID, which was really striking to me, I had, I believe it or not, I had a teacher about two months ago say, oh, we get to go, we're going back online. And I said, you're happy that you're going back online. And he said, yes, we've had an outbreak of COVID, we have to go back online. I can't wait to see the students again. So what we forgot is that when you're with COVID and you go to school, everybody was having masks on and you didn't see anybody and they were sitting far away from each other. But when they were in Zoom, they could see each other. And he says, oh, I was so happy to see my students again. And so I realized that this idea of social contagion or getting to know your students better can actually be additionally beneficial in these settings as well. So I don't see many limitations. I'm sure I'm wrong, but for me, I can't imagine anything. I've worked with a lot of teachers at all levels of education and all subjects. I don't know of one thing you can't teach online and do it successfully, but I know that you need different tools depending on the objectives. And so that's the big choice there, I guess. Go ahead, Nancy. I just went to, I don't know, tell you that we'd really need it. I was craving for a change like this for a long time ago. And I was not thinking about something, of course, like so terrible to happen to us in a way. But I was thinking about when it's going to be the time when we are going to be like using computers all the time, like that the people can have opportunity to learn those skills that for me, like for many of us, like if you go to a university, it's very basic in a way that you have the tools, you have the knowledge, like you learn it, right? Like through the experience and everything, but other people didn't have that option. And even we were fighting so much inside the organizations, just thinking about like ways to improve certain areas. And I was like, okay, good, why don't we bring the computers that are like very in a closet that you can just sign up for like the computers once every week. And it's just, it was like amazing, mind-changing. I'd really love it. And I really hope that we don't go back to the old time because only if we go, they just bring the whole computer and technology with it. Because- I think that you're touching on a huge idea that we've been waiting for these things. But also, I think Yula would speak to this as far as the quality of what gets shared. So on one level, there's a very, very positive thing of the democratization of access to things. For example, during COVID, all these art teachers realized that you can visit any museum in the world. There's all of these websites where you can have a walking tour through all of the museums in the world and see the artwork up close. And you can analyze things or that you could see, I don't know, 10 different people resolve the math problem in 10 different ways. And all of these things became maybe an even playing field. People who didn't normally think that, oh, I have to listen to my teacher, they understood they could listen to anybody and they might be better off. But I think Yulia was pointing out to us that there is also a danger that sometimes people who know how to leverage technology well, who are very slick and good at presenting, oftentimes will give the wrong information. And so sometimes people, and they choose not to go to formal universities or take a regular class because it's easier to watch this kind of dummy down version of information. I'm not sure if I'm saying the right thing Yulia, was that where you were going with your topic? Oh, which one? Well, with your topic about thinking that some people just follow, I mean, education could be going in the wrong direction now because so many people, anybody can present anything online. And so many times people are following people because they're popular and not because they know what they're doing. And you are concerned that that's also the flip side. You have free and access like Nancy is celebrating, there's so much good stuff, but there's also a lot of terrible stuff out there too, right? I actually changed my perspective to think about that this really tool and you choose like which vagans to follow like this one or the right one. And for me technology, it's fascinating. It's a lot of, it's getting our life easier. And I don't know, I excited about it. The only thing it's bothering me is what gonna happen with our brain function with using phone more. And also there's Robin Dunbar number. He said that we have only 13 to 150 relatives, our brain could feed. And now it's becoming like globally, globally with all the social media and probably it's not gonna feed in our brain. So there's study like that. Well, we know that we don't know how much of our brain we use. We know when nobody is using all that they could. And we also know, which is very interesting is that, remember we said that you can't multitask, right? Your brain only does one heavy cognitive load task in the time, which means what's very interesting about phones for example, I can remember my phone number from when I was a small child, but I can't really remember my phone number right now because I don't need to because it's there. So what's so interesting is your brain adapts to what it needs to adapt to, right? And so I don't need headspace to memorize all the presidents in the United States because I can find it, right? So does that now create an opportunity to use my brain for other things that might be more demanding and higher order thinking, right? And so maybe I am offloading some of the lower cognitive things to be able to do harder things. So you're absolutely right. We're in the middle of it, which makes it really hard for us to assess is this good or bad? Because we'll have to have some hindsight at some point that right now moving out of COVID we're beginning to now have enough distance to see what did we do? What happened and what is remaining? Because of these changes. So it's an exciting time. I hope all of this, and I'm glad that we're videoing for documenting this moment in history because it is really a huge sea change in where education is gonna be headed. It's a very big moment. It's so interesting about our brain is adapt to what it does the most. For example, the past week was super stressful for me and I has to be with phone like 24 seven. And today in the morning, I was brushing my teeth and I realized that I'm scrolling. Like, I can't get out of it. And you remember in our class, we talked a lot about the benefits of different levels of consciousness and going into default mode and mind wandering and how beneficial it is. So you should, if you need to, I know it sounds crazy. You should program an alarm on your phone to tell you to put it down and get some distance from it. I think that would be beneficial. Sal, did you want to say something? Yes. Hi, yeah. Thank you very much for the great info and knowledge. Also, I wanna thank you that you mentioned the pandemic revolution for the education. And just like there's always like a negative side and there's a positive side to it. And I hope, okay, the online, the distance learning keeps revolutionizing the education system. So I wanna add like a couple of things that like maybe on top of like this great knowledge that sometimes like I feel like we forget about the simplicity. And also like my motto is less is more and simple is the best. And otherwise I get lost and I get confused. So that's my case. So coming from different backgrounds and also as someone that who doesn't have an educational background like back to back, like a soldier. No, I have lots of gaps. So if the knowledge information is presented like in big chunks, I get a mental indigestion like it can process. So it's no help. So I want to share that one. Also another thing that I wanna quote an experimental psychologist from NYU, Sertuk Şehrin that who did a great job in Turkey for the Syrian refugees that like he mentioned that these refugees, they're staying in Turkey but like we feed them. But if you don't feed them like mentally in the future, things can go like not straight. So, and also he mentioned that the education system comes with lots of problems because the education system has been around like for years and years. Even in the Sumerian tablets, there's a note from a pupil. And he says, like he made a note on a tablet. Said I've been going, maybe he was around seven, eight years old. I've been going to school like six days a week, loss of assignment. It was almost 4,000 years ago. And you know, damn, I miss my mother, things like that. So I think it hasn't changed much but Sertuk Şehrin says, education somehow gives us structure, structure to build on. So like in my case, structure is good but in many cases, like from my experience, I feel like we keep forgetting the infrastructure. I guess it is more important than the structure, I believe. So I want to share these aspects of education and thank you very much. Thanks, Saul. And I think several different ideas there. One of them is being able to take in the information at your own pace is very important, which I think can be leveraged correctly within the technology structure. The other thing about, and I love the idea of this mental overload. I think that's mental indigestion and like that as well. But you're right, this downtime and actually creating those structures is an important idea, which I think maybe you'd find rewatching this and seeing these ideas about instructional design. You know, the choices that we make about where we put emphasis. If this had been my choice, but it was your holiday, so I didn't want to make everybody do another flipped video and all these other things. We decided we'd just do this live and open it up and let anybody come who wanted to come. But my choice with this kind of information would be to have chunked it in three stages, three parts, sent you a video on each of the parts, you watch it beforehand, then we come and we talk about it. That to me would be the ideal way to do this. I would flip the conference would be the best way to do it. So I do agree with you that there's a better way to organize learning moments, but thanks for that. I appreciate it. And I just wanted to say thank you, Monica, for also for being here. Did you have anything you needed to say or comment or observe or anything you want us to think about? Oh, you know, I have thoroughly enjoyed being you Chafee and Cynthia. It's so nice to, you know, just come back to this. But I did actually have a question on how you measure attitudes, I guess, like a good way of being able to tell that there's growth and like what is your metric for that? That's a huge question. And especially when we're talking about, you know, which tools they do use. So if you're gonna measure knowledge and skills, you know, you can measure, you can use a multiple-trace test, but when you measure attitudinal shifts, what can you do? The only thing you can do is have time. If you do not know the starting point where somebody began and where they are now, you can't measure an attitudinal shift. So does this person seem that they've grown in intellectual empathy or in perseverance or whatever? I would say do a two-by-two kind of a thinking. On one scale, you're measuring the product, the process and the progress, right? But the other, and you can't measure progress if you don't have the baseline. So I guess that's very important. The other thing, it also depends on the precise attitude you're working on. So for example, if somebody is working on, you know, oh, I want them to be, you know, to persevere, to show grit, okay. So you measure it in one way. But if you say, I want them to come on. I want them to have, I want them to understand the, to value the land or to value their home environment or the earth or whatever. It's a very different thing, right? So understanding the subtype of attitudinal shift you're looking for can change the evidence that you're willing to accept, right? This is why one of the things that seems to respond to all of that, knowledge, skills and attitudes is documentation through e-portfolio entries. So having a digital portfolio in which you could actually have, you know, personal reflections or journal entries or things over time or observations that you have of students interacting with each other. That's a key way to actually measure, but depends on what attitude you're looking at, right? Did you have a particular one in mind that you were having, you're thinking about? Well, I'm in human resources now, but we're looking at the transdisciplinary goals in AOLA. And so they're like Aloha Aina. There are some soft skills that we're also looking at, which, you know, are like servant leadership, creativity, perseverance, things like that as well. So as a goal, though, we get to pick one and then based on the one that we picked, we have to go on a journey, but I do know like the people that we're working in with in human resources, they're not as, they're not as able to make that shift in measuring growth. So that's a perfect example of, are there 17 elements of the AOLA learner outcomes, I think, I think there's 17, right? Yeah, okay. So one of the things that's really that, I think this was an activity that we'd recommended at one point, I'm not sure where it might be at what stage or which school, but it was to look at, identify the AOLA learning outcomes. And then this is for those of you who don't know the Kamehameha Schools in Hawaii, it's kind of like their mission statement, but these greater goals that they have for their graduates. So one of the things that we talked about was actually developing, if you created a rubric structure where you said the criteria are each of these different AOLA learner outcomes, okay? And for each one you have product, process and progress. Then together, you work with a team that's going to be using this, that it's gonna be measured against this, you develop the criteria with them. It's very powerful to develop the rubric, the judgment, the way you're going to be graded, it's very powerful to do that with the learner. So the person who is going to be under the, be judged by that, if they can talk through, well, what does it really look like to really understand what you really understand? So that would be your product. But, well, where have you come from in your understanding? That's your progress, but how did you actually get there? That's your process, but how do you describe each of those little pieces? So it is a bit more work. It's 17 times three different things that you have to talk through. But in developing that, what's really interesting is the people who develop the rubric are more likely to comply. They're more likely to do it because they own it. They actually contributed to the criteria that they're gonna be using. So that would be, to me, that's the part of the co-construction we were talking about before, where you actually do this with them because imposing an external measurement and then you saying, okay, but this value is how I see it and that's how I'm judging you. They said, well, I have a different interpretation. So unless that comes out first, it can be very difficult to measure attitudes. You're absolutely right. Thank you, Tracy. I really appreciate it. Oh, no, and so happy that you came. It's so wonderful to see you. I really appreciate you being here. Thank you. Dasha, am I saying your name right, Dasha? Is that right? Yeah, that's right. I just wanted to really thank you for this meeting. It was really beneficial for me, also for my patrols as is. And I just wanted to ask you whether the recordings of this session will be available because I wanted to share with my classmates, probably going to present it tomorrow on a class. So... Oh, great. Oh, very happy. Yes, Cynthia, what's the best way for her to get the link? Or is it already gonna be up on the Connections Facebook page or something? It will be on the... I'm actually opening our page, the Connections YouTube channel. Do you want to share your page so that she can see the address? There, I just pasted it in the chat. So it'll be available there. It'll probably be there by tomorrow morning, I'm pretty sure. But yeah, check in tomorrow. Yeah, okay. And Anne, you've been smiley, but quiet. What's up? No, I've just been enjoying the conversation and so much of what's been shared. I did have a question, and I'm sorry, because it's so late, but if you could address it maybe another time, even, but in the idea of an e-portfolio and keeping archives of, keeping track of growth, I love this idea, cross the board, but in my experience, so often tools that were intended to help work against us. And so how do you ensure that it doesn't become the permanent record that I always lived in fear of as a public school child or the CUME folder, where you have an instructor whose perspective may be uninformed. And so you get that pigeonholing or labeling of a student instead of it being used as it's intended. The key, I was just looking for a graphic that I have of this, but the real key is that the owner of the e-portfolio is the student, is the person, is you. So you get to choose who it's shared with and you get to choose the content. So the idea is, and then some teachers will encourage their students and say, I want you to put everything from your very first draft until your final project into your e-portfolio so you can just see your growth. Other teachers will just say choose your best piece and put it in there. So it can be guided in different ways, but the owner of the e-portfolio is forever yours. And that's a very interesting concept that we're working with in Connections is that the teachers that do our trainings or whatever, they get an e-portfolio and it's theirs. We don't touch it, do anything with it. They share with who they want to share. They share which folders they wanna share. So the concept here is kind of like within school systems, if you had with younger kids, what's really interesting is if you have, the more people can weigh in on things when they're in a shared digital format. So parents who always say, well, how come my kid is doing this and the other kids aren't? So you know, but look at where he's come from and show that, measure the kid against the kid, let the kid be his own point of reference is a really strong thing. So e-portfolios have the benefit of being shared with a wider audience, but they can also technically follow the kid throughout school. It's almost like what's happening to medical records these days. Medical records are with a doctor, which is like such a pain because how do you get your dental record? I was trying to get from the dentist in Ecuador to the dentist that I saw the first time versus how crazy is that? I should have my records. So I can go wherever I want and I can be able to. So with health now, we're seeing the shift that there's now this idea of being able to own your own data, right? So in a similar way, it doesn't have to, it can only be beneficial to you because when it comes down to it and you're about to apply for a job and you can show the photo of you when you were three years old teaching another kid letters and they say, hey, you've been a natural teacher your whole life. Well, you get to choose to share that picture or not, right, it's yours. So the idea is the data belongs to the individual. So it's not something that a teacher will choose to share or a blacklist a kid for or whatever, it's you own the data and you own permissions to what is shared. Yeah, does that make sense? Yeah, okay, good. And Saul, you wanted to say something? Yeah, I just wanna confirm that like e-portfolios like e-portfolios are like one of the best ways for someone to express themselves. So like in my case, from experience like when I had my master's degree, there was a class, it was like one credit it was called portfolio class. So we were supposed to like do our best or whatever, you know, like throughout our studies get the best out of like maybe a couple of works but it was kind of like here and there on a paper was kind of we had to print them out and there was much distraction, sorry, instruction. However, afterwards I changed majors and maybe probably after five years I also thought about like that, like I look for my works you know, I had grade writing sometimes once in a while and I will like on papers or a server and I couldn't find them but when I started LaGuardia Community College which is like, you know, it's Community College. So they had the first year seminar which I had the benefit of teaching that class and it was based on e-portfolio and I said, that is great and that is one of the best I could say. And as you mentioned, you could take it anywhere it's transferable and it's portable and also like as someone like your, if you're on the e-portfolio you can do whatever you can make changes right away on spot what server, therefore like in my case I'm trying to go for education psychology to kind of like revolutionize the education some in my own ways. And I believe that the e-portfolio is gonna be one of the essential infrastructure of the education system. Therefore I just wanna share my link but it's just like a start it's not like down what server. So like kind of like named as e-portfolio Academy I had like options university school what server Academy like made more sense and my model is like show up for yourself. So it is just like in the link. That's it, that ends up being and this is the problem we had with some of the schools when we tried to talk about e-portfolios was that some, the kids didn't get jazzed about doing it because they already said, yeah, but I have my Facebook page or I have my whatever page. So they didn't, they thought it was redundant which I thought fascinating because one was academic and one was not but they saw too much crossover. And so that is a very interesting thing. So when does it end up being something that's personalized and when is it something that is mainly for academics or just health or whatever? Do we have multiple e-portfolios of ourselves or can be on other platforms? And so that was something that's worth rethinking about whether or not that is something that should be done. I mean, what is the format of that? And Nancy, I'll let you have the last question because actually I do teach another class today so I do need to get something to eat before that. So go ahead and then I'll look at it. I'm studying my class also. Yeah, I got like extra time during the weekend to plan this class ahead. So I was supposed to plan it like right now but I'm all done. The question that I have is there is a possibility that after product process and progress we can have productivity? I like that, I like that. That's very nice. So that's almost that whole idea. Remember we keep talking about feed forward. What will you do better the next time? So that might be to me after the rubric has been completed you sort of say, okay, now after that self-assessment or after that assessment by others, now what? What do you do with that? Now that I know the product process and progress where does it go from here? And this is what I love about the whole feed forward concept is, and I think that came out of the Kamehameha Schools discussions, right? It's like, what do I do better the next time? As opposed to, oh, look how terrible my past was. It's sort of saying, now that I know this where will I take it? What will I do better the next time? And that is a very beautiful proactive. I like that, that's another P that could work there. But sort of at the end of the day now that you've taken that in, what do you do with it? Love it, very good. Thanks for that. That's a great note to end on. I appreciate that. I was thinking about that. It was very good, I love it. Okay, well, I'm so happy that you guys came and it was very good to see you again, Monica. And I appreciate you sticking around, Amy and Yulea, until the end of this. And so if there's anything else that we can do to support you guys, do stay in touch. You guys all have our emails. So make sure that if there's any follow-up questions or follow-up ideas that you want to speak about, do let us know, okay? All right, take care, you know, run, Monica. Okay, talk to you guys later. See you then. Thank you, thanks. Thank you for this one. Was that, oh, are you gonna do that other one? I don't know how to do that.