 Greetings everyone. Welcome. This is day it's so exciting. I've been at MIT since 2012 and I've been to a fair number of these and it's always exciting. So glad to see you all here. We have people coming in and out throughout the day both in person and via zoom. And again, thank you all for coming. We're going to stick to a really rigorous schedule. You can see the clock in the back of the room might be helpful. I think it's about one minute fast. But that's okay. So each presenter will go for 20 minutes and then it will have 10 minutes of Q&A and then a super quick thing over to the next presenter. And we'll be breaking for lunch at one o'clock. So our first presenter today is Marvez and the title of their thesis is controversial science argumentation skills for teachers in the digital clinical simulation discussion leader. Take it away. Thank you, Heather. I will share my screen. Can't do anything about the bar on the bottom. Hope you. Hello, everyone. My name is Marvez. I'm a graduate students in comparative media studies and in the teaching system lab and you'll be talking about simulation I designed my thesis. Designed to help teachers practice leading controversial discussions on issues with their students before they actually go and implement this lesson and I focused on the ethics of gene therapy. So why are facilitated controversial discussions in school so important. I as a first year student teacher I was 19 I was in an 11th and 12th grade AP English classroom and some of the students were the same age as me. And in that classroom my mentor teacher I worked with cared a lot about debate. He did this one exercise where I believe students read a bunch of pro and con articles about recycling and for obvious reasons all the students were pro recycling. And then he took on the con perspective and argued one against 20 against all the students and being 19 brand new teacher I thought that was both really exciting and terrifying and I'll be really interested in controversial discussions. A lot of people think controversial discussions are only a like social studies or a history class type of activity but I think it's important to do in all the subjects. And the subject I focus on for this simulation is on gene therapy right here at MIT we are designing the gene therapies of the future from new MRA. Just recently in the most recent issue of science, we've learned that a consortium of scientists have almost completely sequence the human genome. And we're also having difficult conversations about who should be the stewards of the human race. When we design your gene therapies we're deciding where humanity will go and so I think that these discussions are really important for high school students to have to be the next generation of researchers designers and innovators. And I'm going to come back to these research questions later but I would like to briefly mention them here. I'm interested in mostly two things how can we increase teachers comfort in leading controversial discussions through a simulation. And through the simulation can we study the teacher dialogue choices that they make and how those might different from those with different levels of experience. So this is a brief agenda of what I'm going to look at today I'm mostly focused on chapter three and chapter four which is my methodology and the results. So first I would like to introduce a couple problems of practice and teach education. First is that teacher candidates rarely have the opportunity just to practice one element by them by itself. And they're evaluated on their ability to combine all aspects of teaching at once this is just part of the Massachusetts rubric for assessing student teachers. As you can see there's a lot of things in just this one bullet point that teacher. Additionally for classroom controversy a lot of teachers report that there are significant challenges to them teaching controversial issues such as a lack of comfort. Concern about pushback from parents administration and their top security and a lack of instruction on how to teach controversial topics. Even despite these challenges teachers report a strong desire for academic freedom in their classrooms and they believe that teaching controversies are critical to democratic citizenship. We know that the classroom is one of the few places where students hear ideas outside of their ideological bubble so we really need to foster this type of environment. So if teachers comfort level is a strong predictor of whether they even engage in classroom discussions this becomes before they plan to do a discussion with their students and they actually implemented. When they do implement it there are some elements of a good discussion that can be practiced such as enforcing class norms. And there's also recent challenges that have become a lot more difficult over the past few years such as where does the teachers opinion belong in a discussion and how can a teacher manage misinformation. And so it is on these four topics that I see that there's an opportunity to design simulations, not only do you need a skill at leading difficult discussions you need to rehearse your knowledge on the. Enforce classroom norms and also work on your digital literacy skills with your students to combat this. And this is where we have the opportunity to improve the practice of improvisational interactions that we'll see in the simulation. So for at least the last 60 years we've been working on the difficult problem of getting teachers to practice difficult elements of teaching in different technologies and mediums. In 1962 curse described the classroom simulator which is a projection environment which teacher would stand in front of a projector watch some classroom moments, and then a panel of experts would decide if the teacher responded appropriately and queue up the next video sequence. What was interesting about this from the 60s is that you had to do it correctly. You didn't do it right, they would make you do it over and over again. And in the 70s to 90s with the rise of home computing we see a lot more computer games for teaching for teaching teachers how to balance lots of classroom task all at once. What I'm affectionately calling TSL at all from 2019 or 2018 to today is this platform called teacher moments, which is a digital platform to create simulations about teacher education, especially in topics of equity and around difficult elements of teaching. In teacher moments you can do linear scenarios branching scenarios like I will describe here. And use audio text and AI back into capture data. And now I would like to talk about specifically the evolution of the discussion leader suite and discussion leaders a type of practice space specifically designed to help practice leading a controversial discussion. Our original version was a linear implementation made in Google slides where you would put in what you say and no matter what you said, the simulation progressed the same way about immigration policy. We eventually moved to twine, which is a choose your own adventure game engine, where we created this branching math branching path model again about immigration policy with over 100 paths and about 12 different endings. This is due to a lot of the feedback and iteration we got from play tests from people who are like, I want this to be more responsive to me and then in our time implementation we only had three or four students in every scenario and we were told that try to get as many students as you can. So we thought about how can we get as many student voices in this discussion as possible without making it to computationally. So with teacher moments being able to take in branching path models we're able to have 20 different student voices in our discussions and we piloted one version about freedom of speech in a loop. And then the current implementation I'm talking about today is a branching path model on gene editing ethics this one has over 4 million paths 21 to the fifth possible options. So there are challenges centered around the elements of a good discussion five different student groups you speak with, and at each group the difficulty gets harder and there's a different field. I'd like to bring up our research questions again I'm really interested in how we can improve people's comfort and we do this through a self report survey, and then looking at the teacher dialogue choices and how they differ by analyzing the past. So in this simulation participants take on the role of a high school biology teacher near the end of a genetic student they're called that students have read two opposing articles for homework. One about cystic fibrosis treatments and one about editing embryos and they're discussing this broader question. If genetic modification treatments in humans should be allowed to what and why and so they have a class of 20 students into five groups of four and each group focuses on a different field that I will show in the next couple times. These also align with the Massachusetts State Standards for high school science so this is an actual lesson that you could teach with your high school students. So the simulation flow is that the participants see the same introduction that I just gave you I collect some data and some demographic information and then they enter they answer some anticipate questions about the possible discussion strategies they're going to utilize. And then for the five groups they speak to the student group and make different choices reflect and then they reflect on the simulation as a whole. So these are the five skills I'm having teachers practice during the simulation. And these are in order of increasing this practice asking probing questions not asking these simple yes or no. Group two focuses on exploration versus telling students in this group want you to want the teacher to give them the answer so how do you work through the problem with students. The third group is practicing whether to disclose a personal opinion to students the fourth one is having very low engagements and how do you work with students and the fifth one is working on digital information with your students. This is a visualization of just what one path could be like through group one. On the left you see the original bit of student dialogue that participants will see and they make some teacher dialogue choice. They get new dialogue to make another choice they get new dialogue and make their final choice and they get a final bit of student dialogue that acts as a bit of feedback for them for how they did in the scenario. And I'm going to be representing these paths that people make using these three diagrams. Everyone starts at the black start node and then based on their first level decision is put into one of these possible 21 paths. And this is what it could look like for someone who goes through all five groups and how they end up on different paths but go through all of them in the same order. So I got 42 people to participate in this experiment. There are undergraduate and graduate education students of fellows in the community to teacher program high school teachers and college professors. And I'm interested in stratifying the data out by how many years of teaching experience people had and there's a pretty good spread between non teachers novice teachers and expert teachers and 50% of participants reported that they had previously led a controversial discussion in the past. So for my first research question I'm really interested in this change of comfort I asked at the beginning and the end of the scenario, how comfortable are you with leading a controversial discussion. And there was an increase from 2.95 to 3.69 and this is a significant change. We can see in the Delta column here that we got a lot of people out of slightly uncomfortable and a lot of people up into slightly comfortable. And then when we stratify this out by years of experience we see that everyone goes up the more than five only goes up slightly which is expected for teachers already really confident with their students and their material. The greatest gain was for those with less than five years of teaching experience whole point up point point. I think that was really good because even if you're a novice teacher with one or two years of experience that's still a lot of teaching experience. So how can we just get you comfortable with teaching controversial issues in your classroom. So now I'm going to go through each of the five groups and the first level choices that people made and how this is and how these are different based on their years of experience. So in the first group the participant approaches the table it's an easy growing group of four students and they're having a calm discussion and all you have to do is practice asking open ended. You can these are the three levels of choices that people can make they could ask an open ended question a limited open ended question or a closed question. And we see that those with less teaching experience were more likely to start the conversation with a closed question and those with more teaching experience were more likely to begin the conversation with the most open ended question. I'm also going to be showing for each of the groups the most common paths that people went on past 35 and past 36 were the most common here, but for those with more than five years of teaching experience 36 was the most common. So what is the difference between these two paths. There's only. These are the same except for the last final dialogue choice and past 35. These participants told the group it seems like they have more to discuss and you leave them to check on the next group. But for fact that half 36 favored by those with more experience you asked you asked another open ended question to the students I think this is an interesting teacher dialogue choice by experience. For group two this is about student directed discussion the students have a technical question to ask you that you know the answer to, but you can choose to give them the answer or let them figure it out for themselves. And we can see here that the those again with more than five years of teaching experience were much less likely to do the teacher directed option and we're more likely to do student directed or a prior knowledge question. And for this one the participants were much more spread out over the simulation with lots of paths being favored especially in the no experience group. The path 27 was pretty common, but what was interesting is that the participants with zero experience, none of them utilize path 27 and actually made choices that cause them to leave the discussion early and move on to the next group. For group three the students are have you got to the table it's described as being loud they're having a very excited debate and they want to know the teacher's opinion. So you can share your opinion declined to share or ask them what they first. Only those with no experience outright declined to share the initial part and those with more teaching experience 22% of participants actually shared their opinion as a path 31 was the most common for these participants. And in this one you asked the group for their opinion first declined to share your opinion again, and then you they sort of asked this one student who's a playing devil's advocate the whole time if they have a fake on the issue. Only 29% of participants ever chose to share their opinion on genetic editing with the group in the anticipate section we asked participants, what they thought about teacher opinion and this was very rarely mentioned. And of participants to reach an opportunity to reinforce classroom norms to keep students from talking over each other 93% chose to do so. In group four this group is not participating at all and you notice that one of the students has actually gone to the bathroom during the discussion so you can start with an open participation and more disciplinary statement or summarize what others. And we see that only those with no experience take the disciplinary track, but those with needs to do experience how sort of even split between an open participation question or summarizing what others have said. And the 15 remaining was the most common for group four. And it's to ask the group what they discussed as a whole ask one student what she thinks and then ask others if they agree. In the scenario, there's a moment where the student who's missing comes back from the bathroom and actually only two participants of 13, even engage with that student at all, because really interesting. And then for the misinformation group gets this group again is described as another loud excited group. They're having a lively debate that you get there and you realize the article is a piece of misinformation that you've seen before. So you can investigate the misinformation with the students deflect or just tell them outright you already know that's false closure Chromebook let's get back on. So those with less teaching experience were more likely to just shut down the students right at the beginning and those are more teaching experience were more likely to say. So where did you get that source from and that's exactly what we see in the most common paths of 13 here. The path 13 is to ask you to explain about the source that she's showing her group. And you ask the team of what organization publish this article you asked you to search a world Daily News report another tab. When you do so you can go to Wikipedia and you see that this is a satirical website. This is the fact checking seal called lateral reading which the Stanford History Education Group has done a lot of research on it's one of the most like powerful fact checking tools and use as teachers to get our students to work through misinformation. So I would like to reconnect to the research questions are for when about comfort we see that this can be a tool and increasing confidence for each for a leading controversial discussions. All groups on an increase and the largest confidence boost was for novice teachers. Again I want to stress that novice teachers does not mean having any experience. Even new teachers have lots of great experience and they know their students and their school and their material so how can we just get them ready to teach these controversial issues. And then for the second research question about analyzing the different moves that people with different levels of experience make more experienced participants tended to selected the choices that allowed the students to control the debate. And those with less experience were more likely to ask the teacher directed question. And by analyzing the common teacher dialogue choices we can show where teachers diverge in their only showed here what happens at the first choice point or more in my thesis what happens at every other point. In the future I would love to be able to expand the possible teacher dialogue choices based on participant. I asked participants in the scenario was there anything you wanted to do in the scenario that was not an option for you we got a lot of great results from that so taking their feedback. Integrating it in and giving them more options that are more authentic to their experience. And also working on generative response student dialogue using something like a Bert model trained on student discussions would be great in which everyone sees the same starting piece of student dialogue you typing your response. It's analyzed by nlp methods and then it generates a completely novel new student dialogue for you to respond. The simulation I designed for my thesis is already very replayable there's lots of different paths but I think it would be really interesting to develop something that's completely novel every time you play it. And finally, this was my original thesis idea that got cut down by covid doing a v test in classrooms where people would complete the simulation and then have teachers implement this lesson with their students. And compare that against a no simulation control group with the same lesson and analyze are the dialogue choices different between those teachers and those students. And now I'd like to do some acknowledgments. Obviously people funded this thing. The teaching systems lab the graduate program in CMS and NSF graduate research fellowship program. The teaching systems lab because I've been here for four years, and I'm going to cry since my first like conference and you know to presenting my master's thesis today. And you know my friends and family this is the only photo I could find of the CMS cohort on zoom. I'm going to take a good one today. You know everyone who's helped me on my thesis ideas and for grad school. So thank you everyone. Now take questions. Yeah, yeah, so we definitely see in like discussion theme teacher literature that this is like one of the biggest things to having a productive discussion it's really hard to let the students go off and have their own ideas with each other like in a Socratic seminar because you're afraid or what if someone says something inflammatory or what if it completely goes off the rails. So that's why establishing on these norms early with your students and co creating these norms I think is really important to fostering a classroom environment. Or you feel comfortable enough as a teacher where it's like, that will not happen. And if it does my students are responsible enough to pause in the discussion and return to norms. Relations like this end up being used how they get in training programs, and then where they. I mean, I guess, just want to expect these controversial issues to be discussed in general, would you expect more private schools to be interested in something like this the public schools, because it's, you know, obviously, there are plenty of school districts that are would like to have no controversial discussion whatsoever in the classroom. And so how do you tackle that issue which is political and is outside of your research in many ways and yet impacts like the implementation of all the work that you've done. Yeah, that's a good question. One of the things that teaching systems lab has done is hosted mooks on for teacher training about different equity issues and embedded the simulations within them. Also, the teacher moments platform is like free and open source so if you're really curious you could go and do the simulation right now. But I think the dissemination is a really interesting problem because like you said a lot of laws are currently being passed that keep teachers from talking about controversial issues in the classroom so how can we, you know, work with those teachers to still make sure that they are meeting their students needs of being able to have democratic dialogue. Justin. Yeah, so we saw that on like groups of five and three had the most convergent and group two was all over the place. And so it is interesting to consider was this a design artifact did I as the designer make the answer to obvious for some people, or people really experimenting and trying out lots of different things. I'm in the past so in my thesis, the write up I will explore further about like, was this actually an artifact of the design or are people making these sort of novel interest. Josh. Would you want to give people as they were going to the different. Yeah, it's a good question. Oh, yes. So Josh asked what types of feedback would I give to participants in the simulation. So one of the anticipation questions that participants saw was exactly the same scenario from five. It says you, you get to a discussion and you notice that students are discussing a piece of misinformation as if it's true and you already know that it's misinformation, what will you do. And a lot of the responses that people said was I would talk to them after class or I would ignore it, not a lot of confronting it in the moment. So that's a piece where I would love to be able to give some feedback to people. Oh, it looks like you might have said that you would have waited until after the discussion to talk to the student but it's actually really important to confront misinformation when it happens. That would be a great moment to give feedback to people. Thomas. Yeah, so how could like civic engagement like people who work in schools use these type of tools. Oh, okay. Thank you. Thomas asked if there could, if this platform is specific to schools or could be used in other areas. I do think that this design could be used in other areas to encourage on sessions like not only the public but also in lots of different areas and politics and other controversial issues. Just my interest is in education but I do think it could be used in many different places to prepare people to have controversial discussions. Meredith. See all the answers is there a perfect path through the simulation did anybody take that perfect path. And then the corollary this is, it seems like there are some right answers and there aren't some answers to explain a little bit about how you. Meredith, Meredith asked are there any right answers to the simulation. I did not design any path to be specifically like the prime path like this is the optimal solution. Because with the replayability of simulations I wanted people to be able to go back and explore different methods they could take with their students so there's not been necessarily a right answer there's things that sort of engage students more. To pick the discussion doesn't sort of shut down and end horribly. The original version of discussion leader I did have that actually where the students would stop talking and your classroom descended into chaos of a participant told me that was not conducive to their learning surprising. So I took that out so it's really more a tool like exploration of different skills. Thank you everyone.