 Thank you, Susanna, for the kind introduction and thanks to Paul for inviting me to speak. I just need to get my slides up. Hopefully I can shut that. All right, so I am going to start the timer. I'm going to try to schedule things so we can still finish on time but have a few minutes for questions. So I'm going to condense a little bit. I'm going to focus really on kind of trying to give you an introduction to FET, but also talk more about the role of open within our project and how that's worked. So just to get a sense from you, how many of you had heard of or seen FET before coming to the conference? How many have actually played with the FET simulation? How many of you have disseminated or used FET within your work in open education? All right, well I am here to try to change that. Okay, so this brief outline, what is FET? And then focusing in on what is the role of open licensing and its impact within our project. And I'm going to leave with just a couple of challenges and opportunities that our project is working on. So FET was founded way back when open started in 2002 by Carl Weiman. He won the Nobel Prize in Physics in 2001 and he used simulations as a way to explain his complicated science to general audiences. And he just found this tool incredibly powerful for making these complex ideas accessible to broad audiences. And he saw that through the sort of complexity of the questions that they were asking back. It was indicative of the fact that they were understanding the fundamental ideas that were a part of his work. So he founded FET with the goals of making STEM learning more engaging, so making it more interactive, letting students really engage and explore and discover the key ideas through interaction to making science more relevant to these students, to a broader range of students, to showing them how science really was explaining the world around them and how they could use science in their everyday lives. Making it more accessible, both just the broad access, getting tools to where tools are needed, but more generally to make it more accessible by making it more intuitive and understandable through the tool that you're using to explain that science. Making it more effective, lots of science is taught without even students engaging in science practices, so integrating that into the process of learning science and also to develop a deeper conceptual understanding. A lot of research in physics education has shown that students at all levels can take a physics class or a science class and leave without developing a conceptual understanding of the underlying ideas and to make it more personalized where students have a lot more agency over how they're learning and understanding science. So focusing in, just to really emphasize this point, is a goal to make science learning more active. So science is an active process. Our curiosity is engaged in understanding the world around us. We engage in experimentation, collecting data, modeling that data, gathering evidence, giving our reasoning. It's really active and so often science learning isn't taught in that way and to integrate that into the whole system and to do that through the use of interactive simulations. So we are situated at the University of Colorado Boulder in the physics department here in this big tower with these nice mountains behind us, just sort of like we have mountains here. We have two sides of our project. We really have a product development side where we're trying to generate high quality OER and put it out to the world. We also engage in research, so we sit within a physics education research group in the physics department and we engage in research around simulation design, how do students engage in learning from a simulation and also how you can integrate simulations into the classroom and how that changes the learning dynamic between students, between students and teachers and between everything and the content that's being learned. Today we have over 150, I think it's 157 simulations, 82 and HTML5 for those. It's really important but not everybody understands why. 2000 simulation based lessons. We cover physics, that was our home that was where we started. We expanded chemistry, math, or science, biology. They're used in 12 and college. It's a middle school, high school and introductory college are leading categories of use. They are an OER, they're licensed under the CCB license, they're translated into over 90 languages and they can run online or offline. So one of our access solutions is to provide an installer for our entire website. So you can download a single simulation and port that around on a thumb drive or you can download a full installer of the entire website and bring that to your unconnected school and install it on computers and then you have all the simulations that were available at that time that you downloaded the installer. So this is our growth over the years. Right now we're at about 120 million runs per year. We've delivered 650 simulations total but plus offline use. So this offline use is impossible for me to tell you how much that could be. I think it could double this number but I can't tell you exactly because we have no data on that. So this is what our content expansions looked like over the years. We started in kind of introductory physics and then we expanded to quantum physics in 2005, chemistry in 2008. Middle school science in 2010 where we did a lot of learning about how simulations can be designed for younger students and we've now integrated that across our simulations. We expanded to math in 2012 and then to sort of focused in on algebraic thinking in 2015. And I am trying to raise money to focus on K1 math because I think that is so influential. I want to do that. Technology advancement, we first built the simulations in English and then in 2004, pretty early on, we made them translatable into Spanish in a very rudimentary way. And then we made the simulations translatable so anybody could translate them into any language. Then we made the website translatable, a bigger task but still doable. We completed our first simulation in HTML5 in 2013. And the reason that's notable is because we started when Java and Flash were the technology of simulations and now they are incredibly hard to run. Flash is sunsetting at the end of 2020 and Java is just a technically challenging thing to run these days. So we're working on moving everything to HTML5 but it is a very resource intensive process. But we're leveraging every opportunity that comes along with having to redo all that work. And part of that was the ability to sort of actually face head on the challenge of how you make things that are so visual accessible to students who are non-visual, how to make them screen reader accessible, how to do the descriptive, interactive descriptive text that allows non-visual students to fully engage in a simulation. And we released our first accessible sim in 2017. We are still working on that technology challenge but it is fun and exciting. So this is what our global use looks like. It's 33% international, essentially used in every country or territory. I kind of plotted this minus the US because the US just dominates otherwise. These are top 15 minus the US countries that are using the simulations online. I can only get Google Analytics. So I am going to show you some simulations now. Since many of you haven't seen them. So they're built as these open exploratory play spaces that they don't come up with directions. You just start engaging with them. Buckets tend to be a very attractive thing for students to start with. So they don't really have trouble starting with that. You can see I'm pulling in protons, neutrons, electrons. Every time I pull in a proton, you see the element name change. Every time I pull in a neutron, you'll see the mass number change. And every time I pull in an electron, you'll see the charge change. And students can easily engage with this in an inquiry-based way to discover the basic rules about atoms. Things that you would normally have to have told them just how atoms work before. Now we can provide them with this interactive and much more engaging environment to learn about these fundamentals. This screen layers up to this idea of how chemists capture this information in the chemical symbol here. And there's a game for students to test their understanding. Sometimes students will jump straight to the game and they'll realize, I don't understand this. But then they always have access to the Explorer screen to go back and see what's going on there. So that is one of our simulations. I'm going to show you a range because we have some different styles. This next one is called circuit construction kit. And it really is a kit. It's more like building blocks and you can build whatever you want to a certain degree. So here you pull out wires, light bulbs and batteries. As soon as you connect them, you see the electrons flow. So you get immediate feedback that that's working. You can do things like change the battery voltage. That's what more like an undergrad might do. An elementary student might just grab another battery and see what happens there. So there's multiple ways to interact with these things. And if I flip this battery, for instance, well, that was at 9 volts. If you keep them at the default, then what you see is essentially that you've canceled out all of your voltage so you don't have anything left. You can pull out volt meters. So you can really use this tool to engage in a whole range of learning from sort of the basics about electric circuits to getting at the college level about parallel and series circuits, taking a set of data. You can even hear change the resistance of the light bulb. So you can do things that are just not possible in the real world to develop your understanding. Within math, we have arranged one of my kind of favorite ones that we did earlier on is this one about fractions. And again, it's just gives students a way to engage in discovering, you know, what does that top number do? What does the bottom number do? They can play with the numbers. They can play with the pieces. They can rearrange the pieces. They can look at different representations, including cake. They can play with the number line. So it integrates all the way to the number line. And again, if you change the denominator, you'll see that show up on your number line. There's a game where students can practice building fractions. If they, if they get it wrong, just pops back out. They can try again. And they can build it however they want. So there I put the two-fourths. I put a half and two-fourths because it is a half. And here I did the opposite. So it's really flexible. There's an open lab for students, for teachers to now discuss. You know, okay, Sammy, come up and build a one-half. Now can somebody else come up and build a one-half in a different way? And so then they, they, teacher can facilitate a lot of discussion around how, multiple ways to build a one-half and how that works. Okay, and this one is gravity and orbit seems to change forces and watch your planet crash into the sun and things like that. So, keep an eye on time. So within our project, we really do focus on a research-based design cycle around building these simulations. We start with the learning goals and with the research base. So the research base around how people learn and everything that goes along with that. The research base around simulation design, but also specifically the research base around student difficulties within a particular content space. With circuits, one of the student difficulties is, do I still have it up? One of the student difficulties is that current is used up. And in, and in this simulation, what you can see is that, you know, current is not used up here. There is the same, the electrons are going the same speed here as here. And if I pull out my current meter, I can actually measure that with my current meter. So we specifically look at the student difficulties around particular content spaces and build that in to how the simulation is designed. And then we also engage in interviews for every simulation. So we bring students in. We watch them use the simulation. We look at the flow. We look at their interpretation of their representations. We make sure that they are leaning into the simulation, asking their own questions, engaged and, and making progress towards the learning goals that we've identified for the simulations. Each one, we have a team of about four to five experts. So those are experts in teaching the content, experts in the content itself, experts in simulation design, and experts, software developers. Each one takes about 200 hours of design and 600 hours of development plus student interviews and quality assurance testing. So these are not, these are things you can, hard to do this level of simulation on the weekends. So one key idea is this idea of implicit scaffolding. So you saw there's no like directives in there about touch this, touch that. So we call that implicit scaffolding. We scaffold through the design by how we place the controls, what dynamic feedback we provide as students interact. We build in the real world connections. We show the models that scientists use. So there's lots of simulations that will show vector models and things like that, that scientists use to actually develop their own understanding. We show the invisible. So we show leptons, atoms, photons. And we do allow these actions that aren't possible because lots of times they will help students develop their understanding. So being able to dynamically change the resistance of the lipo, you can swap a resistance out in the lab, but that dynamic ability to change and see what happens immediately helps build that cause and effect relationship that we're trying to help students develop. So the result is a simulation tool that can support multiple learning goals all at once. So it can support students' content development, their concepts, their models, their understanding of representations and relationships. But it also can support their engagement in science processes and practices. So exploring and questioning and predicting what's going to happen when something changes. When you embed it into a learning environment, it can also support students' ability to engage in argumentation and collaboration, support their development of lab techniques, and one of our main goals is to support their enjoyment of learning science and to not leave class with saying things like, I'm not a science person. That everybody can be a science person and to develop their ability and their identity as a scientist. So we encourage a lot of student agency in the implementation of the simulations. So basically we try to develop these high-quality tools that also have high flexibility. So you can use them in lecture. You can use them in lecture tutorial or small-class discussion. Here, this is a university classroom. It happens a lot at the K-12 level too. You can use them in lab. You can use them as an outside of kind of class time thing, as a pre-lab, a pre-class assignment, a homework. You can basically use them with any kind of pedagogy and any kind of, you know, learning environment, whether you're online or offline. You don't need to be able to run them. That is the one requirement. But you could be a completely traditional instructor and we see lots of them using the simulations as a demo because the simulations are just like they think in their mind, so they really are attracted to them. And then we see a lot of people using the simulations to engage in much more reformed classroom practices. So now I want to engage you in experiencing it a little bit. If I can type. So I'm going to pull up a simulation called Curve Fitting. And I'm going to keep an eye on the time. I think we have time to do this. So here you can pull out data points. And these data points are kind of special. You can change their error bars. Fancy. And you can plot a best fit curve. So I'm going to pose this question to you. How many have engaged in sort of concept tests or peer discussion ever? Oh, nobody. Not very many people. Okay, this is awesome. So in lecture, in college, we use this a lot to make lecture more interactive. And it's good for you guys to wake up, so I'm going to do it. So the way this works is I pose a question and then you're going to turn to your peer, your neighbor. You're going to discuss that question and reason with each other and try to come up with your best collective answer. And then I'm going to ask you to vote. And here I've just given the responses one or two or three. So I'm going to ask you to vote like this. Like a one, two, or three. So nobody else has to see how you vote, really. Just me. If I was in a classroom, I'd have clickers or a colored car. I'm going to go to the graph I just made. So if we increase the error bar on this one data point, what's going to happen to the slope of the best fit line? So this is the best fit line. Is it going to become more negative? So is it going to tilt clockwise? Like this? Or is it going to become less negative? Is it going to tip up like this? Or is it not going to change? OK. So now it's your turn to turn to your neighbor and think about this. So go ahead. Awesome. That's what I love. If I was actually teaching a class in college, I would walk around and listen in on the conversation. So I would get some pre-knowledge about what the students were thinking before I asked for votes. But some insights into what votes meant. But I don't have that much time. So let's go ahead and vote. OK. On the count of three, you're going to hold up a one or two or three just in front of your body. So OK. One, two, three. OK. Not everybody's voting. Everybody should be holding a one or two or a three. Like what your answer is? You're voting your answer. Like do you think it's answer one, answer two, or answer three? OK. So it's a totally split vote. I'm saying one, so I'm saying two. And I'm saying three. And that's not unusual at all. Because it's not that intuitive. But you've had to discuss it. You've had to think about it. You've had to reason about what is going on here. So now we're going to do the experiment. And we're actually going to change this. And you're all going to see what happens. Ready? Why did that happen? So that's why it happened is because these error bars tell me something about how uncertain this point is. If it's really big error bars, that means that data is, excuse my language, crap. So that's not very good. You should just not count it very much. So it's kind of like throwing that point away if you make the data. The error bars really big. And if you make them really small, it's going to count much more and tip it up. But there's other intuitions that you might have had. So if it's really big, the line should definitely go through it. So it's going to shift up. Or the line shouldn't care about it because the central data points aren't changing at all. So there's a lot of ways that students can reason there. And so you could have an interactive discussion. But that was fun. OK. So you did engage in, sorry, my time got shrunk. So here I am. So you did engage in science practices. To think about reflecting on what those were while you were doing that. This is what it can look like in a smaller classroom. So this is a teacher introducing fractions. What? So I thought I had till 10.30. I have a timer. I'm good. I'm sad. I'm on this. They change a conversation with the bottom number. So that is just to show you that the same thing happens with little kids too. You show them a SIM and you give them a prompt, not telling them what to do with a SIM. What does the bottom number do in that fraction I showed you? And they will start talking about it. So it can really impact learning. So we did a study in learning fractions for fourth graders. This was knowledge that they should have learned in third grade. In the pre-test they came in, they knew the procedural knowledge of fractions pretty well. But they were not doing so good on the conceptual ideas within fractions. And after sort of a three-day working session, like three consecutive classes working with the simulation and retesting, they really increased that conceptual knowledge and reinforced their procedural knowledge. The same thing happens in college class. So we had a classic demo around standing waves. And then we used our wave on a string SIM to demo it instead. And in this case, the SIM gave a dramatic improvement in students' ability to visualize what a standing wave was and how things were moving. So moving to the role of open. So we build these simulations as I explained. They're really highly designed. So we actually emphasize four of the five Rs within the simulation itself. Retain, reuse, revise, and parentheses, remix, and redistribute. So math and science are this universal language. So generally, you see the same student difficulties all around the world, no matter who the students are. And the tools are so highly designed that we do not make it that easy to revise. The source code is open and some people do grab the source code, but then it's impossible to maintain because Apple and everybody changes things. So we don't encourage changing the source code that much. But we do make it really easy to access and to adapt. So we make it embeddable and downloadable. We don't require internet. It's easy to reuse. We can add in the language. We also let you kind of use a query parameter to select just one screen so you can better structure the lessons. What we do encourage is adapting SIM-based learning environments. So the SIM is not the only thing. It's just this open tool. You build some activity or task around it. You have teacher facilitation. It's all occurring in a context. And that makes it incredibly reusable and powerful to localize into various contexts. So you have your teacher and your student with your simulation plus activity. This can be, you know, what you do exactly will vary with the grade level, the prior knowledge of the students, their cultural experiences, the teacher's learning goals, their pedagogical approach, their content knowledge. And their pedagogical content knowledge. You're surrounding that by the learning context. Are you in school? Are you in a museum? Are you homeschooling? Are you doing an after-school program? You know, what is the setting that you're choosing to use the simulation in? Is it a large lecture? Are you doing small group discussions? Online courses, textbooks, video tutorials, they can use in any of these contexts. It also will depend on the tech resources you have. So do you just have one projector? Do you have one-to-one devices? Are you in a fully online environment? Do you have other experimental equipment you're trying to work with or math manipulables? You know, what do you have available to you? It also depends on what country and community and culture you're living in. So what language, what national standards, you know, what's your environment or your community and how does that influence what you do? So we build an open ecosystem around the simulations to translate a simulation We have a lesson exchange database where we encourage teachers to share their lessons and search for lessons as usable starting points and all of those are shared as CCBYs. So on any simulation webpage you'll find the submitted activities and you can also submit your own activities. We also partner with other open education projects as part of our remix and redistribution partnership. So OER Commons, OpenStacks uses the simulations in their work. Learning Equality and Calibri brings it to offline communities. Hippocampus integrates it. Golab is a project in the Netherlands that uses it. I can never pronounce that. It uses it in their textbooks. So this is what it looks like around the world. This Burundi, Indonesia, Cambodia, Mexico, Nigeria, Vietnam. So you see just like a huge range of what was available there. But for the most part students were using the simulation, so that's really exciting when you see this. This is something that really excites me. So we are seeing increasing local research on FET simulations. So I just do like every now and then I do a Google scholar search of FET simulation education to sort of get the right context. I don't look at every one of these links because there's too many and some of them are in other languages and I can't understand it. But you can see that back here this was pretty much us doing research. But now there's more than like 700 links on Google Scholar that have FET simulations in the link. And these are real papers and I'm just like, oh my God, there's so many people doing stuff. So in 2019 in South Africa there were some researchers looking at the student impacts on attitudes towards learning chemistry and the third year Bachelor of Education students. So they found basically significantly higher mean post-attitude test scores and that the students liked experience autonomy and enjoyment with the simulations. In Zambia they looked at the impact on E&M learning for 11th grade students and saw positive impact in performance and attitudes. In Thailand they looked at, now pedagogically they looked at integrating simulations with formative assessment. It was in the context of buoyancy. And they found a positive impact of the integration of formative assessment pedagogy with the simulations on the learning outcomes. In Indonesia they looked at FET simulations and its ability to enhance critical thinking skills in the work energy context, small study. Lots of these are kind of small studies. I looked for one in Italy because that's where we are. So in Italy there was this one conference proceedings I think, conference proceedings looking at the ordering of real, using real stuff versus virtual stuff or virtual versus real and how that worked. Which ordering was better for, this is a grade school study for electrostatic attraction. When I looked at the graph and their conclusion was that the ordering didn't matter in the final understanding but it did impact sort of when students got that understanding. The simulation they got it faster. There was one in Lebanon about virtual versus physical experimentation around circuits. Anyway, there's a lot of stuff out there. It's crazy. And it's really exciting to see. So our challenge is an opportunity. We do have a really large effort right now around accessibility and inclusive design and that's led by my colleague Emily Moore. And it's been work going on for a few years now. You can find out a lot more about it at this link under website accessibility. But this simulation is fully accessible, screen reader compatible on our website right now. There's a lot of technical challenges around that. We had to build a technical solution and as well as pedagogical solutions to try to create the same sort of inquiry based experience that you can have with a regular SIM and do that through screen readers. And it actually uses the same tool. It's not a different tool. It's the same tool. It's just the tool that can do both things. Another challenge is sustainability. So we have been working on building a business model. Sustainable mission driven business model is how I call it. So we have various threads that we're trying to do that. But that is certainly a challenge. I don't have the right answer. But we do have a lot of business partners now. They don't pay all that much. But it is mission driven. So every time a simulation is in one of their products, it's benefiting the students that are accessing their products. Like if their school is still using a Pearson product, well at least now they can use a simulation that's good with that Pearson product. So right now we're at about 20% of the budget through our business model. So we still have to really thank our funders because they keep us in business. And we're really grateful to the community that helped support us. This is the team. You know, I'm up here representing us. But they're really a talented group of folks that are very passionate about this work. And thankful for the award that we got this year. So you can find FET online. You can contribute lessons. You can disseminate it if you're interested in partnering in some way. That's my email and you can connect to us on Twitter or Facebook. So thank you. Thank you very much, Katie. I can witness that when we have a workshop about open educational resources, our teachers are usually not aware about repository except FET. FET is always very well known by our teachers. So there is a question somewhere? Please. Before that, okay. Whoops. Well, first of all, I'm really glad to see FET being here because I've been using FET since I think 2014 when I was giving a chemistry and physics course to first year bachelor students. And basically the physics students need to come up to speed on their chemistry. And the chemistry students needed to get up to speed on their physics. And it was an incredible pain to teach all of them at once in the same classroom. And these simulations really, really helped with that to give them some feeling of independence in their own learning process. So first of all, thank you then for that. The second thing, what I was wondering, you were saying that the transition from Java to HTML5 is quite labor intensive, right? So I was wondering, now that FET is really becoming big and a lot of people are using it, and it's open source, is this something you can ask of the community? Please contribute by helping us transition into a more modern digital environment. Would that be a consideration for you? So we have been kind of having volunteers come in to help us with some of that transition. But it takes a while. So FET is built on its own code base. It's HTML5, but it's not just sort of vanilla HTML. It's its own code base and it has its own structure. And so, you know, I'd be interested in exploring it, but right now we do kind of experience that it takes about a year for a developer to really get up to speed. And that's a developer working sort of full time on the project. So I don't know how we'd be able to sort of restructure that so that it could be sort of the more of the open source community where people are taking on little problems at once. Yeah, I'm happy to, if anybody has questions you can come up and I'm happy to stick around. Thank you. Thank you. Thank you again. When I first started in OER, FET is one of the first things that I actually came across. I had two very quick questions. One, is there a sort of feedback mechanism beyond like sending an email? Oh, this is kind of funky or, oh hey, could you maybe do this a little bit differently? And then also, you mentioned that you were having the HTML5 thing. Have you noticed that the converted HTML5 iterations are concentrated in any particular category or subject or they kind of spread out across the entire sort of catalog? So email is really the best. We do read our email and like if there's user suggestion, we'll make a ticket around that within the repository of the simulation. So that's really the, we don't get so many emails that we don't read them. We read them. I don't read them, but we read them. It's really a funding issue. So we are trying to convert our most popular simulations first. So we're kind of going back popularity more than by content space. You'll see a lot of the math are in HTML5 because they were actually funded after we converted. So it's also the oldest ones that are suffering the most because they're all in Java and Flash and anything new is in HTML5. So if people have ideas of funders that might want to help, it's really kind of the most challenging piece of raising money is to get people to redo software.