 The use of online learning, so you can see in the last decade, huge growth and enrollments. Here's another graph of prediction from the general and seeming that show what the expected growth is of online enrollments in just the high school realm. You can see it should pick up dramatically. Here is a chart showing states with virtual school or online programs. Pretty much all the states except for a few up here in the Northeast have virtual schools, some kind of online option. These blue ones are states with virtual school or state led initiative. The yellow are ones where states have a full time state online school and then the cross hatched ones are ones with both. So you can see that pretty much students have access to online options in most states. If we look at the start states here, those represent states that have an online learning requirement. It's becoming increasingly popular as legislatures and others look at online learning really being the way people are going to get professional development in the future. States are starting to require that students have some kind of online experience before they even graduate from high school. So there's a few states that have done that already. There's two states right now that have requirements. These white ones, these white stars are ones who have future requirements. So they're in the works to have requirements. I know here in the state of Utah, there's not a requirement that students take online courses before graduation. But it was just a bill pass that allows them to use funding from the legislature to take up to two credit hours a year. Even if they're attending a face-to-face high school, they take up to two credit hours a year this year and then I think it's supposed to jump to four or six. So there's increased emphasis on online learning. So there are some pains in this area. For example, we know that there's higher attrition rates generally in online learning. This shows the graduation rates across the United States. Especially here in the southeast, the graduation rates are really quite low. And this is not online learning. This is face-to-face learning environments. So if we know that this is something that we're concerned about, because if we know that this high attrition rate is just the regular graduation options, regular high school options, and we know from the research in higher ed that there's even higher attrition rates in online traditionally than in face-to-face, this means that unless we attend to that, there could be really dramatic problems in the K-12 graduation rates as we move more towards online. When we look at the research about why students fail to persist, we have frustration over prompt feedback, unclear instructions, a sense of isolation, absence of personal contact with instructors. We also have research that shows that faculty in online environments oftentimes are feeling just as isolated as the students are. And that doesn't lead to them wanting to persist as a teacher in that environment. So we kind of sum that up as being lack of quality interactions. I'm going to let you present the story part. So when we look at an online environment, there's different types of interaction that have been identified, most famously by Moore. He said that there's learner-instructor interaction. He also said that there's learner-learner interaction and then learner-content interaction. He identified these within a higher education context. Within a K-12 context, we also would have to think about some other types of interaction. So that would be learner-parent interaction. And along with that, we'd also want to look at parent-instructor interaction. And then when you look at within those interactions and try to identify what is the topic or what is the purpose of those different types of interaction, we've gone through the literature and basically see three types. So there's social, administrative, and content. Essentially, I think this is kind of self-explanatory, but if you come down here to content, this is their interaction focused on improving their content understanding, clarifying, expanding the course material. So this is with teachers, peers, and others. They will sometimes get their content from human interaction. Procedural helps with doing well in the course, but it doesn't necessarily directly improve their content understanding. So this would be more due dates, assignment requirements, and things like that. And then we also have social, which I think a big part of social is motivation. Trying to motivate students, and especially K-12 learners, which sometimes lack that motivation, and also encouragement, personal interest in clubs and humor and service projects and things like that if the virtual high school includes those types of things. Before you go to the next page, one thing that's important to understand about this is, in the predominant distance learning frameworks, we talk about learner-learner interaction, learner-instructor interaction, and learner-content interaction. But really, content plays a more important role than just the learner-content interaction because learners are interacting with the content when they're interacting with the instructor or when they're interacting with each other, and also we found when they're interacting with the parents. So the content has to be quality content that not only the students able to understand well, but the other people who are interacting with them around the content, including the parents, can understand. And that comes back to this. I think the power of using OER is that the instructor can adapt it to kind of include their personalities. And there is an unclear distinction between learner-content interaction and learner-instructor interaction because the instructor oftentimes are the ones that have prepared the material. And so in a way, they can insert their personality in that and communicate with the learners. So what do we know about virtual schools? We know that they're popular. We can track the enrollment rates and the number of students that are involved in it. As far as the research on virtual high schools, it's very limited. And especially, so this quote right here is quite telling. It says, one specific area of research that has not received adequate attention is the effect of parental involvement on student achievement in virtual schools as well. This is an area that just wouldn't receive attention in higher education, but in K-12, we definitely need to look into that. These were our research questions. So we've been working with the Open High School of Utah. And what we wanted to ask is what are students and parents' perceived quantity of interactions? So how much time per week are they spending on interactions with the content with the instructor, with peers, with parents? And also, what is the perceived value of that for their education? And then the last question is, do we see correlations between the quantity of interaction and course outcomes? So like I said, we picked the Open High School of Utah. We were quite lucky that this was here in Utah nearby and that we were able to meet with the Open High School of Utah. They've received recognition at the state level and similar recognition at the national level for the work that they're doing. And I was preparing a video that I found on their website of Delaney Tonks, the director of Open High School. And then I saw that she was going to come. So I've asked her to kind of present a little bit about Open High School of Utah and how it's related to OER. Thanks for that, Jared. I wish I had chocolate to give all of you paper honors for the very last conference session of the Open Ed conference. It's been a fabulous workshop. Kudos to David for putting this together. But it's my pleasure to talk to you a little bit about the Open High School of Utah and give you some of the context around it. We're a fully online charter high school and this year is the first year that we rolled out grades 9 through 12. We build our own curriculum. We audition our teachers and I've got two of my, the heart and soul of my school sitting here with me. Mrs. Emily Anderson and Mrs. Jess Mordecais, they are an English and a science teacher. And they are master curriculum developers and writers and also master teachers. And what that looks like at the Open High School of Utah is we make sure that we hire the very best. We give them the tools that they need to be successful curriculum content writers and then we look at the learning outcomes. And so our CRT scores for one example is the state mandated test. Those are all above the state average in every single category from year one, which was a little surprising. I'll be honest. I was doing cartwheels down the hallway when I got our first year's CRT results back. However, you could take our content. You could take all of the tech tools that we use. And if you did that without the personal interaction and the context that the teachers add to those courses, it would not be as successful. It would be very difficult to replicate. I appreciate the work that Jared and Dr. Graham have done at BYU to kind of give some clarity to that issue. Is there anything else you want to be covered? Can you just touch on during the hiring process that you have the candidates for fast? Yes. This is a great process and I have to give props to my curriculum director, Sarah, who worked with me on this and came up with this audition process. So I'll give maybe 300 resumes and as much as I love to read those, I don't have time to look at 300 resumes. So we send an email, cart and parcel, cart and plush to everybody that says congratulations. You have made it to the next level of the interview process. Here's a list of tech tools. Here's a list of places that you can find OAR resources. If you're interested and you want to continue this process, then send us an email. It will give you access to our learning management system. And if they do that, then we send them three standards and say build a lesson for us. Then we look at the data. We look at how much time it took them to build. If it took them five days to come up with a 10-minute lesson, their skill set may not quite be where we want it to be. We'd like to teach the classes in the fall, not three years down the road. We look at the types of stuff that they put in there. If it's just paragraphs slapped into a computer screen, that's independent study. That's a worksheet on a computer screen, not what we're looking for. So I need to see that they're dynamic, that they understand the interactions that are necessary for the teacher and the student to be able to have a comprehensive curriculum. So we're looking for screencasts, we're looking for chunks, we're looking for applicable content that they can move through, that shows that they understand what the learners are going to be interested in and what will engage them. So we take a look at all of those things. We get about 10 people who actually do that out of the 300. Yay for me, 290 resumes, I don't even have to look at. And then we interview the top three. We look for screen presence. That's how they're doing a lot of video conferencing with the teachers. And I got to see that they can reach through that computer screen and engage the students. And then we pick the very best. We hire from all over the state. They're not relegated to a 50-mile radius around a bricks-and-mortar building, and we end up with five-tier teachers. Thank you so much. Can I say something about this, too? One of the reasons why we're really excited to work with the Open High School is because their model has kind of flipped what the traditional model of high school education is. And that is their instructors really focus on developing high-quality content and then helping students individually around that content. So rather than spending their day presenting a lecture for a period and then repeating that the next period and then repeating that the next period, they put their time into creating quality content and then creating interactions with the students around the content. We think that that's a really wise way to use the human resource. The important human resource of the instructor as opposed to spending the time lecturing and in a lot of cases repeating the same basic lesson over and over. The other nice thing about Open High School is if you think about all those types of interaction that we have out there, they encourage all those. And that's not the case with every virtual high school. So we could actually examine those things. The other nice thing is that it has high-principle involvement. They all belong to a parent organization. And by the way, students also have their own organization, leadership organization that they use. They give a survey on the first year, the inaugural year, and 40% of the parents volunteer to assist with school activities. So we know that the parents are being involved with the learning and overall very high level of satisfaction with the instructors as well. So we knew that the teacher quality was very high. So our methods, we decided to use collect data with surveys. We did give out a parent survey and a student survey. For today, we're mostly focusing on the student survey. And we measured the quantity and quality of interaction. So how much time are we spending on these different types of interaction? And were they valuable to your education? And then kind of these course outcomes, student course satisfaction, student perceived learning, and student change and disposition throughout the course. We also obtained student grades from the Open High School. So this student change and disposition is really change in their disposition towards the content. So do they take a science class and then come out liking science more than they did when they started? Or do they take an English class and come out really liking writing more? Or are they coming out saying, I hate it, you know? Don't want to have to do that. Our response rate, we surveyed. We picked two English courses because all the students were in them. So we administered one the first year to a freshman English course. 125 or 127 students. And then the next year we surveyed the incoming freshman class. And so that way we could survey all of the students. In total, we got 82 responses. That's almost 33%. We're hoping for a little higher rate. One of the reasons why we felt that it was lower is that we had to have parent survey responses because in the parent surveys where we got their consent to use the student surveys. And so it was a little lower than we were hoping, but we felt that it was adequate and comparable to other online surveys. It's actually pretty good considering for a high school context, you have to get student informed consent as well as parental informed consent to do anything. So it's kind of challenging. Yeah, I mean we were pleased with it. Obviously, you would always want a higher end. So here's the results of the first. Again, this is the reported quantity of interaction. So how much time per week on average? This was what the students said essentially. So we see that 57% of their time was with the content. And that emphasizes the importance of having high quality content. And that's kind of what we expect. We also found that this learner parent interaction was quite high. 21% of their interaction was with their parents, which also makes sense because the parent is there and so at least a one to one ratio. And then the next was learner to learner, which was 15% in a learner instructor, which also kind of makes sense because these instructors have 125 students. And so they can't spend enormous amounts of time with each student. But what we thought we'd see is a high variance in these interactions because students require different levels of interaction with the instructor. And actually, we ended up seeing that quite a bit. So with the learner instructor, the mean was about 42 minutes a week. And the standard deviation was 45. I was going to say, if you think about that, if you start to add up the numbers, that's actually a lot of personal one-on-one interaction for students. I mean, if you just add up the time, that's a lot of time the teachers are putting in, if you have a class of 100 students. And some of that may have been group, like maybe group emails and things like that. But with this model, they really emphasize the one-to-one tutor model. We also saw a very high standard deviation with learner content interaction. Could I say one other thing about that? You might be going to ask this right now too. This is also, these reports were students, the amount of time students perceived that they were. So we didn't have a mechanism for actually, we thought we had a mechanism for collecting actual time using the online system. But it turned out that that wasn't accurate enough for us. It didn't have all of the interactions in it, so we couldn't really... Is that what you were going to ask Dave? No, I was just going to say this learner-instructor interaction, even if you take the median, say half hour per kid per week, got 120 kids, it just doesn't pass the sanity test. So there's something else, I wonder if they're watching a video of the teacher if they think that's a learner-teacher interaction. So what's that? Looks like the Open High School of Utah is 12 chat boxes all open at the same time. With the teacher going back and forth, there's slow typers. And so the teachers are fast typers, the students are slow typers. And so the students' perception is that that 15 minutes that they're chatting with their teacher is 15 minutes of one-on-one time, but it's not because they have 12 chat boxes open. And they're actually servicing 12 students simultaneously. And on the phone and answering emails. Maybe a hundred teenagers basically right now. My kids are, I've been in virtual school for four years, and I think our kids get 45 minutes a year. So that number eyes like, oh my gosh, and I go, my kids at this school. Well that's one of the reasons that we wanted to look at this school because it really, its model is different from other schools that really are trying to minimize learner-instructor interaction and maximize learner-content interaction. And I think this school really has a different vision about that. I want to give one other anecdotal piece of information. This daughter was looking for an online calculus class to take. And you guys don't offer calculus this year, but I was, yeah, maybe will in the future. Do you have AP help? Do you this year? Oh, well it wasn't listed with me. Oh, okay, darn. No, it was an AP stats class that she was looking for. That's probably an AP stats. But I said, we're looking through a lot of the virtual schools that were available in Utah. I sent an email to the person at the open high school with the little email name. It was about 10 o'clock at night. And like a half an hour later, I had, before I even shut down my email for the night, I had an email back in my box saying, oh, we're happy that you were interested in the open high school. And it wasn't a form letter. It was a personal email, you know. And so I think that gave me confidence also that instructors in the high school are doing the same kind of thing. And it's not, sorry, a lot of it too much. A lot of it too much. Well, one thing that we noticed was that learner-learner interaction was kind of by model. The 17 reported no learner-learner interaction and the 14 reported over three and a half hours. So I think a lot of that, this may be an indicator of engagement, right? I mean, they're choosing to do this in a lot of times. So maybe an indicator of engagement in the course as well. Do you have any data on how well the 14, the ones who did have a lot of interaction with their peers stayed relative to the ones who don't have a lot of interaction with their peers? That's a good question. I think we could run that. And I think we may have. I don't remember if that's one of the ones that me. We ran so many. It's hard to remember off the top of my head. But if you'd like to email me and then I could get you that information. Okay. And I can talk to you after as well. One thing that we noticed as well is that a learner parent, we were expecting more of this to be social. We found that a lot of it was content. And that's the power of having open resources as well is that parents have access to that and that they can use that and kind of relearn it themselves if they're helping their students because we want this to be high quality as well. We also see that that learner parent and learner instructor, it almost mirrors each other, right? And then we look at learner to learner and it's flipped. A large portion of that is social. And so the question is, if they're interacting, how do we make that more academic? So interactions, educational value, we see that all the types of interaction were perceived as valuable. This is on a six point scale. However, we see that parent instructor and learner to learner were significantly less. And I think that can be linked to the previous slide where learner to learner was largely social. And parent instructor, they're not directly involved in. And so they may not understand what's going on in those interactions. Is there a required parent instructor interaction? What we tell our faculty is that they need to include the parents on any relevant communications so that they're aware of what's going on with the student and then to contact them once a month with updates about what's going on in the class in general. Great. Sorry to dig in a little bit more. With social interactions that your students are having, a lot of what we all learned in high school is how do we interact with our peers. So I'm not sure that I would discount it as invaluable if it's purely social, but I also would be really curious to find out exactly how social and how academic those interactions are because I bet there's a lot of very subtle academic conversations going on in there that are reinforcing content in really important ways. Which is why I think it's really interesting to look at how the kids who are interacting with each other a lot do relative to the ones who aren't. Right. And I think, again, this is perception. It would be great if we could do a content analysis of some discussions. The other thing is I think you're right that social interaction, that social presence can be a requirement or prerequisite for cognitive presence and kind of getting to that point. But what we don't want is them to kind of spin their wheels in social and never move beyond that and to get into the content. One thing that we saw with correlations, and we found some fairly interesting correlations that we weren't necessarily expecting, but what's important to remember is that we were looking at the quantity, not the quality of the interactions, and we were looking at course outcomes not necessarily growth. And so we feel like if we could have looked at growth, we would have seen different types of correlations. But here we see that, for instance, we see a negative correlation here between learner-parent interaction and student-improved disposition. This is not a causal relationship, right? What we're guessing is happening is that students that are not liking the content, parents have to step in and encourage them more. And along here with learner instructor, we're expecting to see more of these significantly correlated, but we didn't, which could be a sign that unmotivated or confused students are getting more interaction than other students. So this could be a good sign. In order for us to see that, we'd have to look at growth and not necessarily just course outcomes. I think about a time there was great questions if you have any other questions later. Are we out of time for questions? Hey, this is the last thing that they use. I'm just holding up fingers in the evening. A huge question for me all the way through has been how the students get there in the first place, who they are, self-selected. This is a social class issue at all. Are these home schooling-type parents? All that makes quite a difference to the way I would interpret what I'm seeing. I appreciate a little bit about the students. We know that there's a wide range of SES students at the school. Can you address some of those? The other questions, they're delaying about the demographics. Absolutely. We're public schools, so we take all commerce. It's not a selective process. And we have a wide variety of students from all backgrounds. We have high-achieving students who travel on their afters and afters. We have a home school population. We have a high percentage of students with IEPs and 504s who need additional instruction and help and find this as a safe haven for that constant communication and fostering of relationships that they were lacking in their other places. We have, I'll call it afters with the last list. This is pretty much their last option at the school. And it runs the gamut. Thanks. And you should also say that you provide a laptop for the students. So they're not limited to the family's ability to be able to purchase, you know, the technology to be able to participate in this. How many students are full-time versus kind of the 1D2Z? These are all full-time. Is that the high school itself? Is it all full-time or nothing? Or do you have individual classes that are paid? It used to be all full-time, right, until this year. And now you're allowing some... Mostly full-time. Yeah, it's mostly full-time. So this year with that law in the state of Utah that allows people to take two online courses, they're allowing some students to do that, to be at their own high school but take one course or two courses at the open high school. Correct me if I'm wrong, but I think the first year they all took their core courses at OHSCU. And then they were taking more independent online courses. They had an option of doing more independent learning. And they didn't like it. I mean, they became accustomed to this high level of interaction. And it almost felt like there was this revolt that they wanted more interaction and they wanted the school to offer more of those types of classes as well. So second semester we added two classes, so they wanted full schedule with us. Right. I just love your bullet point number three. I think that might get me out of helping my son with his last homework. Yes. We don't recommend looking at this implicitly. So where does the fun thing come from? It's a charter school. So in conclusion, I'll let Jared say something that one of the things that we hope the people in the OER Open Educational Resource Community will consider is that content, content's important, but content's not keen. You could have high quality content. If you don't have good interaction around the content, the content's not as meaningful, especially for this adolescent population that have lower self-regulation skills than maybe adult learners do. So you really need to consider not only the students as consumers of the content, but also parents and instructors and need to consider the interactions that are happening around the content. I think we're out of time, so thank you for everyone for coming.