 Hello and welcome you early birds to this episode of liquid margins hosted by hypothesis where we're going to be talking about annotating in STEM education. We're going to let folks file in here slowly. So, I'm not going to get started just yet, but I did want to welcome you. And if you're here for liquid margins, then you are in the right place. And we're looking forward to you joining this conversation. So hang tight and we'll get started in just a couple minutes. And once again, for those filing in, this is liquid margins episode 42 hosted by hypothesis. We're here to talk about annotating. In STEM education and we'll get started in just about 32 seconds or so. Just making sure there's not anybody. Any stragglers coming in here in the first minute after the hour to want anybody to miss the kickoff. And all the housekeeping have to go through. But this is liquid margins episode 42. Thank you for joining us. In the summer when you should be on vacation, maybe you're joining us from your vacation. That would be impressive. But I've got two colleagues and co-conspirators here clearly in office type environments, not on vacation to talk about their work. So let's go ahead and get started. I am Jeremy Dean vice president of education and hypothesis. This is liquid margins. So we're here at 42 annotating the future reimagining STEM education. We're here to talk about social annotation in the STEM disciplines. A little bit of housekeeping before we get started. We have an upcoming episode next month. We're kind of slowing things down in the summer, but we have an episode coming next month social annotation as instructional scaffolding. Teachers who annotate not all teachers annotates a lot of teachers use hypothesis social notation for the students to have a conversation. So this is going to be talking about the use cases for why instructors would pre-populated texts with annotations or or responds to their students. The role of the teacher in the annotation space. So please join us for that. If you're interested, it's kind of foreshadowing some features that are coming in from hypothesis that will help teachers create annotations. So we're excited about that episode. This is really about this is liquid marches is really a show about annotation strategies. Talking deep deeply and pedagogically with practicing instructors in the from talking about their classroom experience. It's not a general introduction to hypothesis. If you're interested in general introduction hypothesis. I recommend you reaching out to education at hypothesis. This is more of a pedagogical conversation. There is a q amp a feel free to chime in on the q amp a and then ask a question and myself and the panelists or our customer success team that's that's on hand will be able to answer some of those questions. And then if you need closed captioning, but something that you can turn on yourself using the cease closed caption icon in the zoom menu, which is should be at the bottom of your screen. Alright, without further ado, let's talk about annotation in stem. And I have a big disclaimer at the start here, which is, I'm a doctor. But my doctorate is in English literature. So I can basically prescribe summer reading and that's about it. But we brought in some experts teacher scholars from a range of stem disciplines to talk about their work. We have a mathematician. We have a biochemist and we may at some point have a computer scientists join us who was scheduled to join us but is not here yet. Especially in English is a long tradition of close reading texts that is part of the scholarly and pedagogical history of the discipline. And it's true that much of the early adoption of hypothesis social orientation was in the humanities disciplines like, like English. We've seen huge adoption for example in first year writing freshman composition. Over time though it's been clear that we've started to gain traction in stem disciplines. I was just looking over a list of our summer courses and this is kind of fun. We've seen microbiology, introduction to genetics, primate biology, bacteriology, bio psychology, research lab in psychology, forensic psychology, general physics, information visualization, data science, video game design, elementary statistics with probability, micro economics, pharmacology, evidence based practice in nursing, global nutrition, pharmacology, geo hazards and natural disasters and intro to fingerprints and then my favorite hemp cultivation and post harvest processing. I think there's got to be some science in there. So there's a lot of stem instructors who are using social annotation in their courses and we're here to learn from some of our most active teacher scholars in this area about the role of reading in stem about the pedagogy of social annotation in stem and also just generally and practically how they're using hypothesis social annotation in their courses. So with us today are Ashley McHale, Mathematics Professor at Las Positas College in California and Emily Reagan, Chemistry and Biochemistry Professor at Metro State in Denver. I just want to check here that Russell didn't show up yet. It did not show up yet. Okay, so it's just the three of us so far we'll see if the computer scientist zoom bombs us later. But I just want to start off with allowing you guys to introduce yourselves where you teach what you teach, and a little bit of your teaching philosophy and maybe we can start with you Emily. Yeah, absolutely. So I'm at Metropolitan State University here in Denver. And we were founded back in 1965 to serve the adult population of Denver anyone with a GED or a high school diploma could come get an education here we have recently in 2019 received Hispanic serving institution designation. And we have a high number of transfer students about 52% of our students have transferred from another institution. And I teach general chemistry and biochemistry two courses, and I really value students being able to construct knowledge be able to have an active role in their learning right we have to engage with content to really learn it I think. Which is important, I think, exploring simulations are important I think the lab work is important and I think engaging with readings and a thoughtful way is important. Amazing. I love that students constructing knowledge. Ashley. Tell us a little bit about yourself and last Mercedes. Hi, my name is Ashley McHale I have been teaching at last posted us for 15 years now in the mathematics department. I have. Oh gosh what can I say. So last Mercedes college is one of 116 community colleges in California it is the largest system in the nation. We are also designated as a Hispanic serving institution. Recently last Mercedes has received several accolades from different organizations, intelligent and niche have named us number one community college in California. They named us number one community college in the nation or number six community college in the nation so we're very proud of all of the things that we do and they take into consideration transfer and diversity and all just all of the things that make where I work. Absolutely amazing. I teach specifically mostly statistics and algebra classes and also what we call concurrent support. So they are sidecar classes to transfer level classes that help students do that just in time remediation that they need to do. So we teach in the point a mosa and middle college learning communities. So point a is specifically for students who are interested in the Hispanic culture. They, any student can join but we tend to focus on, you know, things that are relevant to the Hispanic community. So mosa is all around the African American diaspora, and then middle college is students who are currently in high school, who are then coming to community college and taking their high school classes and also community college classes and earning double credit. So we have a lot of really great programs on our campus and I just, I love working there. That's awesome. Cool. I just had a really quick follow up question for you Ashley just about the sort of introductory math courses in the California Community College system. There's a specific math program for, you know, the kind of transfer level math is that right. So California has passed several laws that essentially restrict students who have a high school degree from taking pre transfer level math class math and English classes. So when students enter with a high school diploma into any California Community College, it is mandated that we place them directly into transfer level math and English. So they shouldn't be allowed or they, they can't take pre transfer level math or English. They have to start right away, giving them credit for that high school content that they've learned. So in those courses that you teach, are they some of the courses that are the type of courses that the students just coming into the California Community Colleges or Las Vegas would be taking. Exactly. So I primarily teach statistics, which makes up about two thirds of our entire mathematics department. We have quite a, quite a number of statistics classes on our books. If I can just put a finer point on it, is it, it's true then that five, I don't know how many years ago before those laws are passed, you would be teaching a kind of more remedial for lack of a better word tracks and then transfer level they would be separate. I first started at Las Vegas in 2007, over half of our, over half of our class offerings were pre transfer level. And so just in the past four years, actually, I believe it was starting last year we have completely eliminated pre transfer, with the exception of that middle college cohort, because they are still in high school we are still able to get to number one, basically like an algebra to intermediate algebra level class. And that's the pre transfer level class we teach. All right, maybe, maybe in some of your responses you can touch on, you probably have a sense of why I'm digging in there, but we'll leave it at that for now. And we'll just start talking about, you know, I'm sorry, did I not have this slide up there's the lovely people in front of you and still form and represented here is as Russell the towel who may still be joining us I'll just quickly check the slide here isn't shown up, I wasn't I was excited what to have him back to talk about hypothesis and computer science but we've got a mathematician, we've got a biochemist. So I think we're going to have plenty of surface area here in the stem disciplines to cover. And the first question I have for you and maybe we can start with you Emily is what Judy what drew you to social annotation in your stem courses and I'll just say that part of my bias, one of my biases here is like, did they start reading in biochemistry and math is it really about reading so you know one of my questions, you know, underlying this is, you know, to curious about the importance or the role of the reading in your courses and in addition to, you know what first read a social annotation so Emily why don't you tackle this one first. So I think in the 2015 2016 school year we started a biochemistry major in addition to a chemistry major and we had to develop a second semester biochemistry course that was really a continuation of biochemistry one that continued with lipid metabolism nitrogen metabolism topics that we didn't cover in that first semester course so I got to design that course and I designed some activities for students to work on in the class. And I would draw on papers, some readings for the students to do and I initially was assigning those outside of class and the students were not doing them completely not doing them, showing up to class and then the time that I thought we would be using for discussion and working in groups answering questions, instead was spent letting the students silently read, and then students were complaining why are we reading in class. You could be reading outside of class I'm like you could be reading outside of class but you weren't and that's why we're reading in class. It was a really funny kind of like chicken and egg problem I had for a few semesters. And so finally, in the spring of 2020, I started using hypothesis and that was really fortunate because this is a face to face class, but using hypothesis is a really robust way of having asynchronous student student interactions and that really set me up for the class in that crazy semester where we had to pivot online, halfway through so that was just really lucky. But what I love about hypothesis is it's the one way I actually can get students to do the readings or at least engage with portions of the readings. I guess I don't have a guarantee that they read every word but they're certainly opening the readings and engaging with them and I can see the comments that they're making and they're responding to their classmates and so for me. It's the closest thing to kind of a miracle cure or a magic bullet that I found in education for really prioritizing reading in everyone's to do list because we have a canvas integration I can make it an assignment I can give a small number of points for it. So there's now a due date and some accountability and a little bit of a carrot. And so I'm really thrilled that I actually can get students to do the reading now and that does let us spend our time in class in kind of more exciting ways. Once you heard it here first from a scientist hypothesis is a miracle cure I quote miracle cure. Let's see if we can get such a sound bite from the mathematician. Ashley tell us about what drew you to social annotation and maybe a little bit about you know, why does reading matter in math. So, what drew me first to social annotation was during that pandemic I was searching everywhere for interesting ways to get that student to student interaction to have an online class. They have student to content student to instructor and student to student interaction student to contents easy. They, they watch the videos they, you know, do the homework, student to instructor feedback, both ways, but the student to student, primarily was being with discussion boards, and they are boring, and they are tedious. And I just, I would come up with these great questions and it just, it just felt so artificial. And so when I heard about hypothesis, I jumped on webinars and I learned about it and I found out an English professor at my college was also using it or thinking about using it. And then that was the main reason, a secondary reason, about two years prior, our department had decided to go. So online ed resources, our textbooks were starting to creep up into the 200 300 $400 level. At that point, we have a self accelerated program that every time we got a new addition, we had to then revamp all of the content for the course, and it was becoming overwhelming with the amounts of work. So we decided if we can just solidify the textbook, then we don't have to continue this complete overhaul every two or three years in this, in this mode of learning. So I was one of the first that decided, Okay, let's do statistics, it's fairly straightforward. There are already great books out there and I curated a textbook on Libra text from the OER open stacks statistics textbook along with some others. In that curation, there were mistakes. So a secondary reason for using hypothesis that first semester in 2020 was to get my students to proofread the online book. So they were able to go through and highlight passages that they didn't understand that didn't make sense or that were mistakes in the, in the curation that I could then go back and correct. And then you had a second part, what was your second part I already forgot. Oh, reading in math, specifically. Mathematics, STEM in general, very precise, very formal language is used in these disciplines, and specifically in math, we make use of undefined terms and unproven statements, because we want to avoid the circular reasoning and the circular definitions that can happen. And so it's very important that our language is very precise. So that kind of reading is very different than reading a novel or reading a poem or reading in history. It's just, it's, it can be difficult for the students that haven't engaged in it. And so students would buy this $200 $300 textbook, and then only open it for homework. So that's not helpful. So how do I get them to see the value of the textbook beyond just homework right and mostly in statistics specifically details like who is funding the study, how it was conducted how data was collected, what tests were being run are super important to the validity and the bias of that study. And so helping the students be able to flesh that out on their own and see that in that social annotation to struggle with that content was was really important and amazing to see happen in real time. Yeah, I'm good. And I hop in here because I want to follow up on the OER thread the open educational resources. The other thing that hypothesis allowed me to do was to adopt an OER into my biochemistry to course. This is something to for many years, but the general chemistry class that I teach which is asynchronous online I developed to be OER way back in 2015. But the text I was looking at for my biochemistry to course biochemistry free for all out of some by some faculty at Oregon. Let's see organs. The PDF is like 3600 some pages long. It is insane. It is completely awkward and very difficult to navigate and I just did not know how I was going to use that textbook for my course and then with hypothesis. I just had to go in and make little mini PDFs that were the little chunks I wanted my students to read anywhere from maybe just six pages to 30 pages but whatever that chunk was, and then I just put those in throughout the semester and so I have a reading that's after every single lecture that I do, and that way I can really direct students attention to the portion of the textbook that's relevant of interest that I really want them to look at. So they really also for me hypothesis and OER went hand in hand and it was a way that I could make an open educational resource more usable for my class. Great. Sorry, I think it's really interesting Emily you say you have them read it both. You have them read it after your lectures or after your lessons. I do the opposite I have them read it before and kind of like prime the pump and get them ready for the vocabulary definitions, even though they may not understand everything. And then they come into the lessons and they already have that vocabulary. So I think that that's really interesting that we do it differently. I do it depends on the reading some of the readings are in advance and some are after and there's when they're when it's going over a weekend sometimes I have two readings for them to do so it kind of depends on how the timing works out. And the different topics but I sometimes have them pre reading, and sometimes they're reading the content after they have the lecture on it and it's interesting because I will get students commenting. I've never heard this before I'm like, wait a minute, I definitely covered that in lecture, and it's a humbling reminder to me that just because I say something does not mean that they hear it. So it actually, it's fun to have it both ways. I think you said something like that Ashley that you kind of want to know you did or Emily right that you want to make sure they crack the book that they've been exposed to because sometimes it takes a few times the lecture, the follow up reading the reading before the lecture that solidifies the discussion that does. So it's really just about, you know, reiterating some of the concepts. So a quick follow up for you Ashley about proofreading the OER so the OER had mistakes, you and your students for reading it. You were able to use them or you work together to note mistakes, and then you were able to change the underlying text, because it's like a distance you could go in and change that stuff even though it came from open stacks or Libra text or whatever. Yes, so Libra text has the ability for you to edit content in the so so you can pull from multiple textbooks into one whole textbook and publish it on their on their website and I believe it's out of UC Davis so I think Davis is going to be able to use this but houses it, and then you can go in and either request that the original author change it or make a copy and change it yourself because it is that open license. And when you copied and make it yourself your at your own. Is it end up as a PDF that you use with hypothesis. I PDF it from the website and then use it inside canvas. That's very cool. I mean, I've talked about that in OER the idea of that circularity of like okay you make your OER comes from different sources, you teach with it, whether students are giving you the feedback or you realize wow that section. People did not get that example let's swap out that example. I mean that's the dream of the OER is to have that kind of iterative process and I've always dreamed that hypothesis could be a way to to help revise or note places for revision. I'm very happy to hear that you're actually doing that because it was just an idea in my head several years ago. Awesome. Well, this is great. Let's talk a little bit about. About how you're using hypothesis in your courses and this is why I'd like to know, you know, like, how do you set it up. What are they reading with hypothesis and how do you set it up. So two annotations one reply thing is it something more rigorous is it different at different times in the semester, and this time let's let's start with you Ashley. We're the very first encounter with hypothesis my students have is to annotate my syllabus in it during the pandemic again you know just essentially and I had been an online teacher for years before I taught every now and then an online class. But it wasn't really until the pandemic where you realize, even though I was able to teach it, I probably wasn't as good as I should have been. I wasn't delivering the best way that I could have been doing. So I was scrambling that summer to just learn as much as I could. I think it just, I don't know it lit a fire in me to to get better at what I do. And one of the things that I had learned about with something called a liquid syllabus, where you essentially create your syllabus, you put it on a website. And so students can see your syllabus before they ever enter your class. It allows them to kind of they already do the the professor shopping right they're looking on rate my professor as terrible as that is and as biased as that is. That's what they use. So, instead of giving them that only one point of which professor should I take. Oh hey, find out how I teach my class before you ever sign up for me. So I created an online syllabus. It was just through Google sites. I have it posted in my faculty profile on my on my colleges website, so that students can go to my faculty profile and click on that syllabus. Then, in in the class, they are able to annotate the syllabus I asked for two annotations for the syllabus, they can add and I just use the hypothesis, you know, syllabus suggested examples like what what should you do right there's a great assignment that hypothesis puts up for a syllabus and they go through and they ask their questions they they you know comment on Wow this is really helpful that you you know you understand we have lives outside of our class or that you're willing to help us or whatever it is that I have in that that they're able to, you know, attach and appreciate and question. And with that, it also removes the need on that first day to read through the syllabus and ask for questions that students tend to just that you know their eyes glaze over they don't really pay attention, allows you to give them other activities, more community building activities so it's kind of the syllabus I say would was really nice because you can do two community building activities. If you're doing an in person class you do one in person, and then you get them to do the community building on the syllabus, and they're making connections with hypothesis in the syllabus online. After that, I usually separate my textbook by chapters. So I have the chapter that they're they're going to be learning at the very beginning of the module in my online class, or even in my in person class I still use canvas to interact with my students in class, and they read through this section that they're going to do and I have them post five annotations, most of them, and it can be anything it can be a reply or it can be just a straight out you know, annotation directly to the, to the text. They annotate the chapters throughout the semester and they tend to go back because we're doing like one or two sections per class. So even though I have the whole chapter instead of a section, I kind of like Emily's idea of breaking it up I may think about breaking it up into sections instead of chapters to see if that might get a little more engagement but yeah. I think that's. That's what I do. Can you tell us a little bit about how you use hypothesis. Yeah, so I use hypothesis really robustly in my biochemistry to class which is an upper division second semester biochemistry class. I also have the students annotate the syllabus at the beginning. And, like I mentioned I have these small chunked readings that there's at least one after every class and sometimes I have two readings, most often their excerpts from that biochemistry free for all textbook, but I also use journal articles, especially retrospectives that maybe scientists write about their biggest discoveries that kind of include some research data and some experimentation but it's really from many different papers and they're kind of summarizing the greatest hits these could be Nobel Prize winners and those are kind of fun types of articles for students to get to read to learn about some really major discoveries but also get a chance to analyze some data. So, the majority of my readings are from the textbook but I also have selected journal articles. We've been doing a unit on coven 19 and so even just pulling in something from the World Health Organization about RNA vaccines or you know I can pull in also, even though like the New York Times article that's talking about the journey to making the vaccines I don't know a variety of different sources can come in and support the material that I'm covering so that the students in that class have quite a lot of assignments. I have also a large general chemistry course it's asynchronous online I usually have about three students, and I've been experimenting with just a single hypothesis assignment in groups in that class, and I've used those assignments to get my students to identify elements like claim evidence and reasoning in a text, and have kind of done that for different people depending on are they the first couple people to respond or the third, fourth, fifth person to respond. So that's been fun to play around with and then the other area is general chemistry one is a general studies course and so there's kind of some broad learning objectives that could apply to any introduction to science class or introductory science class and those I always find very challenging. And what I mean by this is, one of these learning objectives is think critically evaluate the credibility of scientific information and interpret the impact of its use or misuse on society, I mean that is huge you guys huge, and I'm teaching a class already has so much content I'm sure. But I found an interesting reading about using technology to detect fake science and the students read that and we had it in hypothesis they were able to have conversations about, you know what is fake science what are different claims they've heard people make how do you analyze things and I realized one reason this is such a hard thing for me to teach is it has elements of psychology and elements of statistics and there's a lot of knowledge that goes into being able to identify the misuse of science so that again that's a very broad learning objective that's been given to me for my course that I've really struggled with how to teach that and I've explored using hypothesis to address that with my students. It reminded me that I also use newspaper articles and online online articles as well as I love incorporating history into mathematics because it just adds that much more and you're exactly right Emily, it makes it relevant to the students and it makes it more interesting than just that dry old textbook. One of my favorite stories from statistics is the story of the student tea distribution and how William Gossett working at Guinness published under that name you know the Guinness Brewing Company and it's like did you know the tea distribution was used in brewing beer and they're like wait what so students get all into that kind of stuff and it's it's just so much fun 2020 was the election and so we had lots of annotations on articles about polling and I try to stay away from partisan politics type stuff it's it's kind of it's a it's a touchy subject in general and so you know so we look at process how how did they pull what you know what is the saying about the polling not about the the you know politics itself. And so yeah it was really rich conversations inside a hypothesis. So one of the, one of the questions that we get a lot from folks who are newer type hypothesis is you know what, how do I get them to annotate how do I direct them, and you guys suggested a few ways and I I'll read it right then but I've also be just curious that you have anything more to say about how you prompt students to have those good discussions that you just mentioned Ashley. I do want to give a shout out actually mentioned the syllabus annotation starter assignment we do have starter assignments so we do have kind of stock language that you can put at the top of an assignment, not just for the syllabus but for a variety of different types of activities that our CS team has created so that's a great resource. Emily I was super fascinated about your point about different prompts at different points in a reading. So I just would like to hear a little bit more from you guys about how you prompt students there's a reading, whether it's the textbook or an article, and you know what's the direction, and does that change over time and maybe Ashley will start with you. Sure. Within the content, a lot of times I'll sprinkle in questions. And so I will create the assignment and then I'll immediately go in and add questions in where I think there might be confusion where there might need clarification or I want them to think a little bit harder about. You know, helping them start that conversation absolutely. And then most of the time, it's the, you know, allowing them the freedom to express themselves in, you know, post a meme that's relevant. You know, some of students have gone out and found memes relevant to statistics are relevant to you know whatever it is we're talking about probability, what have you, and they'll post them in there and get a good laugh or videos that you know a lot of them will reach out for supplemental videos and if they find one. Oh, this explains what a stochastic process is and they'll pop that in. And so other students can see to. So really the guiding questions within and then also just allowing them that freedom to to make their own kind of I kind of like to tell them you're creating this textbook in high, you know you're creating these annotations and hypothesis as a class wide cheat sheet for the for the test. Because in my online classes I do allow open notes open book. And so have the ability to go back and see all of these annotations that they can create as a class, and not just individually. I love that too it's a very open education right you're creating I don't know if I'd use the term cheat sheet but you're creating the textbook is a kind of base material and you're creating something that's a resource for you and your classmates. Moving on you're going to add to it you're going to add the memes that make it more interesting you're going to add the videos that supplement one question though Ashley though is that do you have someplace in your. You know curriculum material where you're saying you know here's some ways you could reply you could create annotations like you can add memes you can have videos so you're teaching them a little bit about the possibilities of what can be done in an annotation. Absolutely at the very beginning with the syllabus annotation assignment as well as within each assignment there there are some guiding, guiding questions or guiding instructions. Here are some things you can do. When you see a word you don't understand go find the definition when you see, you know, an example that you, you know, maybe it doesn't have the complete answer try to answer it and post it for other students to take a look. So that kind of stuff. Great. If that's something you're willing to share we'd love to add it to our open education resource collection of social annotation materials. Emily tell us a little bit more about how you prompt students in the annotation practice. Most of the time it's really just very simple the requirement is to make either three annotations or responses to classmates annotations and. So I just explained that it could be pretty open like things that you found interesting things you would like to know more about. Responding to classmates is super helpful I really love it when there's sometimes there's some students who just love to ask questions and it's really nice when there's other students that love to answer those questions. So what I do then is I review the annotations, a couple hours before I go to lecture for this biochemistry to class which is a face to face class. And I use that to guide the beginning of my lecture so if there's an area that students have flagged questions on I can talk about that some at the beginning of my lecture can say oh okay we want to go a little deeper on this topic let's explore some more here or I realize there's a confusion on this. Let's clarify or even just wow your classmate shared this really cool resource I wanted to bring it to everyone's attention if you haven't come back and seen it so. In general I keep it very simple. There are some specific assignments where I give more specific guidance but in general, it's pretty free form they get to engage with the reading the way they want. And Emily do you annotate it all beside them or ahead of them or. I don't typically annotate ahead of them sometimes it's just things like start here. If this is a part of the textbook or, you know, don't forget to do this, you know, be sure to go down to the bottom to there's good stuff down there like so sometimes there's some guidance like that. I wait to make comments until again shortly before the lecture and then if there's questions that haven't been answered I do try to answer them at that time. Okay. So I just have one more question formally for the for you guys and I just want to remind our audience that if you guys have questions for the panelists, I would love to have you put those in the q amp a and we can surface them in our live q amp a. You know, I turned off my sharing so that people could see the fullness of our faces. I think that helps with the with the zoom experience. But the sort of final formal question is I'm just curious what kind of results you've seen from social annotation and I realized that maybe you haven't conducted an IRB approved study. So those results. This may be anecdotal data. I'm also curious, you know, if you've had feedback from students about the use of the tool. So yeah just what kind of results are you seeing from social annotation in your STEM courses and maybe we'll start with you, Ashley. Improved definition improved understanding of definitions improved understanding of concepts. Seeing it or listening to it or hearing it one time isn't enough and and you know trying to get the students to understand that the time outside of class is meant to study as well as homework. So we talked about the you know the curve of understanding the curve of remembering and and just reminding them that you can't just see it one time and then remember it you've got to see it over and over and over again. And so allowing them that multiple ways of getting that content is incredibly important across the board more engagement more student to student interaction. Not only in my online class because that is obvious when you're looking at the hypothesis annotation there, but also when I use it in my in person class. The students are making connections online and then they come into class and they formalize those connections in person. And a lot of that is happening which is really, it's just beautiful to see them working together, it really is. And then I have one quote the students response to social annotation, most of them just are fascinated by the way this works because when I first saw it, it looked almost like Facebook or, you know, I don't know that you could set it up with if lots of videos it could be like tiktok or Instagram or what have you. And so them already living so heavily online, and allowing them another way to engage in education with another reason that really drew it drew it to me but one quote that I got from a student really early my first semester of using it asynchronous learning is difficult, but annotating the textbook as a class and conversing via discussion posts are good ways to virtually do collaborative exercises and learning. So my students are really valuing the, the work that they're doing in hypothesis. I think I have a quote here from a mathematician saying that hypothesis is the next tiktok. That's what you said, pretty much. We can we can edit it. I actually actually I do want to just draw something out you mentioned this before and you just reminded me, you've been talking about you talked about student to content interactions to the student interactions and student teacher interactions. You're not there without mentioning the, the RSI standards but that's what you're alluding to the kind of regular substantive interaction kind of online standard is that kind of in the back of your mind there. Yes, anytime we're teaching an online class we have those regulations in the back of our heads. You want to think that your job is this beautiful thing that you can you know you have lots of freedom to do but you, you have regulations in the background you have to do these things and you have to check those boxes so that you know accreditation standards are met etc etc. Those those three things I mean probably abstractly they seem valuable like student to content you want them to read student to student you want them to interact peer peer. Build knowledge together and student to teacher you'd like there to be a teacher present and for you with those RSI stands in the back of your head you found hypothesis to be useful for kind of fulfilling them. Absolutely. And yes, I mean they are absolutely valuable. It's the only it makes sense for learning you have to have these three, you know, pillars for learning. And I want to push on the instruct student to instructor because it does have to go both ways. So student instructors to students that's really easy because that's in that feedback that's in the grading that's in, you know, us telling them how they're doing. But I would like to encourage anyone considering any kind of teaching to make sure you're getting that feedback from your students constantly asking them how am I doing. How are how do you think you're doing in this class right so I have a whole level of metacognition and that throughout my class where students are talking about what are they what are they learning what are they struggling with how are they going to correct that struggling how are they going to understand that or what resources are they going to use, and then also, you know what kind what how can I all throughout my classes how can I improve what am I doing that you need me to see change. I'm not so much but I feel like and there's a strength strand with both of you here right if you start off by annotating the syllabus and your sister syllabus is liquid as you described and you're starting off by saying, I want to hear your voice. You know, and you have now a vehicle for that voice which is this tool you can ask a question. When he was talking about flagging a question you're building a culture where that bidirectional feedback from students coming to you is, there's a there's a platform for it. I think, not just in social orientation but more generally and it's a very powerful kind of reconfiguring of the classroom, I commend you on it. What was the question I just got wrapped up Oh, what kind of results have you seen Emily in your courses and again, any kind of student feedback qualitative, you know, thoughts from you anecdotal would be most welcome. I just want to say, actually, I love that you've just clarified for me that this thing I'm getting one of the things I'm getting from hypothesis is that student to instructor interaction instead of just instructor to student. What I want to say about that is, there's always the quiet students or the students that if you're doing, if you have a face to face class are not as likely to speak up in the classroom they're not as prone to ask those questions. But I love about hypothesis is everyone gets to have their ideas heard their voice heard through those annotations and so I think for people that aren't as quick to speak up verbally it can be hard to, to know what those students are thinking or where they're at and hypothesis really glimpse into their thoughts students make so many cool connections between research projects that they're engaged in things they've learned in other classes, like we're really, they're building a web of connections and sharing that with each other and I think it's super cool I also love the positive affect like students say whoa that's so cool I never knew that, and just bringing some of this wonder and awe that gets people really juiced up and also helps us remember things better that's one thing I like, as far as kind of more of a data approach about 85% of my students from spring 2020 through this past spring 2023 semester since I've been using hypothesis about 85% of my students are getting 90 plus points from the annotations I would say 95 to 100% of the points. So the vast majority of my students are engaging to the full extent that I'm asking them to. I do find that there's a handful of students that are maybe just doing about half of what I'm asking them to do, but I've never had a student do zero. At least doing half of what I'm wanting but the majority of them are fully engaging in the way that I hoped. So that, you know, to me is encouraging and students really do seem to enjoy being able to share those thoughts with each other to share the videos and answers to questions and I think it really helps students when readings are hard, especially to feel less alone the fact that someone else had to look up a definition to her word and thought to share it. So sometimes say thank you for doing that I was going to have to look that up to, and it normalizes the fact that what we're doing is hard, it's hard but doable and I think that's really a beautiful thing. I had all sounds of I had all those sounds of a scent through what you were just saying but I was on mute. I love that yeah normalize that it's hard, I think that that's really important. You know, I remember times in college and in grad school feeling like I don't understand this reading maybe I don't belong here. And if you see that other people are asking questions that you know there's a process here you look something up when you understand it, and that you can ask questions and that you can learn from other people then you realize okay that's just part of the work that we do. It is supposed to be hard. And part of it in this tool you know helps us do that kind of work. Quick question. Emily you were saying is hypothesis graded activity for you that contributes to a final grade. And it's a kind of a minor person is like kind of a participation grade. So each little reading assignment is worth three points, and they get one point per annotation, you know again it can be original annotation or response to a classmate so it's very easy for me to grade. Did you put three things into this reading, you get your three points, and you just did to give you two points and you did five I'll give you a happy comment in there to say that was awesome thank you. But so it's a very simple system, but I think it just helps there be some accountability. Do you use the, these speed grader in canvas and I do exactly. And so do you do use the feedback tool where you give private feedback or is it mostly just the grades okay. Especially when those students go above and beyond I like to make a little comment, or if there was. You know usually if there's something I want to respond to I'll just respond in the thread if, you know, I don't know if there's something that just seems to be more appropriate to be one on one to the student I would also use that that feedback there. And I say you're grading, I do the same. So a substantive comment so I can't just be oh yay that's great or I didn't think about that before it needs to be a little bit more than that. But minor 10 point assignments five annotations two points per annotation similar, you know it is like a participation grade. And I try to give my feedback within hypothesis so that it is visible. If it's a, if it's not a substantive comment I do do that privately for the student just to, you know, not call them out and say hey I need to do a little bit better job. You know that that should be within just for the student but if there are questions that they bring up or whatnot directly in hypothesis yeah. So one question that I've seen come up that I think I'm sort of interested. I guess we do have a question about grading. And I think we just addressed it, organically. So that's good. Oh you aggressive address it to actually you're all over the place. But one question I don't know if you guys have something to say about this I did try to get somebody from a K 12 school to to join us today to give us that perspective. And I know you guys are both teaching at the college level. Although you're seeing some students I guess who may still be in high school and some of those programs you were describing Ashley, but both of you I think or especially as you're talking about students that are very new to the discipline that you're talking about into the kind of level of discipline that you're talking about but either of you guys have thoughts about hypothesis in stem. Before, before the college level, or is it any different or could you see a use case there's value Ashley I see the beginning of a thought coming from your mouth before. So, so in that middle college program I'm teaching primarily juniors in high school. So my entire intermediate algebra class in the fall semester I'm gearing up for at this coming fall entire classes juniors in high school. And even though they're on college and they're earning college credit they are still considered juniors at their home at their schools. And so I, I personally don't see it being difficult to incorporate at the high school level. I really can. I mean the students are so savvy with technology. I absolutely could see the value in using a high school textbook or or journals for any kind of discipline or readings that they're doing online and having those students create that class wide social annotation in that content. I've been teaching juniors that I see almost every semester for the past four years, I think I've taught three. Yeah, I've taught three classes in this cohort, and, and they, they excel at it, they engage in it very easily and very well in in the annotation. I mean it strikes me that you're, you're coming about belonging. The earlier we can get students thinking that they belong to the day that they that this reading is something they can do that that science or math or whatever it is is something that they are capable of and could professionalize in is great. What are your personal thoughts about K to 12 and stem and social annotation. Yeah, honestly, anytime you want students to read something and extract meaning from that written text, this could work well because it's directing them to the text and asking them to annotate what what they're reading from the text so and we expect that all over the place I could see using this for pre labs, you know asking the student to read the lab in advance and make some annotations before coming into the classroom. You know, they can be short readings. I think keeping keeping things very short and to the point, so that we're being strategic with our students time is helpful, you know, just a few pages but it's very relevant to your course. It's great for your students directing their attention to. It's great. And I will just add that we do see K through 12 usage of hypothesis, obviously it's not major focus for us yet, but I taught high school for many years and believe that this tool definitely scales down to middle school and high school and maybe because I think as you said Emily, any, any reading can be difficult at different times in your education process and having help having a tool to dig into it and make that a social experiences is powerful. There's one question from anonymous entity. I think that we did cover this. I just wanted to see if there's any other thoughts just around the difficulty of coming up with original substantive comments. This was a, I think kind of a little bit of an echo something that you said, Ashley, like, you're asking for substantive comments. I can't just be like, yeah, I agree. You want something more. It sounds like both of you have nurtured a class, a classroom culture where students are saying quite a bit, but do you have any pro tips for, you know, people that are just getting a lot of I agrees or having a hard time students having a hard time coming up with some some comments start with you Ashley. I haven't had a good opportunity for that yet maybe my classes are 35 to 40 so it's not a whole lot. I don't consider that terribly large. So they seem to be doing a pretty good job with it. I would probably suggest that if they're they're just drawing a blank like I have no idea what's right. Okay, well read through it read through comments, and then maybe look at an example, and take the example, because it's math, take the example and try to work it out on and then use your own words to explain how you did it, not what they're doing, but maybe, you know, did you do it differently. Can you see doing it differently. Would you approach it in a different way. Just trying to get like pushing them a little bit of, you know, you, you have the ability to be creative, even in mathematics. There are, you know, options and opportunities for that so but but sometimes some students I have not encountered it personally but I can see that some students might need a little push they might need a little bit of hand holding to get started. But once you give them the idea, most of them are are okay with it and they'll take it and go. I agree setting norms early in the semester is important and that's when you can put most of your time into doing that gentle encouragement like you know really I'm looking for something more substantial I will say though, because the students are centered on a reading. It's much easier to have substantive comments than if you're just doing a discussion post. So I really feel like that just the nature of hypothesis leads to more substantive responses and interactions because we're grounded in that text. I need to let that one sit because there was such a profound encapsulation of the power of annotation in comparison to something like a discussion board you have context you're not it's not a blank page there's something to draw on. That's beautiful. I'll just add that I think both of Ashley and Emily have sort of described this that they have a little, I don't know what you call it like a set of ways to interact with the text through annotation right could be a meme could be this right so questions things like that. So we see a lot of teachers developing that bank of like, I want three annotations on this your annotations could be any of these things and they could be dozens of things right the question could be a definition could be a meme could be. You know, a paraphrase, maybe not a paraphrase, but you know, lots of options depending on the class. One question one piece of the question from the anonymous attendees about the size of the class. And I'll just add that hypothesis has a grouping mechanism and all LMS integration so in canvas, you know it can map to canvas groups. And so even in a large class, you can segment that down into a smaller group of students so that you know it is true if you have 100 students on a five page text that a lot of real estate and ideas get taken up quickly. You can always break down a class into smaller groups. So that there's more space for students to interact with the text and, and get their voice heard. Wow, I am just really with inspiration from this conversation. Your students, Ashley and Emily at at Las Rositas and Metro State are incredibly lucky. They're very lucky to have dedicated inspiring instructors like you working, especially in public education in our country. So, you know, I'm just thrilled to have had this conversation I do have a couple housekeeping things to finish with here. So if your school is not yet using hypothesis, we are offering a fall back to school special. So you can reach out to education at hypothesis and get hypothesis installed eager learning management system to do the kinds of things that Emily and Ashley are talking about here we have a discounted deal here just starting off and includes unlimited access to the tool and part of what it includes and this is also a thank you to the customer success management team at hypothesis who have been fielding some of the Q&As and dropping things dropping resources into the chat. We have a phenomenal customer success management manager team here customer success team here. They build out resources activities assignments like the one that actually described, they lead workshops for all our partners they design custom workshops for specific campuses and specific disciplines. It really is one of the things that makes hypothesis different probably has something to do with the fact that a lot of us are educators and don't really want to leave the classroom and so are still writing assignments and creating workshops and training modules for for for our colleagues that are still in the classroom so it's really something to do with the hypothesis the, the success team and the success resources. Unfortunately, our summer workshop series is over you missed it. They're all recorded in on YouTube. And you can stay tuned for our work our back to school workshop series which will kick off, you know, in August. And again, it goes through things like annotation starter assignments. It goes to things like creative ways to use hypothesis how to use multimedia. And grading that we just had yesterday that went really well shout out to Suzanne Miller. So, stay tuned for fall workshops. We also have hypothesis Academy, which is a asynchronous to each course. There's a one oh one version which is really an introduction to hypothesis. It's a really cool way to actually get to know hypothesis but also to dig deeper Emily Ashley by you done hypothesis Academy yet. We have lots of veterans who take it. It's a great opportunity to deepen the practice. We do have another cohort for one oh one starting on August 1. And then we also have now a social annotation in the age of AI. How did we get through an hour of conversation without talking about chat GPT. It's amazing it's first time in the hour since last December that I was able to go an hour without talking about chat GPT. We were old school today that's great. And we are engaging in conversation around chat GPT and the pedagogy of chat GPT and how to discourage the use of chat GPT through certain kinds of assignments, but also how to engage with chat GPT and I think hypothesis can be useful in both those trajectories. With that, I just want to say thank you to the audience for joining us today. Thanks to the CS team and to the marketing team for their support on the call. And then most of all, thank you Emily and Ashley. I mean, I'm not just saying that I'm really impressed and inspired by the conversation and just excited that I have my job to be able to talk to people like you and that we can spread the word about this cool tool and how it can be used and super interesting conversation really appreciate you both. Thanks Jeremy, this is fun. Thank you. Bye everybody. Have a great afternoon.