 Alright, so we'll get started with just some introductions and everything as some folks join in, please keep in mind throughout the session that if you have any questions, we would love to hear from you in our Q&A function. So you can drop questions into that Q&A box and I'll be passing them along to our panelists as we go. You can also turn on closed captioning as part of your Zoom menu if that would enhance your accessibility experience. So again, welcome everyone to our session of liquid margins today. Hi, my name is Christy DeCarolis. I'm a customer success manager here at Hypothesis and an instructional designer. And I also adjunct and use hypothesis in my own courses, but really I'm just serving as our emcee for the day. And we have two panelists that are joining us and I will let them introduce themselves. Christy, do you mind starting with a quick introduction? Sure. Hi everyone, I'm Kristen Palosado. And as you can see on the slide, I'm at Kingsborough Community College in New York City. And our campus is right on the ocean, like it's 50 feet from my window here, although it's terrible weather today. And I'm really happy to be here and share my experiences using Hypothesis. Thanks so much for joining, Kristen. Yeah, I'm in New Jersey in the East Coast really. I feel like it's been raining here for like, I don't know. I'm Francie, do you mind introducing yourself? Hi, I'm Francie Quasburyman. I teach at Cerritos College, which is in Southern California. So I'm on the opposite side of the country right now. I think I said I teach English. Yeah, there. Thanks so much. Hopefully it's not raining so much there. No, it's very sunny, although I'm heading your way in a couple of days. So I hope it's sunny when I get there. Yeah, hopefully it'll clear up a little bit. Alright, so now that we've done our introductions, I'm going to stop my screen share so we can dive into our discussion about Hypothesis. So I first just want to start out by getting a sense of how you've used Hypothesis in your courses before we get into results. So Francie, since you went second for introductions, do you mind just sharing what problems you were hoping to solve with Hypothesis? Like why did you turn into Hypothesis in the first place? When I first tried out Hypothesis, it was in my readings in drama class. And when I was originally scheduled to teach the class, it was pre-pandemic. So I wasn't expecting to teach it online. And the first semester, quite honestly, was a disaster. I wasn't using Hypothesis at the time. I was trying to just do regular discussion boards. So they were reading in one place and then discussing the text in another place and the engagement was low. It was just, it was miserable. And then the next year, but before I taught it the next time, I was introduced to Hypothesis and it was really a game changer because the discussion wasn't happening separately from the reading. They had the reading and they could, I could post resources for them to help them understand how to read the play and then they could discuss the play in the same space. So I was able to layer resources and the discussion together and it transformed the engagement in that process. So that was how I got started with it. Hi, while we're on that subject, can you explain a little bit more how you integrate it into your courses? Do you, like, do you use it with every reading that the students are doing? Or do you just use it in very specific instances? So now I'm using it, I use it for all kinds of things. Obviously in the drama class, we're reading the plays using that. In my freshman comp class, I'm using it with the readings. I'm using it with YouTube videos that they're watching. I'm having them review the syllabus on Hypothesis so they can ask questions on the syllabus. I'm having them read the essay prompts that way. So I'm using it for all kinds of things now. And again, all my classes are online. So it's a way for me to answer questions for everyone, especially when we're looking at the essay prompts. I can answer questions for everyone in that space so everyone can see the answers. I can also see if they're focusing on the things I want them to focus on and if they're not, I can say, hey, I didn't see anybody commenting on this, but I want to bring that to your attention. So it's a way for me to monitor what they're focusing on as they're reading. It's a way to engage in a conversation about the reading. It's just, there's so many things you can do with it. Yeah, I can't imagine teaching without it at this point. I totally agree. Thanks for sharing. And Kristen, what about you? What problems were you hoping to solve with Hypothesis? What made you turn to the tool? So other people on my campus faculty had been using this tool and there was a little bit of buzz. And I was like, oh, okay, Hypothesis must be definitely STEM related. But it wasn't at all. And so at first I was like, oh, okay, well, I don't need, we don't annotate in this class. Right. But then, as we were merging from the pandemic. And I realized that, you know, something terrible happened to these poor students during this pandemic. And they were really struggling to climb back out of this hole. And, you know, we're doing, we went not back to fully in person, but back to hybrid classes. So they didn't have me as this like live guide. Let's read this little section together. Let's discuss this. They just saw me in lab. And that was it. And they were just struggling with this material because they needed to discuss it and talk. And discussion group was not the way to do that. I'm sure a lot of you have had that experience with discussion groups on the learning management system. So I took, I think it was the first cohort that Christie did this Hypothesis Academy was like two years ago now. Anyway, whenever it was a year ago, maybe. And I was like, Oh, this could really solve that problem of a way to discuss the material and interact about it. That could actually work virtually, not like the, I don't know why the discussion board wasn't the way I envisioned, but it wasn't. So I thought, let me try this because it's very user friendly. And it's quick for students and kind of like Francie was saying, when you're reading is when you're, you're responding like in your head usually. But if you can make the notes right there to the other people and the instructor, that's much better than having to remember it later in class to bring it up. Anyway, or to write it in a separate notebook. So that's the problem I was looking to solve is that kind of engagement and interaction. When you're not together that gets the same result that that you get when you are together. So that was my journey to starting using it. So, okay, so it sounds like Francie is teaching online courses and you're teaching hybrid courses. Did you notice that the annotations impacted your in person meetings at all after the students were annotating. Yes. So, first of all, they were getting to know each other a little better, which was almost like, I didn't think that would happen. I thought, oh, you get to know people when you're sitting next to them in lab and working on something together and they did, but they I would hear them referring back to a conversation they've been having on hypothesis and like extending it when they came into lab, and we're working, or Oh, did you see so and so posted a video about this that can help explain it better. If you look on the PowerPoint we were annotating or whatever. And that was really nice than what and that's what kind of convinced me. Oh, this really is an interaction they're not I was grading it. It was for a grade the annotations but they were doing it for other reasons to which is why not just to get those points. It's great that they're kind of bringing it in. I guess that's similar to like them getting to know each other on like Snapchat or tick tock or whatever the hot social media of the day is, and you know, I don't even know at this point. To kind of go back to those particular conversations and hypothesis. When I talk to faculty I hear a lot about it can be difficult to get students to make meaningful annotations and to have conversations that are going to impact their learning. Francy, what kinds of prompts or instructions do you typically give to students in order to make those successful conversations happen. So I also did the hypothesis Academy so I've used a lot of the strategies that I got from that so my favorite one is the sit responses so they have to identify something surprising. Something interesting and something troubling or tricky. So I've used that one a lot. I also I do have word count requirements on the responses and they have so and they have a certain number they have to do. But one of the things I tried to do to kind of relax them a little bit about that is I just did here's what you know 50 words looks like here's what 100 words look like here's what 25 words looks like. So they could see it's not really that much. If you just have a thoughtful response, you can pretty easily come up with that word count so don't get hung up on the word count. Try to think about what do I think about this and trying to push them to think more deeply about that. So that's usually how I get them engaged in the readings and drama class I do push them to engage throughout the play. I have had students who they they're just hoop jumping and okay I got my couple of things in. But I'm also not giving full credit for that you know I want to see that you're engaging with the whole play. And once they do that and I can and this this last week was fun we were reading the curious incident of the dog in the nighttime. And it's a mystery. And as things were happening you know they were trying to make predictions because we were focusing on foreshadowing and they were trying to make predictions and then when they got to some of the reveals. And it was crazy in the comments and it was fun to watch that interaction happening as a word. Oh my gosh I didn't expect to see that happening either. Oh, you know and I really appreciated seeing that level of engagement with the play so. The requirements help, but once they can get engaged in the material I think sometimes in it. Like you were saying Christian kind of takes on a life of its own. And you also mentioned that sometimes you'll go and like add your own annotations to add additional resources or context. Do students typically like reply to those or they just like standalone. And do students ever bring in outside resources as well. Yes. So in the readings and drama class I am trying to help them understand how to read the plays so there's, I'm trying to show the structure here's a structure of the play for this particular play here's some things to watch for look like you can see it happening right here. So I do add that I also add a lot of resources a lot of the plays we're reading are based on actual historical events. So I provide a lot of resources that will help them make those connections and see those connections. And I encourage them hey if you find a link to a production or if you find, you know somebody doing a monologue or performing this. Yes, share that video with us and they do. So I always appreciate that I don't participate so much in the discussions in the English 100 because I'm trying not to sway because they're persuasive topics. And I don't want to sway they're thinking about those one way or the other I really want them to be free to discuss those ideas. So in the first reading I have them do I do have questions and I show them how to do things but it's me demonstrating here's what it looks like, and giving them space to respond but once they've learned how to use the tool. Then I kind of stay out of it a little bit more until we're doing the responses because we'll do the initial post and then we'll have another time space to do the responses to those. And then I'll get in and comment a little bit, but I'm trying to be a little bit hands off in there. I'm much more engaged in the drama class because we're discussing the text in a different way. Yeah, that definitely makes sense to change the way you're interacting depending on what the goals for the particular text are. Kristen, I heard you mentioned in our earlier comments something about students annotating PowerPoints. What types of documents do you typically have students annotating and what kinds of instructions do you give them in order to get them to make annotations that are meaningful or helpful for their learning. It's funny that I threw that out there because that's the type of document they least often annotate. PowerPoints that much for my for what we're doing, but I start like Francis that I started with the syllabus, which is not when I first used hypothesis that's not what I was doing. But now I do, I ended up taking a professional development seminar on equity and revamped our syllabi and then one of the things I wanted to do was have student input into the syllabus that used hypothesis to do it. So that was one type of document. And then videos that are short that are about a particular concept for the week. And then I have a list of very detailed learning objectives for each week. And I've had them annotate that. And sometimes I have them annotate research articles like primary literature or even an article from the newspaper that relates to our topic for the week. So whatever I feel like the possibilities are endless. I keep finding new new things they can do with hypothesis as far as as helping them to make a comment that's adding to their learning. In the in the Hypothesis Academy, there were a lot of example assignments, and I borrowed heavily from the first. And what surprised me so much, because of my experience with discussion boards where you had to keep repeating the rubric even though it was like, clear, they just weren't earning the points because they weren't saying anything to substantive or whatever. With, with the instructions we had from Hypothesis, they seem to get it instantly the whole class, not just that upper third that you know, things come easy for but everyone understood how to make a comment that was substantive or that was a question or that was a disagreement with the preview. It was so easy for them. And so I just keep using those same, you know, modified a little bit to fit my classes, instructions that I got in the very first Hypothesis Academy and students are doing it brilliantly and then they take off on their own like we've been saying you don't have to worry at all. And for Auntie mentioned a couple of different approaches with how much she participates in the annotations. What do you tend to do do you kind of like let the students go off on their own conversations, or do you go in and like model annotations, especially with like research articles and things like that. Yeah, totally like you said, totally depends what it is. But I haven't found that I always kind of check as they're getting started with something that they, you know, not quite sure what to say or, but I haven't found that I need to say much. And lately I start just scanning for the spots where someone saying maybe the professor can chime in when they're not sure of an answer. Right. And that's the only place I need to say anything. But if if we're like working on the syllabus together, then I have a lot more to say back to them. So it just depends what kind of, you know, activity we're engaged in. Yeah, that makes sense. And the thing that's the most intriguing for me and kind of connects to, I think that the case that you will look at a little bit more later is how you had them annotate learning objectives. What types of annotations are they making when when they're reviewing those learning objectives. So I used, I don't remember the acronym, but some kind of acronym so they could do one of four or five things. Right. So either they were going to write that the learning objectives are worded kind of like an exam question, like a short essay. They could just answer it in their own words, which I was like, I'm just going to ask AI to do it, but it doesn't seem like many of them did. But, or they could ask a clarifying question about it, or they could bring in an outside resource to post that they thought was good for explaining that objective. And those are only three, but there were like four things that could do in response to that to get their credit. And it's usually a list of like 10 or 12 objectives for the week, and they had to post five five annotations and respond to a couple of other people's something like that. Okay, great. It sounds like it'll be an interesting. That's one thing I haven't had my students see as an IT learning objectives. So I'm trying to get my own ideas here too. Okay, the only problem with that was I should have done it in groups, because when you got only 12 objectives for the week and there's 20 people commenting, they can't come up with enough new stuff to say, and it's very messy. It's not not very functional. And I didn't have them. I didn't have them do that the next semester, but if I did it again, it would definitely be in smaller groups. Okay, yeah, that makes a lot of sense. Great. So we have heard about kind of how you've been using hypothesis in the classroom. Can you talk a little bit about the changes you've seen in your own teaching style and in student learning in your courses since you started using hypothesis teaching style. If any, it could be nothing. I think that with the reading and the changes in the reading that there's just more engagement with it, because before, okay, I'm not going to have a discussion board about everything. I would have, you know, sometimes have them read things and then it's, you're going to use it later, but we're not going to necessarily do anything with it now. And I know full well when I do that nobody's reading it, they're just not. So, I think now because I am having them engage in some way with all of the texts that that's been a big change. And so it's also trying to balance the workload, knowing that now I'm asking them to engage in the tech so it is going to take longer for them to go through this particular reading. So maybe I trim it down and we don't read quite so many things we just focus more deeply on a couple of things that are going to really help us develop the essays that we're going to develop. And then with the readings and drama class it's completely different because again before the resources would be there but they weren't, I wasn't showing them where the resources connected to the text, where now I can show them exactly where those resources connect to the text, because before all the resources would be over here and they're on a list and they're available for you to look at and I'll have information about how they connect, but you still have to make that that leap. And so now I can make it right there for them. Yeah, and sometimes I always wonder if it's like separate will they actually go click on it, but if it's something that they need when they're reading maybe if it's right there they'll actually click because they're like oh this can help me understand. So do you this might I'm not sure how difficult this may be to kind of assess with fully online courses but do you feel like the interactions among your students have changed with hypothesis. Well, it from what I see online I would say yes, I don't. Well there are still definitely students who are hoop jumping for points. I also see that there are students who are genuinely engaged in the dialogue. Where I when it was on the discussion board, I just, it all felt like hoop jumping. It's, you know, here's my post here's my two responses. That was it. But now I see far more students who are going above and beyond what the minimum expectation is, and really engaging with their classmates, and I can see it because it's so easy to see. And like you were talking about answering questions to the tags make that so much easier because I can go. I encourage them to use the tags, you know so they could put question or whatever and then it's easier for me to go through and spot those and respond quickly to any questions that pop up. Yeah, I think that's probably the most frequent use of the tags that I see is people trying to, you know, really easily surface the questions that come up. So if you haven't used those before Kristen that might be a good thing to explore in the future. So what about you Kristen has any other thing about your teaching approach changed since you started implementing hypothesis. Yeah, so because I had done this shotgun approach let's use hypothesis for everything. I could see now how students were engaging with the various resources. And so they always had a textbook, but I never knew how they engaged with it. And now I can see so they the textbooks online, they can annotate and I can see kind of how much they really like it or get out of it, more than I ever could before. And also which parts of it speak to them and which parts seem kind of useless to all of that, and kind of skipping to make this story short. I wasn't so sure the textbook was as valuable as I thought it was, and as valuable as a date. They would die if they didn't have a textbook but the truth is they wouldn't. They just feel like you've got to have a textbook. But for me that was a eye opener. I'm like, I'm not sure we really need a textbook in this class. In fact, I'm not sure it's not making things harder for them to learn what I'm really getting at. They're trying to just memorize a bunch of details instead. Anyway, so that was a big change for me and I really deemphasize the textbook now. It's still there because they would die if I didn't give them one. But there's no like graded assignments based on the textbook and I say right in the syllabus. These are the resources we have for mastering learning objectives the same set of resources every week textbook is one of them but notice that it's not at the top. If you like it by all means use it, but it's free anyway, but you don't have to use it to do well in this class. So that was a big change for me to see how to see how a textbook works for students. Yeah, that does sound like a big change and I know you already mentioned how you saw a change in student engagement in class because of the annotations they were doing outside of class as well. So, I'm wondering because we both you both of you mentioned. Rating briefly, and this is something that I get a lot of questions about when I talk to a faculty. Because there's so many different ways you can use hypothesis to also so many different ways you could grade annotations. Do you mind sharing Kristen how you decide or choose to grade the annotations in your course. So, so far, I've always attached some points to them. It just depends what it is like even for the syllabus I wanted them to chime in with their ideas so I made it to points, something small, but they'll do it. They like points. And because it's integrated with the LMS pretty easy the only, actually, I, maybe you've already fixed this because I know it was on the list. The default points were always the same, like I could make it whatever points I wanted but on hypothesis it was always 10 or 20 something like that. So that was a little annoying but not even that wasn't even that annoying. I just would grade. Did they do what I said in the rubric they should. Okay, so it doesn't matter if the comments were long or short or whatever did they do the 10 I asked, and it wasn't just. I know that's really cool, or thank you, right, or good job. And because I needed time to get through all of that grading. So, if I'm trying to say more to them. I've realized I have to assign less so if I want to make it sort of based on articles that we're going to go into deeply where I really want to have a conversation. And it's going to take a lot longer to to assess their work. And so maybe I do five of those a semester, and not all the annotations of learning objectives and stuff I'm still kind of finding that balance for grading. But, you know, from the technical side, it's really easy hypothesis has made it really easy. It's well integrated to the LMS. Yeah, that makes sense. And I'll have to follow up with your school to see if they are on the correct version to get that change where the point value should be whatever you make it. Now, yeah. I'm not teaching this semester. So I'm not quite sure. Yeah. What about you, Francie, how do you tend to assess the students with their annotations for the, because they're English classes I do need to see that they're writing. So they have word count minimums that they have to do. And then there's a specific number of posts that they have to do posts and then responses to classmates. But they're, they're very quick to grade. Again, I get more involved in the readings and drama class because that's more of a discussion. So I am and they're responding to them. But in the freshman comp class where they're reading through an article they're going to later use for an essay. I can very quickly glance at those make sure they're on topic. Have they, you know, does this meet the word count requirement. And was it generated by AI. So I every once in a while I'll pull them and check them. And yeah, no, dude, I don't know what you're doing. So I do check those. But you still use that part usually grades pretty quickly. It's when I'm involved in the discussion takes a little bit longer but it's actually the fun part so I enjoyed that. I've heard that from a lot of folks that they end up taking a while to grade annotations because they like reading the annotations I find that with my own class too. I'm like, wow, they're really contributing some interesting thoughts outside resources. And I also find them enjoyable to read. So I definitely relate to that. Okay, so I know this session was titled around increasing grades and retention and student success and we've been talking about all these other details so far. But I think all that stuff is important and kind of lying the foundation for how do we get those better grades and retention and success in our courses. Now that we know a little bit more about how you've used hypothesis, how you encourage students to entity and how you assess the annotations. I think we can start looking at the results of all of your experimentation. So Kristen, do you mind getting us started. What results have you seen in your courses with regards to student grades and retention after using hypothesis. Sure, thanks. I was really surprised, I have to say that it. It's true that I did other interventions at the same time, like the equity stuff that I was talking about. And I did my exams differently, but this was the biggest change from what I had been doing previously. And I didn't expect it to have the impact that I had. So the rates of withdrawal were the same. That didn't change as far as retention over those semesters before and after hypothesis. But the students who remained the whole distribution shift in the grades. And suddenly I had like 10 people getting a plus out of, out of, you know, 24. And it was, and I was like, am I just being too easy on them? No, I don't think so. They just really engaging more, which is the whole point. That's what I wanted because I knew if they did engage with the material and interact over it, that they would understand it better in the way that I intended. So what what they were really good at students have always been good at is, here's a bunch of words or concepts, and I will tell you the meanings of them all the definitions. That was not a problem for them, or I will look up anything you want. But it was very different to try to piece the concepts together into some kind of cohesive whole, and also difficult to apply them to a little word problem. For example, and this is what they seem to be getting better at and I thought they would if they were, if they were, if they were giving and receiving more words of their own, then it would be able to do this on a test better than they had before, because I wasn't ever asking them to just spit out definitions like they wished I would. So I had that little, you know, Kara wrote up that little case study and she made a graph of the data when I sent it to her to show how many more A's we were having than we had before. So the students again it didn't change retention really didn't seem to, but it had a huge effect on students final grade in the course, I think it was at least, you know, significantly due to this new tool. Yeah, I think that makes a lot of sense. I mean, if for my educational geeks out there blooms taxonomy at the bottom we have our basics like memorizing things like that and hypothesis I think is letting our students kind of climb up into those higher higher order skills like our application skills and our evaluation skills and that's really what's going to allow them to do better on those summative assessments. So thanks Kristen it sounds like you had a pretty good results in your class. What about you Francie with in terms of student grades and retention and what kind of results did you see. So the, the most dramatic change was the first year so the, like when the first year I was using hypothesis in the drama class. The previous semester the pre, well, I teach it once a year so the previous year. The retention had been not good. The grades were okay but the class just dwindled to a very small group. And the next year, it was still a small class to begin with, but I had 100% retention in that class which I've never had, and then almost all of them had A's or B's. So that was kind of shocking. The retention has kind of gone up and down but it's honestly for us right now it's a little hard to judge because we're having a horrible problem with big students. And so it's a little hard right now to determine who is a real person and who is not a real person. So the retention numbers are kind of sketchy. So the people I have right now, they are, they seem to all be real people, and they are doing well. So I'm optimistic. It's just, it, our numbers are so strange right now because of this bot student problem we're having. That is an interesting problem I was not expecting to encounter when asking that question. No, I don't think any of us are and it's a very strange. Yeah, it's very strange right now. So I just want to point out if for those who haven't checked the chat. They did post the case study from Kristen's course as well as from Francis course regarding the grades and retention and the results that we that they saw. So if you want to check out more details about those results, you can find the case studies linked in the chat. I did also want to just quickly mention someone asked. What tool do you use to check for AI or does hypothesis offer such a tool. Right now there is not a tool in hypothesis that can check for AI. Most of what I've read is that AI detectors generally don't work. I will say from my experience teaching that if I think that I've come across something that's generated by AI it's because the student has simply summarized something that is happening in the text. Which I explicitly ask them not to do so they just don't get credit because they're not following the instructions of the annotation assignment because I'm asking them to do some kind of deeper analysis. I don't know if Francie or Kristen if you have come across any similar situations. I was just answering it in the Q&A and saying that our student population here at least they their speeches very is very for sure. But none of them sound like the way I sounds when it comments on general biology, which is still. I'm not saying that I'm a great detector, definitely not. But I wrote in the Q&A that I think you take longer for them to think up the right prompt for the sort of annotation I'm asking them to do. Then it would just write their informal thoughts, which is what I want. Like how did you respond to this or can you find objective three in this chapter or whatever it is. That it would take longer to figure out how to ask AI to do that task that would be just try. And it's kind of low stakes the way I have set up for grading you're just really rewarded for trying and for sharing. So the C is okay. What about you Francie and then also someone asked you to expand on what the bot student problem means. Oh, so, okay, well, there's there's two parts to that. So the same kind of thing with you can tell the difference between the student writing. And I've been very purposeful. I did the hypothesis class on AI last summer. And so I've been very intentional about doing things in the prompts that you could only get from the class. So you can't just go out and ask an AI tool to give you this information because it's, it's not going to have it. So, like Kristen said, the responses don't match what you're asking them for. And then the writing style is very different. So it's not hard to spot. And then I'll just run it through a, like if I suspect it, which is pretty high, you know, it's like, okay, it's not looking right. I'll run it through a detector. And if it comes up, then I'm, let's talk about this. I also have discovered that if they run it through something like Grammarly, it can come up as AI generated. So I usually talk to them. And then we talk about, you don't just turn over your papers to the computer and let your computer take ownership. So that's what I've been doing to try to deal with that. It's not perfect, but trying to talk to them about what's appropriate use and what's not appropriate uses seems to be the way I'm going right now. With the bot student thing, I don't understand it all. I just know that for some reason, and it's not just our school, it's a bunch of schools are having trouble with these students that are not real people. Somehow they have information, but they're not real people. So, honestly, I don't have a lot of information about that. I just know, why do I have all these people who are not real? And I don't know. It's a California thing, I guess, I don't know. I love those bots in California. It's horrible, honestly. From automated cars to automated people. It's really been quite horrible because then you find out, oh, this person's not real. And now your retention is abysmal because they kicked out all the people who were not real people. So it's a very strange thing. Yeah, that certainly sounds strange. But I'm not high enough up on the food chain to know exactly what's happening. So to turn our attention back or attention back to actual students. Yes. We heard how what kind of results you've had in your classes. And it sounds like you both are pretty enthusiastic about using hypothesis, but how have your students responded to your students tend to like using hypothesis. I can't talk today. Do they tend to like using hypothesis? Do they have negative reactions to it? What, what feedback have you on, if any, do you mind starting with that Kristen? Sure. So I need to ask them that specifically, which I haven't done. But first of all, it's easy for them to use. I was amazed. I thought the first two weeks are going to be a disaster with technological questions. No, it's completely user friendly and they figured it out all by themselves. So that was good. But other than that, they haven't really commented, which is, I think that is a comment. It's so easy to use it's transparent, if that makes sense. So they, they don't have to even think about, Oh, I'm using a tool now. They're just doing their thing, participating in class. And I think that they, they'll do whatever I asked them to do. Like I want the great tool you set in front of me. But obviously it's a good one if they're able to do that with ease. Yeah, that's so true. What about you, Francie? Have you gotten any feedback from your students? Yeah, I do ask about it specifically when they do their end of the semester feedback. And overall, the response is very positive that they like that it's easy to use. They like the interaction they like being able to see what their classmates have to say about the reading. They like the resources that are there for them. So, overall, it's very positive. Oh, that's great. Yeah, I tend to think that sometimes students might not even rise that they're using something that's not technically part of the learning management system. And so you both have used hypothesis in lots of different ways. So I'm not sure how you might answer this last question because it sounds like you kind of run the gamut and on what you can use hypothesis for. But do you plan in future courses to make any changes on how you're using hypothesis or is there anything you want to try with hypothesis in the future? Francie, can you start with that one? I think no until you all come up with some different things. Like, one of the things I would love to be able to do. So I guess here's my wish list. One of the things I would love to be able to do is to get them to annotate my lecture videos, but all those are done in studio and unless I upload them onto YouTube, which I don't really want to do, they can't. So there's my wish list. I would love to be able to have them annotate those, not all of them, but some of those lecture videos out of studio. So there. So, but otherwise, at this moment, I feel like I've kind of maxed it out until I think of something else. So I'm not planning any big changes. So that might be on our roadmap, but I'm not going to say anything official here. Oh, look, I see it in the chat. Okay, maybe it's okay if I say it then. What about you, Kristen? Is there anything new that you want to try in the future with hypothesis? Yes. So now I'm having students, I've just sort of started playing around with this. I'm having students write the exam themselves. And we're trying to, you know, workshop it and discuss it ahead of time or after one or the other both. And I didn't use hypothesis to do it. It was a combination of discussion board and in class discussion, but I was like, hypothesis would be much better for this. So that's the next thing I want to do. And I have been loving, I don't know if they love writing their own exam, but I have been loving it. I'm like, these are good questions. They're different every semester. So that's my plan. Do they try to make the questions really easier, really hard? They, it depends on the student. Like some of them are like, this is the kind of question I would like to have. And I'm like, well, make it harder than that. And then some others of them are like, ooh, a competition for who can write the hardest questions. So we get a whole range. And I guess on the flip side, since you also have tried using hypothesis in lots of different ways, were there any lessons learned and how you implemented it? Like if you were to give advice to someone who was using hypothesis for the first time, what might you, I don't know if that's two different questions. So feel free to answer either angle there. What do you think, Kristen? Yeah, I already mentioned about often thinking about the group size for whatever task you're doing. And then the other thing would be, I think I did too much at the same time, like it was hypothesis overload for the students who are like, oh, more entertaining. So just to be really intentional about what tasks, because what goal do you have that you're really accomplishing by using this tool and not just, oh, here's another way that's easy to grade. Throw that into anyway. That's what I would recommend. Yeah, the group size is something that comes up a lot. And is also, I feel like a hotly debated topic, like people, I don't know, people will tell me so many different numbers that are the ideal number to annotate together. How many do you tend to have to annotate together in a group, maybe outside of your learning objective one, Kristen? It depends how many I think might drop that semester. So if I think they're going to start it out with a group of six, but I don't want it to be less than three. Two is not ideal. And four is better. So four or five is the, for me, that's the sweet spot. Okay, is that the same for the learning objective annotation assignment? Yeah, yeah. So if we're annotating a research article that's technical and difficult, fine to have the whole class. My whole class is usually around 20 for me, 15 to 20. Okay. But if they're going to do a lot of back and forth and adding of resources and all that kind of thing at the group of four to five is much better. What about you, Francie? How big or small do you tend to keep your groups? It depends. A lot of times on the length of whatever they're annotating. If it's something short, I'm going to keep the groups smaller so that everybody has something they can comment on. But for the drama class, they're all, it's no groups. They're just all in there discussing it together. And then the other thing I try to do is once I see how they're responding and when they respond, I try to put early posters together because I am asking them to respond to each other. So if I know, hey, you're an early poster, let me get in the group with the early posters so that you guys can get this done that I know you want to, you know, and then I'll put the other people in a different group. So sometimes I'm moving the groups around. They don't stay static throughout the semester. I will move them around. I just learned you have to not move them after they've commented. Wait till there's a gap and then move them. Yes. That's a really great pro tip about putting the early annotators together and the later annotators together. Definitely take notes. Anyone watching out there because I haven't heard that one before. And I think it's a really good suggestion. So, before we kind of roll into our final slides and announcements of the day. I want to give attendees a chance to remember please you can put questions in the Q and A and then Francine Kristen was there anything I didn't ask that you just wanted to say about hypothesis that you were hoping to share the answer. Can we know I just wanted to kind of put it out there. If you're on the fence and trying to decide if it's something you want to try, definitely give it a try. It's worth, it's not a steep learning curve. It's easy to learn how to use. It's easy for the students to learn how to use and it's easy to integrate into your class. So give it a try. For sure. And I was going to just going to repeat what was said earlier about how it's fun to read all their comments and it's hard not to want to answer a whole bunch of things like you could spend hours. I totally agree. Obviously. Yes. And some people are asking about the recording. The recording will be sent out. And just to wrap up our last few minutes, I just want to share some resources that are coming up and where you can find more information. Both Kristin and Francie talked about Hypothesis Academy. If you're a partner school with Hypothesis, you have access to taking this course. Our social annotation in the age of AI course. That course is launching today. So if you register today, you can still join. And our next cohort of social annotation 101 is launching in April. If you want to check that out, you'll get lots of assignment ideas in addition to learning how to use Hypothesis. And lots of feedback from me in case you just want to hear me more. For those of you who are not partners with Hypothesis, you can check out our kickstart promotion. And you'll get a few things as part of this promotion included a discounted pricing and access to that Hypothesis Academy. As well as some other goodies. So please reach out to the email on the screen if you're interested in learning more about getting access to Hypothesis at your school. We also have a big event coming up next month. We have a conference, a virtual conference that is free, no cost, on April 17th and 18th, 12pm to 3pm Eastern time both days. So we're going to have lots of folks sharing much like people are today about how they use social annotation in their courses. I know people are asking about social annotation and specific subjects in the Q&A. We're definitely going to have lots of different disciplines represented there. If you want to see how folks are using social annotation and different levels of education in different subjects, lots of different ideas to be learned on April 18th, 17th and 18th of April, 12 to 3pm Eastern. So I believe I close my chat window, but you can find more information and register at the link in the chat. So that's everything we have for the day today. I'm so glad that you all were able to join. Thank you especially to our panelists Kristin and Francie for sharing their experiences and the results. It's so good to see both of you again. And if there's anything we can do to help, please don't hesitate to reach out to us at Education at Hypothesis. So I hope everyone has a great rest of your week and thanks again for joining today everyone.