 Hello and welcome everyone. Just going to wait another minute so folks can join. Hope everyone is having a good Thursday so far today, no matter where you're joining from. Just a heads up also that I will be asking a couple poll questions today so please be ready to answer my questions. Just because. In working with folks, especially we'll see people define things like large courses very differently, so that'll be interesting to see as well. So it looks like it's starting to slow the attendees coming in trickling a little bit slower now so we can get started. And if folks join, you know, we'll get them caught up. So thanks everyone for joining today our session on using hypothesis social annotation in large courses. We know that large courses and teaching large enrollment courses often present different challenges than if we're teaching a smaller course. So our slides today I will pop into the chat a little bit later on the link to those just so that you can access them. And you can make use of the resources that will be going over. So the agenda for today is I just want to start out by going over what social annotation is and why it might be useful in your course. So if you are a tried and true hypothesis user, sorry if this is a repeat for you, but we will have some new folks joining us today and I want to make sure we're on the same page. Go briefly over how hypothesis works in your learning management system. And then I'll focus on kind of the meat and potatoes of the session today, which is strategies for structuring hypothesis annotation assignments in your large courses. And also strategies to consider for grading because I know grading in particular can be a challenge for folks in trying to manage those large enrollment courses. So my name is Christina careless. I'm a customer success manager here hypothesis, and I am an instructional designer by trade. So I guess my background has always been in education actually started at the high school history teacher before moving into instructional design and supporting faculty in their teaching which I did for about eight years. Before I moved here to hypothesis, I adjunct still and I use hypothesis in my own courses. So my students have an annotation assignment due tonight. And so some of my experiences I bring along with not just working faculty at different schools, but from my own experiences in using the tools so really thankful that you have all joined today and happy to be supporting you in exploring hypothesis. The first thing I want to do is just look at what it looks like to entity with hypothesis for those of you that are new. I do want to launch this poll to see what everyone's experience with hypothesis is so I can kind of get a sense of who's out in the audience. So if you could just take the poll really quickly, then that would really help me and better knowing where everyone's coming from. And yes, if you have questions throughout the session, please feel free to add them into the Q&A section and I will be checking that occasionally to see if there are things I can address there. If you have a more personal technical question I would ask for you to write into our support team instead and any kind of broader questions that we can answer here and be happy to address. Looks like most folks have answered the poll, and we are pretty split on the newer side so it looks like about 60% of you are brand new to hypothesis and 40% of you have used it before. So thanks for taking that poll for me that's really helpful. Since a lot of you have never used it before. I do want to go into what it looks like to annotate with hypothesis. And I will first pop into my course here. I'm looking at Canvas but really this will look very similar no matter what learning management system you're in. So I'm going to open a document from this Canvas course, but once I'm in hypothesis this experience is very similar, no matter if you're using Canvas or Blackboard or Moodle, Brightspace, whatever the case is there. So in this instance I have loaded a chapter from a free physics textbook. It's an open educational textbook that's just available for students freely online. Loaded that into my course. You don't have to technically use a textbook chapter. I'll go into the documents that you could use with hypothesis. This is just the example I have here. So hypothesis is essentially taking this document that I've loaded into my learning management system and has added this sidebar on the right hand side of my text. So the whole functionality of hypothesis is within this sidebar and essentially this sidebar is providing a space for me as the instructor and my students to have a conversation about the text as we're reading. So you'll notice as I'm kind of scrolling through these annotations that the highlighted portions of the text on the left hand side are actually changing color. So the highlighted portions of the text on the left are areas of text that me or my students have selected to add a question to or a comment to or have just a general conversation about. So that indicates that that area of text has been annotated and I can connect those annotations or those areas of text to the annotations by hovering over those annotations and the quote the highlighted text is quoted within the annotation as well. So each of these annotations are anchored very specifically to a word or phrase or sentence in the text. And because of that me and the students can have a conversation about a very specific piece of text. In this example, you can see this equation that Jennifer has annotated. She has kind of written out the syntax of the equation in her annotation. And if I click on this option to show replies which some of the annotations have Malika has come in and asked a question about that particular equation. In this example, the students are having a conversation about just that equation in the text. And if I were a physics professor which I am not, I could answer the question or I could see if maybe another student could answer the question, or maybe I would even bring in that conversation into my face to face lecture class and address the question there so there's a lot of different options and how I could handle that. But again, we're having this conversation around just this one equation in the text. And then if I scroll even further, I can see another example of Jennifer and Malika having a conversation about the law of conservation of momentum. So here Jennifer is kind of writing about the law of conservation of momentum. And Malika comes in and says hey this video can help you better understand that. So these same two students are having a conversation about two different pieces of the text, kind of simultaneously not exactly the same time but they're reading the text and having multiple threaded discussions about this text. And if I wanted to add an annotation as I read, I could select the text I'd like to annotate. And that annotate button would pop up right there, and I could add an annotation, and then continue on with reading. And this is why I think hypothesis is pretty distinct from a more traditional discussion board, because students and me as the instructor can add annotations right alongside the text as we read the text. So instead of reading being this isolated singular activity where then they move on and complete a discussion or a problem set or a reflection later on. So everything is kind of happening together, they're reading and reflecting at the same time. And that is happening collaboratively. So here on the right you can see me and the students annotating the default setting and hypothesis is that everybody in the class can see each other's annotations and respond to each other, which in a large course in particular could present some challenges so we'll talk about that, but it makes reading a more collaborative and less isolated process. So that is the basics of what hypothesis is adding as far as functionality goes in the learning management system. And why could that be helpful for your course in particular. Here hypothesis we like to say it makes reading active visible and social. It makes reading more active for our students because we're asking them to engage in metacognition as they read. So a lot of times students are either not doing the reading. They could be skimming through the reading so they're not really retaining or processing what they read. And metacognition can really help with retention and comprehension of the text, asking them to annotate is asking them to reflect as they read and think about. What do I not understand about this? How is this connected maybe to what we've learned before in this class? How is this connected to what we have learned in other classes? So asking them to annotate is asking them to participate in metacognition so that they can add those thoughts to the text. It makes reading visible for us as instructors because it gives us the information that we can then use to drive our teaching decisions. So, like I showed with the example of Jennifer and Malika annotating this equation here. You know, that's information for me to then take to class and better know that this maybe needs something is something that we need to spend more time on. And then it makes reading social for students and this can really help with student sense of belonging in courses and especially in STEM courses. Students say that hypothesis helps them better understand the text that they're reading because they can see their peers interpretations of the text. And then they can also see if other people have the same questions that they do or the same interpretations that they do of the text and that can help boost student confidence and self advocacy as they complete their readings. So how exactly can hypothesis help in large courses. I want to put another poll out there and kind of see maybe what we're all thinking about what our goals might be in using this in large courses. So going to put another poll if you could answer that and you can choose more than one response for this poll, which it looks like some of you are doing. All right, it looks like most of you have answered. Let's share out those results and check them out. So it looks like most of you have said that you want to engage more of the students in the class. So if you're teaching a large class you want to get more students involved in engaging with the course materials. And then creating more active learning opportunities in a large course. So we do have some strategies for that as well. In addition to just getting a better sense of what student understanding is in the text, creating community and holding students accountable for reading, which I definitely can understand. So hopefully some of the strategies here that we'll talk about today will address most of these concerns. So thanks for answering that poll for me. So the first thing is just kind of keeping students engaged in our course readings can be challenging throughout the term, especially in large classes when we aren't necessarily like talking to each individual students quite as each individual student quite as much. A case study that we did with UT Austin showed that using hypothesis in our courses can hold long student engagement with the text throughout the term. So let me explain that a little bit. I know we're kind of looking at a lot on this slide. This was comparing two physics courses that's using hypothesis and these are larger courses with hundreds of students in them. One course was not using hypothesis and one course was so there are two physics courses with the same textbook, one using hypothesis and one not using hypothesis. The course that was not using hypothesis is represented on the graph in red. So you can see on the graph at the very left hand side of that. You can see that there's a lot of engagement with the course text at the beginning of the semester so that X axis of the graph is representing the student time throughout the semester engaging with the course text. So at the beginning of the semester they are engaging with the text frequently and then by the end of the semester you can see they basically don't open the textbook anymore. Whereas the students who are using hypothesis represented in the blue here they are engaging with the text throughout the course. So introducing hypothesis to the course reading enables students to engage or encourages students rather to engage with the course reading as the semester progresses instead of maybe dropping off as other focuses come into play. So and that resulted in a median of 36 days active with the course text the students that were using hypothesis versus a median of 70s active active with the course text when they were not using hypothesis. So one of the goals you all mentioned was just getting students more more students engaged in the in the course text. This is one way to definitely increase their engagement with the course materials. And then how does hypothesis work in your learning management system in particular. I just want to go over some basics before we start looking at how you could use it with your large courses. It makes your readings easily annotatable in most major learning management systems so we work mostly with schools that are using canvas blackboard d 12 bright space Moodle and then we have a few ones like high in school G as well. And we're pretty deeply integrated into all of those learning management systems so the student experience is going to be pretty similar no matter what learning management system you're using. In canvas here you'll see all I have to do is open up a reading. And then I can start annotating in hypothesis and that's true of all of my learning management systems that we work with the students don't have to log into anything or go to a third party website to access their hypothesis assignments. They're just accessing the readings through the LMS and they can start annotating. Because of that the technical barrier for entry for students is pretty low and that's nice to especially in large courses where it might not be feasible to support, you know, hundreds of students in learning a new technology. This does not require students to learn a lot technology wise in order to start annotating. Another part of I don't think I have that here yet. So another part of our integration with the learning management systems is that you can also grade your annotations and set up your annotation assignments right within your learning management system. So you don't have to set up the assignments on a third party website. You don't have to grade on a third party website. Everything is done right in your system and sent to your grade book if you do want to grade those annotations. On the slide deck, which I will pop the link to the slides in the chat. There is a getting started page for each of our learning management systems and that'll walk you through how you can start using hypothesis in your learning management system if you don't know how to set up an assignment. So a couple of other details about using hypothesis social annotation before we dive into large courses and want to make sure everyone's on the same page here. What exactly can you annotate in hypothesis? I showed you all an example of this physics open educational textbook. This is a PDF. Most folks are using PDFs with hypothesis. Hypothesis tends to work best with documents that are freely available to students on the web, although there are some exceptions to that. And most of the time that are not pay walled or have some kind of login. You can also annotate web pages and online articles. So you can see here in this example with the data science text. This is just a website that's being annotated. So you can use URLs. You can use open textbooks and open educational resources, whether that's like a press blog, open stacks, if they can be loaded into your learning management system, you can annotate them. We also recently released the ability to annotate YouTube video transcripts. So in addition to annotating traditional documents, students can watch the video and annotate the YouTube video transcripts at the same time. And then the one exception that there is to annotating things with a login are integrations with JSTOR and vital source. So JSTOR, if you have students reading a JSTOR article, then they can annotate the JSTOR article without having to re-log into your library. And your library will still get those access stats. And then vital source independent schools can have students annotate paid e-text in those particular cases. So that is the one exception for when students can annotate something that is behind a login or a pay wall. And I see there are a couple of questions related to this. So I'm going to pop over to the Q&A and just look at that really quickly. So Meg asked in the question, one limitation I face is it is only able to add a single URL link as the destination. I often want to select more than one chapter of a book or multiple websites for the whole assignment, but that doesn't seem possible. Is there any solution to that? So yeah, that can be a challenge. And that is difficult because some people do really want to break up each chapter into separate hypothesis assignments. In the case where there are multiple links, you're right in that you can only share one link at a time within a hypothesis-enabled reading. If you do want to combine multiple texts or multiple links, what I usually recommend to folks is to save those websites as a PDF and then combine the PDFs into one large PDF and use that as your hypothesis-enabled reading. So it does take a little bit more setup time, but if you just want all of that in one space, that's probably the best way to do that. And then I noticed Katie also mentioned she loves being able to explain concepts that some but not all students might have trouble with and wouldn't want to spend class time on this. Excuse me, sorry about that. But an annotation could be skipped. Yes, this is a great way to use hypothesis that we can dig into a little bit more. So you can provide, especially in those big classes, more guidance for students as they read. Like almost act as like an individual tutor for them as they're reading without actually having to be there through the annotations. So thanks for that comment, Katie. And it's good to see you here, Katie. And then on the other side of things, we talked about what kind of text you can annotate. What can you put in an annotation itself? So text is usually what folks are really using with annotation. But there are lots of other opportunities to have students engage with readings using images and video. You can embed images and video in annotation. They can link to external websites and add tags and emoji. And they can also use latex to add equations to annotations. So there's a lot of opportunity for students to demonstrate their understanding using multimedia in the annotations. And for you as the instructor to pull in multiple resources. Again, when you're managing a lot of students, sometimes bringing those resources in and trying to organize them in your course can be a challenge. But when we saw the solution here where Malika is sharing this video about the law of conservation and momentum. It's a great space to just bring in video and images to help students better understand what's happening in the text without having to leave the page. And we have plenty of faculty that go in and add their own annotations before the students are reading to add resources like this and help further that comprehension. So I want to now move into looking at social annotation in large courses. I know you're probably like, okay, Christina, you've been talking forever and we haven't talked anything about big courses yet. So this is really going to be my focus here. But before I get into this, I want to ask what is a large course to you because everyone I talk to is has a different definition for this. So I really want to see when you're talking about what is a large course? What does that mean to you personally? And we'll kind of see where everyone is at because I think some of these strategies are going to be more feasible for someone on the smaller end of the large courses than maybe other strategies. Like you might have to consider some strategies more than others depending on how you're defining what a large course is. All right, let's see. I'm going to share these results because I always think that's interesting. So we have, I know, and my number ranges, I know they're very arbitrary and like random. So I'm sorry if you thought of something in the middle of one of these. So it looks like about a third of you are saying just like anything over 30 students is large. Whereas I talked to someone recently who was like, yeah, when I have 800 students, that's a large course, which is to me unfathomable. And we did have 2% of people saying 500 more than 500 students is a large course. But so there's a big range of what we're considering large here with most folks falling into either over 30 students or between like 50 and 100 students as being a larger course. So keep that in mind that I'm trying to give some strategies that could reach either end of the spectrum and some strategies are better fit for the smaller end of the spectrum versus the larger end and vice versa. So thank you again for that. And I do see some questions coming and I will pause soon and make sure that we address those so we'll hang on tight for that. So I think in large courses that consistently using social annotation through the semester can help provide guidance in our course readings, create more community for our students and opportunities for active learning. And increase reading engagement throughout the term. So we saw earlier with that case study with UT Austin that students will be more engaged with the course text through the entire semester when they're using social annotation. And they'll have the opportunity to engage with all of their classmates or maybe a subset of their classmates within the annotations. And as Katie mentioned in the Q&A, it gives us an opportunity to provide guidance for the students in the course readings. So we can go in and add our own annotations to the text before students are accessing the text to either provide context, add resources, add questions, and maybe just some thoughts that we wouldn't necessarily, as Katie mentioned, want to spend class time on. So as I always say, if you came to last week's webinar, this is literally a slide that I showed last week. Because I always start with this, no matter what you're teaching and what your class size is, I always want to start with why are you personally wanting to use social annotation. Because your purpose and your goal for using social annotation is going to impact how I would advise you to structure assignments and to manage your grading. Because lots of different people have lots of different goals for using social annotation. So someone who wants to use social annotation to create more opportunities for active learning in their face to face class. I'm going to advise them to structure their assignments differently than someone who wants to use social annotation to make sure that students are keeping up with their reading and is not really bringing that into the class as much. So please keep this in mind and think about what is your own purpose as I'm going through these strategies. So the first thing I'm going to start with is talking about how we can structure our assignments to best manage a large volume of annotations. Because if we have a large class, that means lots of students are annotating a document. And I often get, like, comments from folks that documents can get very crowded with highlights if you have lots of students annotating the same document. So how can we structure our assignments to manage this? I first want to start with an example of this professor who Melanie Walsh and how she handles annotation in her large classes. I'm actually going to stop my screen share and reshare because I don't know if I did this properly for video. So I just want to make sure I do this in the right way so that you can hear the video in this. The big class that I'm using hypothesis in this quarter, it's called foundational skills for data science. There are 200 students who are in it. And in this class, I have each week, one reading that the students are required to make two annotations on. Similarly, they're required to make a substantive annotation and this can be a comment or a question or a reply to another student's comment, which is something that I think the students really enjoy. So I'm really glad to be able to include replies to actually get that kind of dialogue going. That's one of the big motivations for using hypothesis for me. In the past when I had done canvas discussion boards or, you know, like use WordPress to have students try to have a discussion in a blog forum. Students would just like repeat the same points over and over again after a really complex dense or interesting reading and it was really hard to like encourage them to engage with specific passages and that's able to happen much more organically I feel like and hypothesis. So I think we can really just zoom in on particular passages that you're interested in. So, so yeah to substantive annotations and I give them like a whole kind of buffet of suggestions about what that might mean like you can make a connection to one of our lectures you can make a connection to another reading you can bring in your personal experience as long as it's done and such and such way so I try to provide some guidelines for for what's required. I suppose the way that I use it Oh so then for the large lecture courses, what has been essential for me is similarly. We have eight different sections that the students are in they have lab sessions every week so I organize them into those eight sections that have about 25 students. So the students are only annotating within their section and they can only see this the annotations from those other students. I'm kind of curious about what it would even be like to have 200 students annotating one reading. I think that I've just assumed that it would be far too chaotic but I think it could be kind of an interesting exercise in some regards but Well, I don't mean if someone else is doing that I'm very curious to hear if you're doing that. But yeah, just 25 students, a piece for me and the main way I've been using it is just to. I basically review it as I'm preparing for the lecture and I can see like what are the big questions that students have what are the big areas of confusion what are students excited about. That really helped me like have them lead the discussion or like know what's going to be kind of a vibrant discussion in smaller classes I'm also able to like note down. Okay, a student made a comment about this or I'll remember that a student made a comment about this and I can like, you know, reference them by name, if the discussion is kind of flagging and be like, you know I remember that you, you know had something to say about this and I can elaborate on this point that you made in a larger lecture course I think that's a little bit more challenging but I can still kind of sometimes remember specific things that I know students were talking about and I can say, you know, a lot of you were saying, and I like doing that, just being able to say like I saw a lot of questions about like this, and it is like one way of like acknowledging like I'm, I'm invested in your conversation to. Okay, so that was just one example of how she structures the big class that I'm sorry about those playing again her own annotation assignments and her large classes. I also so she has sections of 25 students. I teach a course that has about 25 students I find that to be a fine number of students to be annotating together. And I've talked to a lot of faculty who think 25 students annotating together is a little bit intense and prefer to break students into groups of maybe eight or nine. So right now, the main features that support breaking students up into groups and managing large volume of annotations are our group integration and our tag feature. We have groups integrations currently with canvas detail bright space and blackboard and what these integrations allow us to do is break our students up into smaller groups and have a separate annotation space for each of those small groups. So I'm going to hop into a group assignment just to show you what that looks like. So if I open up this particular group assignment and load this into my browser here. What hypothesis will do when you create your reading as a group assignment and in a canvas bright space and blackboard you would set up the groups in your learning management system and you could use the groups that are in your learning management system for this. And what that looks like is it creates a separate annotation space for you to see each of the groups annotating in. So here I have three different annotation groups that are available as a drop down from my annotation sidebar, and I can toggle between these groups. And I think they all have the same annotations right now so that's very, it's you can't. Now you can see this group has different annotations from this group and I can hop between the groups as the instructor and see what students are discussing. On the student end, they would only see the annotations that are in their own group so they would be restricted to seeing just what their group members are saying, and then that would, you know, cut down on the number of highlights that they're seeing within the group. So that is how the group's integration works in those three learning management systems. Another tip that I want to point out here if you do have a document that is very heavily highlighted is this little eyeball icon on the left hand side of the annotation sidebar. I like to point that out to students to tell them that they can turn off the highlights if they want to read the document kind of cleanly the first time through. And then the tag feature is another feature you can use to help you manage a large volume of annotations. So each annotation when you go to annotate has this ability to add tags. So if I click annotate here, I have a space to add tags and tags show up like as little labels on the annotation. So you see this one is tagged as resource and multimedia. And then as the instructor, I can go to search and search tag and search that tag word, and it will show me any annotation with that tag in it. So some folks like to use tags to manage questions and to search for questions before class. And then other folks like to use tags if you want to have students broken up into groups using tags instead of the group functionality. You can have a student tag like group one, group two, different topics, things like that. So one strategy for structuring assignments in your large classes could be breaking your students into groups either using the group integration or using tags. And then assign one student per assignment as a group leader to summarize the contributions made in the annotations each week. So each week, maybe you're having one student from each group submit some kind of summary instead of having to review all of the annotations or make that group later responsible for bringing questions and relevant topics into class. So rotating that group role, the group leader role rather can engage a greater multitude of students perhaps in a more manageable way and also takes the load off of you for reviewing those annotations. So that is one strategy you could use in a larger class. Another strategy to use is to provide some flexibility in annotation assignments to reduce the overall load of annotations that you are doing. So I have seen this done in a couple of different ways, and I actually don't have one of them as a bullet here. So one of the folks, one of the professors that we've worked with would schedule break weeks in where students don't add any and don't have any documents to annotate so they could catch up on grading. You could always provide students the option to only annotate, you know, if they're annotating 10 documents through the semester, maybe they have to pick seven out of the 10 to annotate. And then that can reduce the overall load and give students some flexibility as far as their participation goes as well. And then a third option for this that actually don't have here is just assigning a subset of annotation students each week. So each week, maybe a different group is responsible for annotating the document and the other students and maybe they have to just observe or bring relevant questions to class and things like that, kind of like a fishbowl discussion, annotation style. And then as far as another strategy, you could incorporate student annotations into active learning groups in your class. So this is something that a professor I work with actually does. She has classes of over 200 students, the students annotate before class, and then in class they get into their annotation groups. They review the annotations together, and then they complete a document together in class and submit that in class. So each group of like seven or eight students is submitting something as part of their active learning within the class itself. So if one of your goals was to create a more active learning activity, you could have annotation as a pre assignment where students are then reviewing those together and working together in class to create some kind of follow up assignment for that. So before I move on to developing a grading strategy, I do see some questions coming in so I'm going to pause and address those. Currently, we don't have a group integration for Moodle. I will definitely put a plug in for work on that. But right now documents would have to be loaded in separately from Moodle and restricted by group. In addition, you could use the tagging feature as a way to have students like tag what group they're in as well. So we do recognize that as something that is desired for the Moodle platform. And then just reviewing these questions. And someone said, if you select a group assignment, the reading will automatically appear separately for each group that's established in the LMS. Yes, kind of. So the first students for the group assignment. And again, I'm just looking at Canvas, but it's similar for Blackboard and Brightspace as well. So the students will just see one assignment so it doesn't create multiple readings. All it does is create the separate annotation spaces over the same reading. So students all look like they're going into the same reading and then the different groups get separate annotation spaces. So hopefully that clarifies that question. Let me know if it does not. And Katie, if you want to pop in any various online imaging hosting services, because I know Tiffany was asking about students embedding images, that would be helpful too. Thank you. All right, so I know another part of managing large courses is just managing grading for lots of students. It's time consuming. I mean, we already have a lot of demands on us. I know it can be a lot to think about grading annotations on top of handling, you know, all the other responsibilities that are on your plate. So I'm going to start out with just a clip from another professor who has a large course in how he handles grading. And so I then considering like the massive amount that would be required to grade something like that, a massive amount of time, I would, I set them into like five week blocks and that's sort of like just how I break up the semester. So then in a distance learning course it can get kind of tedious, especially in the spring where people start getting like sort of exhausted around week 10. So this is up where we sort of like a renewed sort of section that we're doing each five weeks. So four weeks they have readings all the readings are socially annotated and then the fifth week they can catch up on any of those readings so if they missed some of them they can have like a week where they can go back and like fill in the gaps and that's also the week where they can do the grading. And to grade that I will take from each week just a sample like a random text. Usually it's the text that I think is the most difficult, you know that maybe like that they need or the most important one. And then I'll see like how they've, how they've annotated on that and the requirements for me are that they have at least three sentences, questions, they can be questions or responses but they should be three sentences. So there should be a substantial content, not just like, you know, like, hey, this was great. What color was the hair of that character, you know, things that like really get, get more to it. And so that reduces the amount of like time I spend in, in actually reviewing what they're posting and so I get sort of a sense for their grade and then I put that in for the, and I do that so every five weeks. And I do like sort of like a little break in, in the other other areas of where my attention would be, but I think that's sort of like is a very general sort of outline of how I, how I use it. And it's been encouraging to see like their engagement. And then when people are kind of off, I will like just say, hey, if you go back and redo this, they get the practice of it and then they know like what they what's expected going forward. Alright, so that is how Kevin Richards handles grading and I think his course is over 100 students. So that idea of the break week and kind of just selecting specific annotation assignments to review could be a helpful way to manage that grading. So some other strategies that you might consider is grading only a subset of students for each annotation assignment. So going in and maybe if you have assigned discussion group leaders just assigning a grade for the group leader for that week or selecting groups at students at random to grade and kind of rotate through each week. Another strategy is to kind of just make a very low stakes assignment. So giving the students are really baseline of what they could receive for a complete incomplete grade. This is honestly how I grade my own students, even though my course isn't super large. I have 25 students. I give them a pass fail or complete incomplete for their annotations because I wanted to be a space where they can try on ideas for the first time, you know, they don't necessarily have to be right about things. Really just want them engaging in the texts. So that also takes some of the grading seriousness of the grading off of me because I'm really reviewing and just looking for, you know, what students might need more guidance on things like that over like how correct are they or the level of analysis that they're doing just doing the complete incomplete option. And then another strategy that you could use again depending on what your goals for annotation are and how you want to incorporate grading into annotation is to use the hypothesis notebook to review all the students contributions either just at the end of the term or maybe a couple times through the term, maybe at midterm and at the very end. And the hypothesis notebook is something that not a lot of people are aware of. It's kind of like a hidden little feature, but it can be really useful to see how a student has engaged throughout the term. So if I go back into my physics example here at the top right corner you see like a little avatar in the annotation sidebar, and then there's this option to open notebook. And if I click on that it will bring me to all of the annotations in this course or group and reverse chronological order. So the newest annotations are at the top. This is showing me everyone's annotations. But the other nice thing about the notebook is I can just select a single student to look at. So if I wanted to grade Malika's annotations and see how she was engaging throughout the semester and how often and frequently she engaged, I could filter and see this through the notebook. So I could just choose to open the notebook at the midterm kind of skim through the annotations and give her overall grade at that point and then do that at the final part of the term as well. So that is another strategy that you can use. So before I get into a kind of our final pieces for today, I do want to go back into our Q&A and also see how folks are feeling. I hope some of these strategies that give you some ideas for how you might start to dig into hypothesis in your large courses. But I want to see, like I said, how everyone is feeling at this point and what you might need further assistance with. Okay, there are some good questions in the Q&A. So I'm going to go through these. So Meg asked, are there any auto grading options in Canvas for a certain amount of annotations or a certain number of words per annotation? There aren't any auto grading options at this time. That's why I often encourage doing a complete and complete or something like that so that the grading can move fairly quickly that way. Carl asked, is there an option and hypothesis to evaluate annotations to determine if there was a large language model contribution? There are not. I would say I have seen a lot of content out there saying that large language model detectors are oftentimes incorrect in identifying something that's been crafted by AI. But in my experience it tends to be a little obvious if a student has used AI to annotate because the AI tends to summarize what the article is saying. So if a student is just kind of regurgitating what a particular paragraph has said, I would kind of flag that for AI and I would also just tell your students you don't want them to be summarizing in the annotation. So that's kind of where I will kind of flag my students. Not necessarily that I think they're using AI, but they are just not following the instructions because I don't want them to summarize the text. And then another question about grading. Do you have an example of grading is hypothesis connected to the grade book in Canvas. Hypothesis is connected to the grade book in all of the learning management systems we work with to just show you very quickly in Canvas. It is connected to SpeedGrader. So if I go into SpeedGrader in Canvas, it will filter the annotations for each of the students here. And I can see just one student's annotations at a time enter my grade. You could also use the rubric in Canvas to kind of quickly grade a student as well. And then I'll hop into another learning management system just so that you can see what it looks like to grade in Blackboard, Brightspace, and Moodle because all of them have the same grading functionality. So if I'm just hopping into Blackboard here, but like I said, Blackboard, Brightspace, and Moodle all have the same grading bar at the top of the screen here. So when you log into your assignment, you'll see a grading bar at the top. Any students that have annotated, you can find what they have contributed and it will filter and just show those annotations. And then you can enter a grade and we actually have a new feature where you can enter feedback here as well for the student and move on to the next student. So this one doesn't have any, but you can see each individual students, both original annotations and replies as you grade, enter the grades and all of those grades will get sent back to your gradebook, whether you're using Canvas, Blackboard, Brightspace, Moodle, any of those. They are connected to your gradebook. Those are all good questions. So thank you again for answering all of my polls. And I will go through some final slides here to wrap up our session for the day. If you do have any kind of final questions for this afternoon, please make sure to get those into the Q&A so I can address them before we finish up. But like I said, I hope you get some, you've gotten some good ideas to consider. Some other things to get you started. You can use group roles. We have examples here in the slides, the little pencil icon will take you to that. If you want to have students take roles like the discussion leader, you can check out that assignment. We have an assignment about guiding your students through the reading as Katie was suggesting earlier in the Q&A, how you might model annotations, ask questions and provide clarification. And then we also have examples linked here on how you might have students annotate course videos and documents. So even if students aren't annotating course readings, they could do things like annotate a study guide or project instructions as well. So I would definitely check out our annotation starter assignments if you've never used hypothesis before. I'm popping this into the chat because these can be really helpful in prompting your students to annotate. And I think this is really important to getting meaningful annotations out of our students. Sometimes if we just ask our students to add three annotations to the text, they tend to do things like summarize the text and not really kind of dig into it. So some of these sample assignments can help us better provide guidance for our students, whether you're teaching a large class or not. So definitely check those out. And then what do you get when you are partnering with hypothesis so if your school already has access to hypothesis. I'm always happy to meet with you and brainstorm how you can best implement this into your large class. So I have keep talking to Katie who mentioned stuff in the Q and A because I've met with Katie before so I'm happy to meet with you too. And we can either walk through setting up an assignment together or talk about how to best implement hypothesis in your class. Our technical support teams also really great at answering your questions very quickly. So if you're a partner school, you have access to all these things. Partner schools also have access to hypothesis Academy. These are two week asynchronous courses that teach you how not only how to use the tool, but ways to consider incorporating it into your class. So our next social annotation 101 cohort launches in about a week and a half and social annotation in the age of AI launches in March. And these two week courses will have you develop your own hypothesis annotation assignment for using in your class and you'll be able to collaborate with other faculty that are using hypothesis as part of the class as well, which is always fun. And then on top of all this stuff, we have weekly workshops available to any schools that are using hypothesis and we take a deep dive into different topics like we have today. So we'll talk about things like grading and feedback for social annotation, annotating and stem subjects. Lots of different things each week. So check out our list of partner workshops and our schedule coming up to register for those. And thanks for the shadow on for social annotation 101 Katie. That is good to hear. And then next. Not next week in two weeks. I'm like, what days today in two weeks we'll be having a liquid margins faculty panel on AI and the future of learning. So if you are interested in talking about social annotation and the way AI will kind of integrate with this or impact this definitely register for our webinar coming up in two weeks. And then finally, those of you who do not have access to hypothesis currently, you kind of find out more about our access to hypothesis through our spring starter offer. That is linked in the chat if you'd like to learn more fill out the form there so you can get access to all this stuff that I just went over. And then finally, my email is here if you have any questions if you're still not sure how you might want to use hypothesis in your large course. Please don't hesitate to reach out because I always love talking to faculty and helping them figure out how to best use hypothesis in their courses. So I know we still have a couple minutes to wrap up. If you do have any questions, please feel free to sneak those into the Q&A before we sign off for the day. But thank you so much everyone for taking the time out to join us and I hope everyone has a good rest of your term.