 Welcome to Liquid Margins. We have some great guests today. We have Katie Cotton. She's a history teacher at St. George's School. We also have Justin Sarenzia. He's the director of the Merck Center for Teaching. He's a history teacher and the Dean of Teaching and Learning, also at St. George's School. And then our moderator today is Jeremy Dean, the VP of Education at Hypothesis. And with that, I'm gonna turn it over to Jeremy. Actually, wait, sorry, mess up. It's Friday. I would like to give our guests a chance to say something about themselves. Thanks. Great, hi everyone. My name's Katie Cotton and I am a history teacher mainly at St. George's, though we just became a humanities department and I've been spearheading our first true humanities class, which is on humanities one for a freshman, which is a place-based class that looks at the school's history and the history of the island that we're on and a few other things. In addition to teaching, I'm the curriculum coordinator for the humanities department as well as the director of our summer programs. And I've been using Hypothesis primarily in my American studies class. This is going on my fourth year for those of you who were in here earlier. I just said my class this morning used Hypothesis for the first time this year and I'm very excited about it. Justin and I teach the same American studies course. So he definitely brought me into the fold and I've been a huge hypothesis kind of advocate and fanatic since then. So thank you for having me today. Thanks, Katie. Hi everybody. I'm Justin Sarenzia, Dean of Teaching and Learning at St. George's School. This is year number eight for me back at the school. And while I do a lot of administrative stuff these days, I find that I find my most positive experiences working with students in the classroom. So came upon Hypothesis about five years ago just as an act of serendipity in lots of ways and have really started to run with it in really fun and interesting ways as we sort of scratch the surface of digital humanities at St. George's at the secondary level. So excited to be here today and to chat with you all. Hi folks, Jeremy Dean, Vice President of Education at Hypothesis. I've been at Hypothesis for about five years and was at a company called Rap Genius before that for four years. And before that I was a high school English teacher and I was actually teaching high school English when I first discovered Rap Genius which is a collaborative annotation platform for song lyrics. Students told me about it because I was always talking about annotation in the classroom and they're like, you should end rap music. So they told me to check out Rap Genius and I dropped everything I was doing at the time digitally and just said, okay, this semester just sign up for Rap Genius. It was a little of a weird thing to say to the kids. And that's what we're gonna use as our sort of digital tool. No more Twitter, Facebook, Wiki, whatever else we were using. And so that is how I got into collaborative annotation and I was offered a job by Rap Genius because the folks there were interested in this use case of students using their technology to annotate. I wasn't just rap lyrics, I was doing it with literature and other things or you know, canonical literature I'd say. And that's how I got into this game. So I wanna start off by asking you to elaborate, Justin, on the discovery of Hypothesis or your discovery of collaborative annotation technologies and tell me a little bit about the serendipity there and tell me your story about discovery. So when I sort of, I was teaching APUS history around a really standard sort of college board oriented curriculum and then had the opportunity to shift away from that. And that was exciting for a variety of reasons, primarily because of the rich history that our school has on our island, which Katie alluded to moments ago. So I started to explore many different textbook options and again, through just a matter of luck and just a lot of research and probably being on Twitter at the right time came upon the American Yop, which is a collaboratively built online open source textbook and said, wow, this looks like a great resource to use in my US history class, but I know students are going to bristle against reading an electronic text and not being able to mark it up or to physically hold it in their hands. I just was hoping or wishing for some form of annotation and I don't know how I don't remember but I just came upon hypothesis sort of simultaneously parallel to the discovery of the American Yop and just ran with it that summer and rolled it out five years ago and was really, really heartened by the types of learning and what I was seeing in the classroom. So that's the short version of it. Don't worry about being short, we've got plenty of time here. And so one question I have as a follow-up to that, Justin, is you were talking about, they normally have a paper book, right? So maybe you were talking about with them, had you talked to them about annotation before this discovery of this tool and the need for it in the sort of digital environment? I didn't, but since this class was primarily for juniors I sort of made the decision late in the spring to prepare for the next year and they're certainly familiar as 11th graders in high school with the concept of annotation but only in really the physical sense by that point. I think maybe in some instances they had seen some like a New York Times annotation of an inaugural address or a state of the union but they were never able to create the annotations layered on top of a digital text. So that was the one way that I felt confident that I'd be able to sell them on this digital text. It also helped that it was a free text and was open resource. I knew that that would be the thing that could give me the inroads to having them commit to the experience. There's a joke amongst high school history teachers that when you give them a reading the book just simply becomes a doorstop for them and they're not really gonna do the reading and maybe you quiz them and I just wanted to get beyond that. Quizzing can have real value but I wanted it to be more organic and I wanted it to feel like the conversation was happening in the text in a way that a discussion board in an LMS typically doesn't recreate. So thinking about use cases in that regard I started to just think about all the possibilities that existed and then committed to it over the summer knowing that it could fall in its face but luckily had a good group of students in that first year and that sort of sold me on the experience as well. And Katie, before you were introduced to hypothesis was annotation something you talked to students about either an analog or digital context or was it a practice you tried to instill? Yeah, it definitely was and I can feel myself doing that with my freshman right now whose texts are analog they do have paper in front of them for the most part right now although I do plan on introducing hypothesis. So I think definitely, I think Justin and I have similar language around how we talk about annotations whether it's on paper or online about having a conversation with the text talking to the text, things like that but I think what's great about hypothesis is that that thinking becomes public or public to our group and it's you aren't just talking to the text you're talking to the text and talking to your classmates at the same time. So I certainly still teach analog annotation and I think there is something to be said about doing those basics of how do I identify the key importance of a text that's going to translate from pen and paper to the internet but I think the collaborative and communal component of it is what elevates it because I think otherwise it's just a silo and I would often have kids say, well, why do I need to underline or write in the margins or ask questions like it's just for me, can I just do that in my notebook? Something like that and this then speaks to a larger purpose for that which is that communal and I have two separate sections of the same class but I have them all in one hypothesis group together so I have 25 minds on one text at the same time which is great. That's awesome. Do you guys ever get any pushback around that sort of transition between either from students or from parents I'd ask or maybe even from colleagues around that transition from like annotation as a kind of private act that's for the individual versus something shared. I'll just, this is maybe ancient history but I know I look quite young but when I was teaching 15 years ago in high school I remember like introducing discussion forums and I had to have a meeting with a parent once because I was like, I think the assignment was like, share your thesis idea for your paper in the discussion board and parent actually called a meeting with me to say, you know, this is Johnny's ideas and his individual work and he shouldn't have to share it with his classmates. Maybe that's again, ancient history but do you ever have pushback against that idea of sort of socializing making it more public your thinking? Justin, why don't you start? Yeah, I'll start. I mean, this is, I'm generalizing here but I feel like in terms of generations they have been more publicly facing for more of their lives than we have and I think it also speaks to like what is the nature of the assignment? If it's just a closed loop between teacher and student that's one form of writing or one form of learning that they might express but if we lead with the assignment, the activity, the learning being more publicly facing I think that's one way to get beyond that and I'll say like, I have experienced some resistance but I think it's just resistance from not knowing and the second you start to illuminate and describe possibilities and for me, like hypothesis has become a feedback for my own teaching. If I go in and check out the annotations from a text that can inform what I'm doing in class that day, that week what didn't land well and it makes me a more efficient and effective teacher for their learning. So when I say that to parents like that immediately goes, oh, I get this. I mean, I'll be honest, I use hypothesis on the parent's family weekend when they come in like I will show them some of the annotations that their sons or daughters are making on the text and it's this moment of like making this really powerful thinking visible that simply doesn't exist in a traditional text and when you show that back to them they get really excited and then they see the conversations that are taking place that then continue into the lunch table beyond the classroom and into the dorm rooms later that night and it sort of becomes not romanticized but it does demonstrate the power and the extension of classroom opportunities that exist through the sort of medium. So I'll kick it to Katie there. Yeah, I would, I've never faced any pushback from parents. I think like Justin said, I think when parents and other colleagues learn about it they're excited about it. I think from the colleagues standpoint I've never had anyone criticize it but I'd had people not want to get on board with it or be like, ugh, like that's not something I want to kind of like touch and play with but since COVID and how we've all been how education has just moved to this digital space so quickly so many of our teachers have hopped on board and I was creating little videos for them last year about signing up for hypothesis and how to use it and showing them what my kids were capable of with it. So with parents and colleagues it hasn't been pushed back against it but either an indifference or a, hey, that's cool. I don't know what you're doing over there kind of thing but with students I would say the only time that I've really received pushback is more when a student doesn't feel like they are learning as an individual from annotating on hypothesis I get sometimes comments that kids are like it makes me stop while I'm reading and it's not so much hypothesis that it is just the annotation process in general I have to stop and I have to break it up and I have to think about it and comment on it but then I think I go to where Justin's kind of saying A, that's the whole point and B, I try really hard in my classes to make everything we do a collaborative experience we will learn better if we are all learning together and that might be an inconvenience for you but you also sometimes don't wanna talk in class but you do it because that's benefiting the discussion and conversation there so I tried to move it past the individuals benefit or lack thereof and talk about it in again that kind of communal sense. Thanks, I wanna sort of start talking a little bit about what kinds of directions and assignments you guys give to students at the secondary level with annotation but I wanna start with something that Justin said the difference between writing and assignments that are for or inclusive of others public or social writing assignments versus what you described Justin as kind of the closed loop assignment and frankly, I'm asking for some help for when I talk to folks because almost in every presentation I've given over the past few months as we've onboarded a lot of schools somebody always inevitably asks me can I make it so that the students annotations are just visible to me and so that I would just see Justin's annotations on say Love Song a Jail for Proof Rock and just Katie's annotations on Proof Rock not in discussion with each other and just Franny's in isolation that again I think that's that closed loop model and I guess I'm just interested in hearing your thoughts I will sometimes say when people ask me that I will say that sort of seems to me like a more of a secondary level pedagogical value or interest, right? Because for example, if I was teaching the Great Gadsby at the secondary level I'd probably do need to know what Franny thinks of the green light and how she formulates some thinking about a particular symbol in the class and if Nate comes in and kind of hogs the space and sort of gives the definition or a full answer like it makes it harder for others or something like that. It would probably be the reverse as Nate is suggesting in the chat that he would be the one at a loss for words and Franny would have taken up the gotten the answer right. But in any case, I don't know, I'm just interested in having you riff on the need for kind of closed loopness in these in an annotation versus the messiness and difficulty of open loopness and this is just something that's a reality I think once Franny has an annotation on the green light it is harder, it is a different thing to talk about once somebody has already kind of provided an answer either you find other real estate to find your voice or you have to build on her thinking and that's a much more higher order activity but nonetheless, I constantly get pushed back people want I want the closed loop version of annotation. Anyway, riff on that. Katie, I'd be curious to have you start here particularly around the Y-par stuff that you did and just to help publicly facing that while like the youth participatory action research that you did with your class last year like I have some ideas and certainly some comments there but I know you do a better job in American studies and often being like very publicly facing in the learning. Like I think for me, when I hear you ask that question Jeremy, I would ask the educators like what is the purpose of this learning? Like what are we doing? What is the purpose of the social experience in a classroom? Like why are we gathering together? If that's only the case if we're only gonna have that closed loopness then we never need to really gather in a social setting either. Like so I'd go to that level but I'll defer to Katie and then tag in because I think Katie might have some good ideas there. Yeah, so for me, it definitely boils down to classroom culture from day one. So I'm on day three right now in class and I can think of every day something I've talked about how like we're not competing with one another like we're all trying to learn from this. It's not about quantity, it's about quality like it's really emphasizing those things that I think our education system in the broadest of generalizations can sometimes hinder in kids that I need to do more and more is better and all of those things. So for me in that sense that if someone got to that annotation about the green light first then my other students are still learning from that annotation in some way and they might have a different example or instance to reply to that or they might be able to elaborate or link to something or bring in a piece of art which are all functionalities of hypothesis which are also wonderful as well or they might throw in a light bulb gif or something like that which Justin loves to play with gifs and I know he calls them gifts and that was our first fight as colleagues. But I do really strive to be very externally focused as a teacher in a I try not to make things only for my own consumption as an educator. So in American Studies, like Justin said we did a whole youth participatory action project a couple of years ago with Newport interviewing different locals identifying challenges in the community doing research about those and then creating some type of resource for the community. One of my final projects for the year is often writing a letter to a local business owner or politician about some kind of topic and making it a historically informed plea to them. So I try to make sure that we know what we can do with the history that we are learning. So in a very like circular way I think that I see the benefit to the closed loop and I understand that but I think that can also be accomplished in different ways. Give kids five checkpoint questions after they read a homework assignment and ask them what their understanding of the green light is. I don't think that necessarily has to be the space that hypothesis takes up in a way. So that might be my approach to that. Justin, do you have anything to add? Yeah, I think that last part is precisely where I sort of land on that. Like I think of for me hypothesis and Katie used the phrase like play and joy. Like I like annotation as being playful and sort of joyful and a sandbox of ideas. I mean, I certainly encourage students to turn off the highlights just to sort of click the little eyeball as they're reading. So they're not being influenced and informed by the highlights that they may see from their peers before they're reading. If you think about buying a used text and the entire text is highlighted that could inform what your eye is drawn to. So that's one part of the closed loop piece. And I think it also just gets back to the social nature of learning. And also, frankly, being in a US history class or an American studies class, it's not just the one text of Gatsby, right? Like it's this big tent giant robust history that we might consider around all different sorts of issues and periods and concepts. So we center a lot of the early learning on some historical lenses and frameworks for thinking about the past. There's things like space in place and history of violence and production and consumption of culture. So I want them to see those themes among others and to come up with their own themes that become more apparent collectively over the course of the year. And when we do that socially, like it inevitably leads to different or new kinds of learning that didn't take place with previous classes in years prior. So that's where I would say I just simply, it's a bit of a block for me. I don't understand the need for that closed loop when you can have the closed loop learning in other formats. For me, just, I don't want hypothesis to be that space. I want it to be a lot of voices coming to the forefront. That's actually really helpful for me and how I'll be able to respond to instructors that ask me that in the future. Justin, since you started five years ago, you must have started before we even had private groups, right? So your students were annotating publicly at that time. Can you talk a little bit about actually full public annotation and correct me if I'm wrong? And also the fact that I, by my memory, there were other courses, students as part of other courses using American Yelp and annotating American Yelp at that time. So I doubt that you guys are, I don't think you were the only ones there. You're absolutely right, Jeremy. I think the private groups came in mid-year of our first year. And I did switch to it just for in Yelp, in particular, because so many people were using it. It was really hard to navigate and see where my students were aside from looking, lumping them into a particular group, even if it was publicly oriented. I will say, there are times when I intentionally have students annotate publicly to have them be a part of the conversation. And an example of that, I'm oftentimes wanting or lamenting the fact that there aren't more electronic digital texts for history. And there are more coming to the forefront every day. There's a really good one in world history right now through the open ed resource project. But an example of this, like I used a blog post, broad post from Waldo Jaquith, talking about the impracticality of a cheeseburger, which is sort of an interesting way, like how you can't make a cheeseburger on your own. You couldn't raise the cattle, make the cheese, grow the lettuce, the tomatoes, the wheat to make the buns. And we were doing it around this concept of, you know, Colombian exchange, interconnection of the world brought through what we might consider like an early modern era. And I had my students annotate publicly and then I tagged Waldo on Twitter. And wouldn't you know it, he then saw that and went into the blog post and started commenting on the students' posts and sort of pushing and prodding them as the author of this initial thing. And it was just this great little moment of the connectivity that exists when you don't have that closed loop. So for me, I, you know, there are times very intentionally when I want them to annotate publicly because also if we think about message boards or comments on websites, like having them be really thoughtful and intentional digital citizens is also sort of a part and parcel to what I'm seeking them to get from an American Studies course. So if they're annotating publicly, they're more likely to be more responsible and intentional with what they're posting and how they're posting and how they're conversing. And I think that's, I mean, it's 2020. We can see that playing out in a number of ways in students' lives. So as somebody who is very publicly facing in their own sort of social media presence, I think it's important for them to get that piece too. That's great. Thank you. I'm a student in your class. It's day one. How do you introduce hypothesis? Or if you went on one of the first day you introduce it, Katie, like, how do you let me, how do you introduce me to this new thing I'm going to be doing? Assuming I wasn't in Justin's course, you know, last semester. So it's a new thing to me. Yeah. So actually, when I first started using it, I pulled some of Justin's annotations from the prior year. He had added me to their group after I got hired before I was teaching here to be able to follow along and see the work that they were doing. So I actually went in and grabbed some sample annotations that I thought were great models, one that felt like a good amount of, like, link and significance. Another that was quite kind of elaborate and had connected to information that they had already covered in the class or a different section. And then a last one that did some good job, like linking to other sources and things like that. So I try to grab a few. And then I have some bullet points that I definitely stole from Justin at one point about, like, reminding them at this point. Again, they're juniors. And Justin and I teach what's our advanced level at our school. So one of the higher levels of the class. So I try not to talk too much about, like, what does it mean to annotate? But I really try to emphasize the quality over quantity. And I try to say that they're, like, not allowed to use the word interesting ever, because it doesn't mean anything. And it's a huge cop out. So that one I try to avoid right from the start, but really show them those examples and show them how to reply and say that that counts as an annotation too, to continue that dialogue. And then I try to tweak from there. I think going to your point before about, I really just want to see one kid's annotations. Yeah, they're all in one group, but you can go in the hypothesis group, click on the username and see what that student has contributed to that text. So we just had our first ones today. So later tonight, I'll go through and just see if there's any feedback I want to give them in terms of depth or quantity or anything like that. But I just think exemplars in this case and reminding them what the point is to annotate. One last thing I'll say is I also try to be really transparent about how I'm going to use the annotations. And I think this is something that Justin alluded to earlier. I'm a big fan of scanning annotations before class and I tend to pull out particular passages that kids really like honed in on or I will grab discussion questions right out of the questions that kids pose on there. And I do that to make sure that we're on the same page about what is interesting and holding their attention. But I also use it as a type of ownership too. So I could say, hey, Justin, last day on that reading, you asked this question. Can you tell me like why that came up for you? And can you say a little more? Does anyone have a thought about that? So I'm also then putting it on the students to bring. I really try to shape it as a pre-conversation. So I really try to like own that when I bring it to the room and carry on the conversation from there. Yeah. For me, having done this for five years now, like I don't like to spend a ton of classroom time like walking through the logistics of how to get on and how to like create your account and what your username should be like. I will say this is the first year in five years where every student did it correctly, like got their account logged into the group. So I feel good about my workflows there. I love the hypothesis animation on the intro that exists on the website. You know, sitting around a campfire in the beginning, like I just love that as a quick little introduction. And then I borrowed this from Remy Kahlir. I just, I have them annotate their syllabus on really the first assignment with hypothesis and have them think about what that means. So I give them some scaffolded instructions like one thing you like, one thing that's unclear and then one thing that you could change because I'm trying to get them have some ownership and agency in their own learning and the idea that they might be able to change something in a syllabus is oftentimes really like far into them. It's new. And then that leads to a really interesting conversation about what their learning should look like and could be in our class for the year. And we typically spend a lot of time in the second or third class talking about that because they've never really been asked that question before. And wouldn't you know it? Like one thing you liked about the syllabus, the word that Katie alluded to, interesting. Oh, I thought this was interesting and then we talk about what that means and really what it means to be additive in annotations. So we do a lot of time early on really trying to onboard them around what a good annotation could look like. So for tomorrow, they're doing Langston Hughes' on Let America Be America again. And they're using some of the frameworks, some of the tags. I'm going to go heavy into tags this year around like the lenses that we might consider often appearing in our class. So I'm asking them to pick out some of the experiences that we've alluded to in the first few days together. But really just a lot of like practice and hands-on for lack of a better word. But in a way that is really meaningful to them. Can you talk more, Justin? And then I want to hear the same from Katie in terms of building on this idea of lessons, right? In terms of our lenses, sorry. And you know, more than just say something smart and additive, how you direct students to be structured in their discussion and commentary on a text. Lenses is one way using the tags. You can elaborate more on that particular, like what kind of lenses you're bringing in, but are there other uses of tags or other ways to structure the sort of annotation exercise that you deploy? You know, so often in history class, it's simply been like on this date, this thing happened or this white guy spoke and this person died. And we weren't particularly in American studies class that's looking at the history side of things. We want to get beyond that and look to like textual, cultural lenses and just a different way. Again, the phrase is big 10 history. Trying to get them to think about the big picture, but there are certainly themes that routinely pop up regardless of the text that you're using, whether it's Yop or phone or give me liberty. Like there are trend lines in American history around narratives that are critical of the past or that are promotional of America's past. I use a number of them that are largely informed by just sort of the American, a more traditional American studies approach around things like race, ethnicity and indigeneity, democracy, activism in class, space in place, production and consumption of culture, like America in the world. So all of these, like no matter what we're looking at in the past, we can oftentimes find multiple layers of lenses that we might consider to be defining that experience, that event, those instances. But what I really like is when the students start to come up with their own collectively, when they start to find new trend lines or they start to connect the dots across time and space, I have found that those sort of tags and those lenses help them to see the bigger picture and not simply just go, okay, we are now 1491 to 1607, it's pre-contact to Jamestown and now we're done with that unit. Well, no, no, like the history, it's gonna build upon that unit and we need to see those themes, those trends, those experiences shaping what comes next and what comes after. So for me, it helps with periodization and for getting them to think chronologically, but also connected across time and space as we go through the course of the year. I would agree, I think in dealing with older students predominantly in what I've done with hypothesis so far, I think as I tackle it, this is only the second year of shop freshmen. So I'm reminding myself already of what that's like. I'm gonna have to create some more structures there, but I think eventually, like Justin said, I think they settle into their own patterns and own connections, I think as we reinforce that, but again, transparency is the big one for me. I try to be really clear, we use Canvas as our LMS and I always write not only the assignment for the next class, but also what we're doing in class, the next class. So that gives them some guidance. Justin and I, when we use Yacht, we'll use like the page notes feature sometimes to add in guiding questions and key terms as well. But my students say we read for our summer reading book, John Lewis Gaddis' The Landscape of History and we were talking about a particular section today that talked about how freedom comes out of the collision between oppression and liberation and they knew that going into their reading for last night. So they read Nicole Hannah-Jones' article for the 1619 project about Black Americans and democracy. So they knew going into that, that they were looking for those themes to be prominent there because that was going to fuel our discussion for next class. So I think transparency is kind of my best suggestion there for folks. Thanks, Katie. And this is my last question and then I think we'll open it up to the chat and to others to chime in. I want to get at something you were pointing out, Katie, about the difference between what it might look like for freshmen at St. George's to be annotating versus seniors. But I want to contextualize that by saying that the vast majority of hypothesis users and partners are in higher ed. So a large part of the audience for liquid margins, although we'll be sharing this with our secondary partners obviously is students and teachers in colleges and universities. And I imagine seniors at St. George's are pretty college ready, right? So they would maybe be equivalent to a lot of, you know, college annotation experience. So can you talk about just what annotation means at high school and maybe especially in terms of like introducing it and early level to freshmen and why it's important then? Yeah, I know in the humanities department we've been working on a skills curriculum the past two years and I think we see our earlier years at St. George's about learning particular skills around English and history and the basics of how to write and how to read and how to research. And then I think we see in junior year everyone's taking American history and American literature really solidifying and honing those skills. And then in our senior year, I think the hope is to then apply them to more specialized courses like like economics or government or something like that. And so for me especially when we hit kind of that junior and senior year annotation is to be able to come to this collaborative space and to create your own meaning from a text. I think in younger years you're kind of given a text and the analysis level is kind of like factual and synthesis like it's more definitions or how does this compare to this other thing we read a little bit more straightforward. Whereas when we're getting into the older years we're asking kids to really create meaning and analysis on their own. So as I begin thinking about working with it with freshmen I think I have that lens of they're going to need more guide like they're guiding questions or are going to need to be more specific and it might be about the content but it might also be the structure. Can you identify like the topic sentence in this paragraph? Like it might be more basic along those skill lines or how does this compare to what we read last night? Whereas I see like Justin alluded to with the juniors the hope is and they actually are pretty good about doing it is that they're doing that naturally in their annotations they're seeing those connections across topic and chronology and sometimes even class subject as well. I was psyched already on day one today with my kids are bringing in stuff from their English American Studies class today and they're reading Claudia Rankin's Citizen and they were talking about Black Americans and citizenship and what that meant. So we're lucky again to have some really strong kids that maybe mimic some of those college age folks but I think you need to lay that groundwork and guide that identification so that they can then can do that for themselves later on. Hey, I think I'm going to jump in here because we're running out of time. We have a couple of questions. Thanks everyone who asked questions and a lot of them were answered right in the chat by their people so that's great. But John Pettis wanted to know about grading and he says, how can you reduce me hating my life when it's time to do grading? What have you guys heard as rubrics for grading annotation and do you have any features to make that less painful for teachers? So if you could kind of speak to that and I guess I'd also add on to that by asking do those rubrics change depending on the class and how they take to annotation and what kinds of things they do. So if there's like this group think in the class do you then adjust your grading criteria? I'll start because Justin's laughing at me and I know he's laughing at me. I do what Justin calls ungrading in my class so I'm essentially like a feedback only teacher. We do have broader rubrics for the class but we sit down at the end of each marking period and kids essentially self reflect propose a grade and we have a conversation about it. So I'm pretty on really on one end of the spectrum with this but I will say I talked to a lot of teachers that are interested in de-emphasizing grades but don't go completely gradeless or to the extent that I do and I will say and I like I read one of these comments in the chat already about making it part of just kind of participation and engagement in general and that's how I actually frame it in my class. I don't use the word like participation or participation grade. I use the word engagement and hypothesis is one aspect of engagement as is verbal communication in class as is participating in group work as is like taking notes in class. So I try to look at engagement as more holistic and their annotations instead of grading them like I said I'll sometimes click on a username in our hypothesis group and kind of scan their annotations as more of a collective and we'll give that student individual feedback. Hey, I had one student say, hey, hey student one. I see that your annotations are quite brief and seem to just be reiterating on what someone else has written. Can you next class try to work on like expanding your comment a little bit or adding another question to the end. So I think another aspect is kind of to look at the collective experience of that student and either create or provide feedback on that. And I'll let Justin now. No, I mean, I think Katie said it really well for me in causation correlation oftentimes the students who are not not doing the most but are most consistently annotating tend to perform really well across all of the different types of assignments I might give to them in the class. So it's a it's for them. It's a formative piece of assessment. And to Jeremy's point earlier and Katie's point earlier about like, I have to stop and then I have to annotate like, yes, precisely that we want you to read closely and annotating your texts requires you to read closely and with intention as opposed to just simply scanning text and having eyes on screen and then saying that you're done when you annotate you have to stop think reflect digest and distill that information in a way that's useful for you. So it becomes a bellwether for me to see where that student is particularly early on if they're only doing one or two annotations a night and they're not very additive I'm gonna that gives me a chance to pull that student in and have a conversation with them about what this should be and how it could look. I typically don't want to grade annotations beyond that participation piece. Sometimes I'll give a very intentional designed annotation assignment but even still, you know, for me like the formative quizzes and stuff it's always very low stakes it's meant to be about the learning and I always tell them like these annotations are for you and that's the collective view. That's why we're doing it socially so you can learn from one another. I will have students who will not they just they don't like to speak in groups but when they get into hypothesis they drive conversation they drive discourse and they'll have all of their peers jumping onto comment and engage in ways that just don't happen in the classroom and I love that because then those students have this really powerful voice and it gives them an opportunity to demonstrate their learning and to really drive discourse and conversation so much so that that will oftentimes serve as a jumping off point for me in the next class. So that's great. I mean, I often think of it as a vehicle to let every student raise their hand because when I in the way back days when I taught composition there were so many students who just would like never raise their hand literally those back of the class students and then the students to raise their hand all the time and you wanted to, you know, say, you know this great I'm glad you're raising your hand a lot but you really want to hear from those others and it was almost impossible to get some of them to raise their hand. So I think with collaborative annotation there is that, you know, maybe it's less intimidating or something but anyway, we are really out of time and I just want to thank everybody who joined us today and also remind you that next week next Friday the 11th the topic will be world languages. So it's global margins annotating world languages and we hope to see you all there. And I would like to give our wonderful guests a chance to say goodbye and leave us with whatever wisdom or happiness they want to leave us with. Yes, I'll start by I think not seeing hypothesis as another like tech tool. I think oftentimes with tech we just want especially right now during COVID but it's like add the thing, add the thing, add the thing. And I think that seeing hypothesis as a tool that is really low lift if you design it that way for the faculty and huge payoff and it's not complicated to use or on board or anything like that. So I think just seeing the additive nature of hypothesis to your students learning is massive. So I don't know if I need to convince people of that but I think the intellectual payoff for how much it takes to use is just wonderful. So thank you, Justin. Yeah, I would agree with that. I mean, I am somebody who's very tech heavy in their teaching and like a lot of bells and whistles and flashy and novelty just as a way of engaging students where they are now but for me in 15 years in the secondary school classroom like this is the one tool that I could point to and go like nope, this has made me a better and more effective teacher and it has supported student learning in really like measurable ways and as a result, I'm just a tremendous fan of it. So I'm so happy to have found it five years ago. I just want to say one last thing for you which is to thank Justin for his many years of collaboration and partnership and it's very exciting that St. George's is moving forward with an official pilot of the hypothesis LMS integration. There are a number of independent secondary schools that have come on in the past few months as you know that model of education is moving online and needing tools like this. And just finally, I just want to say I think I would really like us on liquid margins to continue this conversation and include some other institutions and types of institutions. You know, St. George's one school that we're talking to two wonderful teachers at and look for other secondary school conversations that might include public schools and different contexts in the future on liquid margins. Yeah, thank you for saying that Jeremy. We're definitely going to do that. It's been such a great discussion. Thank you again everyone and don't forget to join us next week and we'll see you then. Have a great weekend.