 Welcome to this hypothesis webinar on grading using our LMS app. I'm Jeremy Dean, the director of education and hypothesis. We're going to be talking today about hypothesis collaborative annotation tool as it plays within learning management systems and our new greater functionality. There's a link on the slide is not public. Nate, could you, I guess I need to do that. Let me get out of here. Thanks for the note. I think I can fix it. If you can't. Let me know, but thanks for the heads up about that Joe. So you can I was just about to say open this deck through the bit. We link there at the bottom. Hopefully Nate's going to rectify the fact that it's not public yet. Thanks for the note, folks. But just for posterity sake, the bit. We link is bit. We slash grade. Anno. And you don't need it to this presentation right now. But there are some links that might be helpful down the road in terms of getting started with the hypothesis LMS app and checking out our documentation around grading. Nate, should I just quickly get out of present mode to open this. Okay, let us know if you guys can open it now. It's a bit. We slash grade. So here's a little outline for our morning together or hour together this morning and now for a brief introduction by pop this organization and what we've been doing in education, an annotation for teaching and learning and talk about the hypothesis LMS up. I'm going to move through those first five sections pretty quickly because I think some of you may be used to this. And then really get down into the grading, the new grading feature that we've launched across the LMS is some time for discussion at the end. I really think this is the beginning of a conversation and work to be done around. What does it mean to grade annotations, how do we grade annotations, how do we grade annotations in different contexts so the discussion piece will be important today and ongoing collaboration will be important about how we move forward with this together. All right, briefly about the hypothesis organization where open source software shop, we develop annotation software for the web. We're here to talk about our education tool. But we have an open source standard based browser extension that's free hypothesis accounts are free. They lack the wild version of hypothesis lacks some of the key functionality that the LMS that provides that streamline students for educators, which is why we focused on our LMS app and its development and will continue to do so for as an enterprise offering for educational institutions. But it's still open source code. And really our company operates by wide range of open open principles. And in the spirit of openness here's the team. I want to give a shout out to everybody there. So you can see the humans behind the technology. It's a really amazing group of folks already this morning troubleshooting some things for me around the LMS app and how it's functioning a couple different places, rapid response. Very much appreciate that Nate Angel is on the call today. You can probably see him co host and my other colleague Michael I saw joined as well. So they'll be in the chat if you ask questions they can they can respond. So annotation is ed tech. Again, many of you have heard this before but when I taught high school and college English. I used to hand out this poem by Billy Collins and oh Dan and annotation marginally at the beginning of every term. So day one I really want to make a point of encouraging students to write in their books. Because I believe that annotation was the most critical practice that would influence their performance and almost every aspect of the course, the class preparation, the test taking the paper writing everything. From my opinion, there's perhaps nothing more essential to learning than reading. And there's nothing more essential to reading than annotation. So as Billy Collins says, all sees the white perimeter as their own and reached for a panic only to show we've not just lays in an armchair turning pages we press the thought into the wayside planted an impression along the verge. So this technology of annotation has been around for a while it's not it's nothing new, right folks have been writing in books scholars students have been writing in books for centuries. But as the as books and other assigned readings move online we lose this essential ability to practice active reading skills to write in the margins to to, you know, be better and more attentive readers more active and critical readers. And large part of what hypothesis do is does is bringing that, you know, age old technology of annotation to digital writing to digital readings online. What that looks like is this. Any website article or ebook document piece of multimedia can have multiple layers of annotation. You can have that traditional private layer of marginal notes. You can also find a public layer for commentary on documents, and you can create any number of private groups, a group for your colleagues maybe you're teaching the same book, or a new publication not in your field of specialization, and any number of private groups for your individual courses, your fall courses your spring courses your fall courses the following year. So all of these different layers of annotation can exist upon a single document. So we've been playing in education for a while now I was brought on to start the Education Department hypothesis. And we've always listened to our users and to our teachers about what they find useful about the tool that's how we sort of focus in on our value proposition and how we decide on what features to do next which is why we release grade book feature integration because it really is something that teachers have been asking about for a while. There's sort of three different ways that teachers talk about hypothesis helping them in their teaching and helping their students in their learning. The first is nothing new. It's this idea of, you know, hypothesis makes reading active. This is what annotation has always done. But as books move online, it becomes even harder for students to be active readers they're more easily distracted, they're less engaged they tend to skim rather than read closely. So maintaining that active reading practice through annotation in the digital space is part of what hypothesis provides for classrooms. It has been shown in research to sort of counteract the trend of skimming and disengagement from online reading and reinstalling and reinvigorating critical reading practices in the digital age. The second piece is new hypothesis makes reading social. So those marginal notes that we took as as young ins in our in our books as good students that were were private they were just for us. And now with the affordances of sort of networked online culture we can share our notes we're not reading the text alone anymore for confused we can ask for help. We can have conversations that help us more deeply engage and extend the course material. I honestly can't say better than this comment from a student of Robin DeRosa is at Plumas State several years ago blogged about her experiences and hypothesis and she writes hypothesis is my literary Facebook when I'm reading I sometimes wonder does anyone actually understand this am I crazy with this brilliant tool I know I'm not alone. And then finally hypothesis makes reading visible and this I think is is something new and something very relevant to our conversation today about grading. I'm sure many of you who teach wonder before whether students have done the assigned reading. The author repeated testimonials from professors using hypothesis is that collaborative annotation lets them know that students have done it there's a trace of their presence in the text. And this isn't just a matter of making sure they do their readings about seeing how they've done the reading where students are confused and might need some kind of intervention where they're excited and might need some guidance and how to develop that excitement to ideas for paper or something like that. And also enabling the teacher to be present in the margins to see how students are are reading and be there with them for that process. The greater functionality we've added will be and be demoing today brings further visibility to the reading process. So I think it's technically by enabling a clear view of individual annotation activity, but also pedagogically in that we can focus in and better teach essential skills that a practice in the process of reading and annotating. So as Linda Parsons at Ohio State says student and patients give me a window to their thoughts and understandings that I couldn't access otherwise I wouldn't get this depth of interaction in a thread of discussion and indeed, a lot of folks have a discussion forum and a kind of evolution of the discussion forum, where those discussions are taking place on top of the text, rather than in some separate tab, they're more authentic, they can be student driven. They're not just sort of teacher prompt student response, they're really more like discussion. In addition to the sort of active deep reading that annotation is known for there are other things that sort of arise out of the way that annotation is practiced online and one of the big ones is is collaboration students are differentiating together or collaborating deeply they're learning to work together to both comprehend information and create knowledge in ways that can be applied beyond the reading course material. And again this is also something that we can look closely at now that we have the grading functionality that we've more easily look closely at now that we have the grading functionality that we're launching. The work that they're doing all this knowledge that they're creating is collecting a kind of dynamic portfolio that follows them from course to course and can be mined at different points in their academic careers. So they have this archive of all their annotations that is useful to them as students moving through a course of study but also very, very interesting for faculty at different levels to look at to see their growth and development and also possibly you know tutors and administrators and others that that might have access to that content. And just briefly as I as always like to point out you know this isn't just an ed tech thing. It's not just a tool for the classroom that gets thrown away at the end of the term. We're building a tool and that we all of you can sign up outside of the LMS for our POTS account and begin annotating your local newspapers online. Because we have this, we are cultivating and building a tool that is for a practice that is for the health of the internet that is for the kinds of practices and skills that we develop in school, close reading collaboration. We're building to each other creating knowledge together, we're developing a tool that's for the classroom but also for the web. Alright hypothesis in the LMS. Last December we launched our LMS app. I'm a tremendous response from education community. We have pilot program with over 20 schools now piloting the technology this spring will be moving our first pilots into into production, all based on this LMS app. Those of you that have been using hypothesis for a while know that you can use that wild app for education purposes, but you're asking a lot of your students and yourself to do so you're asking them to sign up for an account. You're probably asking them to switch from whatever crappy browser they use to a better one like Firefox or Chrome. So there's a lot of onboarding that's that's involved and you're not getting to the beautiful stuff I was talking about above the collaborative annotation. And the LTI app the LMS app really through single sign on streamlines all that so you can get straight down to that good work of reading the assigned readings and discussing them in margins. For those that are unfamiliar. If hypothesis is active on a text. The LMS app enables you to activate hypothesis on select text and if it's if it's active then you can simply select text highlight text to annotate. You can annotate publicly you can annotate privately you can highlight. And in LMS all annotations are default to a private group can apply to annotation so the idea here really it's about discussion threads. And you can annotate in private groups as I mentioned. All right, before we get into the showstopper of gradebook integration I want to talk a little bit about some of the other features that were developing based largely on feedback. And those are groups and sectioning right now there's a single course group provision for a course based on the course shell and LMS and we've from feedback both from classes big and small but especially from larger classes you know through in a person class can't have all three students annotate the same text. So we're going to have some mechanism whether it's through LTI through canvas groups or the grouping structure and other LMS is to create smaller groups sections within within a course. And we're going to develop a better way for students and teachers to view annotations across texts. I call these annotation portfolios but they're really, you know one way thing about them is just activity streams a place where Nick could go and see all his annotations from one course and another course all in one place cross documents, cross time. And then finally learning analytics. And really this just means sort of what what data points are interesting to faculty and students about how they've been annotating, you have the intimacy of conversation with an annotation. But what does that conversation look like at a distance, you know, how many annotations are students creating how many replies, which students are more of replyers and annotators, which threads are the deepest that have had the most interaction, which texts have had the most annotations on them. And we're learning along with teachers about what and students about what data points are interesting in this regard and and what we can say about them research and for how they can change students practice if they notice something about their annotation behavior or how teachers preparing class material help affect their practice as well. All right. Before I go into demoing the hypothesis greater how it works I want to talk about the why of grading annotations. You know, making an annotation social was already a pretty radical move. This was, you know, for centuries as very private practice. And it's one thing to ask students to share their first impressions with a text with their classmates with instructors. It's a whole other thing to say that those first impressions are now accessible. I'll start with a quote from the great late great American literary critic Robert Scholes. We normally acknowledge, however grudgingly that writing must be taught and continue to be taught from high school to college and perhaps beyond. We accept it. I believe because we can see writing, and we know that much of the writing we see is not good enough, but we do not see reading. We see some writing about reading to be sure, but we do not see reading. I would argue, and I think it's not just a single sort of academic or scholastic practice that is contained there. We, you know, the reading material is the input. And the output is some summative assessment. And what happens in the middle is a sort of black box, we don't necessarily always focus closely on or pay close attention to how students are processing reading, how they're annotating, how they're discussing their and how all of that builds into some more traditional summative assessment like a test or paper and I think making annotations gradeable enables us to look a little bit more closely into this black box. Not just about how students are performing but about what aspects of reading comprehension analysis need to be more closely taught, not just more closely practiced. Why grade annotations? Well, first of all, you don't have to. And all the LMS is you can choose to make an annotatable reading either gradeable or not gradeable. So it can be a completely low stakes, not entering into the gradebook kind of thing. But now we've rolled out this ability to make it align in the gradebook and to expedite the ability to send that grade to the gradebook. I do think, you know, if you're asking students annotates, you should at least be reading those annotations and interacting. Whether or not it's a big stakes formal grade, acknowledging them in the form of giving them literally credit for it, I think is, you know, at least for some I would argue is valuable. I also think the work that they're doing is a set of fundamental scholastic skills of encountering a text and processing a text, working with others to understand a text, build knowledge beyond a text. And these are fundamental scholastic skills that can be developed with practice and guidance. And maybe it's to my own shame, but I would just hand out the marginalia poem and say, I look forward to seeing your final papers at the end of the term. And, you know, you should be reading closely, you should be annotating should be highlighting should be finding key passages you should be understanding what they are you should be identifying literary elements in them. If you're a language teacher, you should be adding analysis that connects these different passages in the text. I didn't spend a lot of time teaching all those skills. I spent a lot of time assessing them almost punitively sometimes I would say in grading a paper. But I think the gradebook functionality hypothesis generally but also great functionality offers us new ways to to really hone in and teach those skills. So you can identify and read and then writing skills to be developed and assessed, maybe you're focusing on summary, maybe focusing on paraphrase can depends on the level of teaching that you're at maybe focusing on more higher order analysis. And you can also identify disciplinary practices. One thing hypothesis does not want to do is tell you. First of all, we don't want to tell you that you should grade you have the option not to, but I also want to tell you how to grade for every single assignment, discipline, age level. It's going to be different how folks are leveraging annotation what is expected of students out of annotation. And so we don't have any prescription around how things should be assessed we do in partnership with with faculty and instructors hope to build a repository of rubrics and documents that can guide folks in how they assign annotation and assess annotation. But really, you know, if you've ever played with hypothesis is just this open little bubble that, you know, any, you can do anything. You can write anything you can put images and video in. And so it's really dependent on the specific educational context about how that space is used. All right, without further ado, really quick shout out to Allison I know she made his day she said she was going to on Twitter. But we didn't get this blog post that we published last week or the webinar that you're watching now out in time ahead of the development team on our at our company. So at least the grade book functionality which is great people have been asking for it. They've been chomping at the bit for a while and so some teachers noticed this cropping into their workflow already and already there was praise from Allison at Colgate about how much easier was going to make her workflow. So for those that choose to assess student annotations. Hypothesis greater is going to make their workflow a lot easier. So this is what it looks like in speed greater canvas kind of speed greater. We've had this up for a while actually we launched this ahead of the semester. And it's specific to canvas because of this canvas speed greater functionality. And what you see is the red box there is from canvas it's the speed greater functionality allows you to enter a grade and a comment. Hypothesis does in conjunction with speed greater is focus in on a particular students contributions to the broader class conversation. So right here. I'm just seeing a student main teachers pets annotations on this Mary Oliver poem there may be dozens of annotations and threads from all 25 of my students on this particular poem and I can see that in one view. So you can see I can punch show all there and and see them all even in this view, but this allows me to really focus in on teachers pets contributions, provide some feedback and if I want to a grade and again in canvas, how I grade that pass fail out of 100 whatever out of whatever number is up to me, through the LMS not through hypothesis. And for those that are interested in getting started that is a guide to grading in in canvas down there at the bottom. It doesn't look all that different in the other LMS is we had to develop specifically for canvas first and then we have a broad LTI hypothesis greater that looks quite the same across the other LMS is this is blackboard but there's really no indication that this is blackboard except for the fact that I've said so, because I've cut out the, you know, the framing that the blackboard might put around it for a cleaner view here. And if you are in blackboard that is that link up there is a link to the blackboard grading tutorial so you can, you can go there if you want to get started grading in blackboard. It doesn't look all that different in D2L either. I am going to show it demo it live in Moodle, which again doesn't look all that different but basically what you have in these views is you see the text. And at the top we've done our own kind of imitation of speed grader, where a teacher can toggle through student by student and be just their annotations focusing on just their intentions and then send a grade to the grade book and for now this first one is default, you know, out of 10. And it converts into whatever you've set the grading scheme to be yourself in your own LMS. I'll show this in a second but one neat thing for those that are in Moodle is that in all the other LMS is and this is something we're working to work around and move beyond but in all the other LMS is the teacher is the one that chooses the reading and adds hypothesis. There's no mechanism yet for a student to choose a reading and upload it with hypothesis to the learning management systems, except for in Moodle because of their sort of social constructive constructivist philosophy behind the technology. They have a feature that allows a teacher to hand over the kind of teacherly privileges of an assignment to a student, meaning that they can create a hypothesis assignment that is unconfigured with a reading yet to be selected. And then the student can do the selection process. So a couple examples of how this could work and there's a blog link down here at the bottom of this slide. You know, I might be teaching a poetry course I might have selected largely the readings for the courses. But then at the end I want students to go and find a poem that they bring to the to the group and have and have their classmates annotate. And so here's an example of students that have selected selecting poems. You could also obviously do this with student papers for peer review where there's a unconfigured assignment for each students paper. That the teacher creates within the teacher the students upload the papers themselves. All right, let's jump over into live demo of Moodle. All right, no longer in the slide space. I'm looking at poems here. This is a course called poetry 101. And which poem that I want to look at let's see us look at David Berman snow. I can see annotations in a new window. The hypothesis sidebar pops out. I can see annotations on the document for those that are unfamiliar. I can select text and given the option of highlighting or annotating. Actually, this one, it's not letting me annotate right now because I think it's a completed assignment but I can see that there have been annotations created by the student and a Leav Vygotsky. But can you zoom in on your screen a little bit more so people can see some of that. Better. Yeah, cool. Thanks Nate. So right now I'm viewing all annotations and then again up here there is a little box for the grading. I've already graded Moodle student for six out of 10. But I haven't yet graded Lev. Give them a nine out of 10 and submit that grade. And then let's go to the grades over here. Another tab. Fresh. And there you have, I just entered Leav's grade. I got a lot of grading to do. It's that time of year, right folks. I'm going to go back and grade all these paper, these annotations assignments. But it's as simple as that. This will appear on any assignments that have been marked for grading. The way that you do that in Moodle, and we have again tutorials for all of these, but the way that you do that in Moodle. And there's a way in every single LMS to do this is here to say that you want to accept grades from the tool. It's a little different than every LMS, but I could toggle this off and make this an ungradable assignment. As some of these other readings are. So this one, for example, we won't see the grader at the top of this one. Successfully. No great, no grading up here because I've designated it a non gradable annotation assignment. All right. I only have a couple minutes left here to talk and then if there's questions to surface me and Michael can do that or if there's folks that are still in attendance. I just wanted to ask questions coming up, but I did just want to briefly mention the hypothesis LMS app is free to install and test and use in a limited way in a single course maybe for a semester like a pioneer and teacher that wants to sort of to check it out. But thereafter we asked schools to join our pilot program. So an institution can evaluate the technology for purchase or to move straight if they're already convinced to purchase. And we have a pricing model that's flexible that allows schools to buy in at different levels but the pilot program to refresh this. We're adding new pilots every single day. It's that time here where folks are signing up for the, for the spring semester, and it's certainly not too late from our perspective to bring on new pilots. These are some of the schools that are already piloting. We have some conversations with with you and folks at your school institutions to talk about a spring pilot or, you know, work through getting the LMS app installed and and testing it out and look into pilot in the spring, or in the summer in the fall. But please reach out if you're interested the pilot program provides technical support obviously we have a great knowledge base. Michael is a large contributor there. We offer, you know, personalized or individualized technical configuration sessions and tier one support for instructors and staff. But we also provide tech pedagogical support to whatever degree your institution wants us to sort of run the pilot and work closely with students with teachers. We're happy to do that. Michael's a former educator. I'm a former educator, Caitlin who's on our success to team was an instructional designer and why you so we all have a background in the academic technology and education space. We're here to talk, you know, to introduce obviously to introduce and orient folks to the to the platform to the tool, but also to have one on one conversations with with with teachers about how to apply annotation in that particular context. So, I think it's a great added feature not just to sort of give you the technology and and leave you alone with it but to really be there and make sure every teachers and their classes are successful with the annotation. And then finally, we really are trying to develop a community out of this group of schools. So if you're a frequenter of the major Ed Tech conferences and teaching with technology conferences, we will be doing presentations at some of those as part of the formal conference proceedings but also sometimes piggybacking and doing our own, what we call annotate in conjunction with these partners and I think it's a it's a valuable thing for different schools. You know, instructional designers at different schools but also say composition instructors at different schools to collaborate across institution and the annotate Ed events and and community really provide a forum for folks that are interested in sharing best practices sharing struggles, collaborating on research and publication presentation and things like that. So yeah, I encourage folks to think about a spring pilot. You can go ahead and install the app today for testing as I mentioned, but the pilot program is meant to be really lightweight in its ask of institutions. If you are instructor yourself and you want to use it then you're a third of the way there to getting our minimum required instructors for a pilot cohort. You may be able to just go down the hall and grab a couple colleagues to have yourself a pilot cohort and then get a somebody in the LMS office or in the Center for Teaching and Learning to sign off on on making the pilot official. And then we'll come in and take care of a lot of the programming from there. So, you can click on spring pilot there for more information about the spring pilot, you can click on install the app to get the app installed. And I think I've been talking enough. I don't know if there have been questions or are there are questions now but I want to sort of open it up at this point. Hey Jeremy, yeah, we've gotten some interesting questions. One is around the degree to which hypothesis integrates with Canvas's Rubik tool and Michael has been answering and providing some information around that in chat. I don't know if there's more that you know about that you want to add or maybe Michael wants to say something verbally. My understanding is that, go ahead Michael. I was just going to sum up. I mean, what I wrote in there. So this is, it seems like this is a Canvas issue that affects all all tools that integrate in the same way that hypothesis integrates with Canvas. And the workaround is start off by making an assignment that is not linked to an external tool, put a rubric in it, go back and edit the assignment and then change the change the link to external tool and add hypothesis. I also in the chat added a link to where Canvas users can upvote the idea that Canvas fixes this issue and make sure that you can add rubrics to external tools. And so if your Canvas user is native saying in the chat, please click on that link and vote for it and Canvas has said if it gets to the top 10% of requested features they'll implement it. Yeah, I thought I just thought that rubrics are something that you add to that little speed grater sidebar as a reference in a sense. I think I've seen people with hypothesis assignments that definitely had the rubric in place. Yeah, I'm not off the top of my head. I'm not sure about like adding it just be greater, but there's a like codified like adding rubrics to assignments in Canvas and unfortunately with like LTI tools like ours, you have to use this workaround to have it be there. But it does work if you do the workaround. I did see a question while Nate's digging for other ones about Canvas files. So in Canvas, there's a unique feature actually that will be bringing to the other LMS is which allows you to grab a PDF from the course repository from the course files. It does require a developer key at install. We do allow or we do hypothesis and able to work with a scoped dev key that all may be above some folks's heads in terms of the LMS LTI of it all but the short answer is yes, the longer answer is that when you install the tool you need to take one extra step to make sure that the PDF the instructors will have access to those Canvas files. I just dropped a link. Oh, sorry, I just dropped a link into the chat for our knowledge based article on how to get the canvas file picker button added. If it's not there it's something you may have to pass along to your it administrators and like that if you are not a person already. Hey, while we're just on that subject Leo just asked about annotating canvas pages. You know the native canvas. So you can annotate a Navy's Canvas page with our wild apps as I call it the browser extension. It wouldn't be tied into the, you know, LMS integration. And you can also if you have permission to add JavaScript to a canvas instance or canvas course I think you can add hypothesis so that hypothesis is sort of native to a course if that makes sense. This is like a WordPress app where you can basically turn hypothesis on across a WordPress blog. But again that would be separate from the LMS integration. I totally see the value of what you're asking for Leo and we get asked about this a lot, especially with some of the cool things that folks like Remy Kalir doing with annotating the syllabus, which of course you could upload as a PDF and make an annotatable assignment using an external tool. But more often some of that content is in the pages of a canvas course, canvas course modules and would be need for students to be able to annotate, especially about what assignments they found confusing and such. But, but that's not yet connected to the to the to the LMS integration it's something we're looking into though. So it's all a matter of how extra where and how external tools like hypothesis can play in the LMS space or in the canvas space in this case. Hey Jeremy there's a couple of other questions lined up in our Q&A tool here or zoom Q&A tool I should say. So Victoria asked, she's wondering, and you addressed this a little bit already but maybe you want to say a little bit more about the degree to which a hypothesis account inside the LMS might be connected to what we calling in the wild hypothesis account outside the LMS. Is this from Victoria? Yeah. So, the hypothesis LMS accounts are separate from hypothesis wild accounts. So, when I was in Moodle there, I don't know that my name was Jeremy Dean but I'm a different Jeremy Dean or it's a different Jeremy Dean account than my, you know, Jeremy Dean account that I used to annotate the Austin American Statesman down here in Texas, my local newspaper. They're separate. It is part of the, you know, midterm roadmap to connect those so that like with a Gmail account I might be able to toggle between my hypothesis Gmail account and my personal Gmail account that is depending on where I was. We also, you know, I very much want to make student work within hypothesis within LMS exportable. So, even before we may make a mechanism for the Jeremy Dean in Canvas and the Jeremy Dean on the web to be connected, there should at least be a way for Jeremy Dean in a canvas to grab all my annotations and take them with me. So, there's a couple different ways we're trying to address the connectivity between annotation and hypothesis exists in LMS and annotation and hypothesis as they exist on the web, but very much part of our long term goal to make people annotate the world to connect those things. Did you want to say anything more about that. No, I was just looking down at Martha's question. Yeah, so Martha's up next. Right. So, Martha, what happens is that the LMS changes courses are deleted or the annotated content is removed. So, the first two pieces. The last two pieces there basically you know my my hypothesis canvas account. There's a identity to it. There's some gobbledygook number that that everything is attached to my annotations are attached to there. My reference that I've highlighted are are attached there. And that connectivity is robust another question went away. That connectivity is robust so Jeremy Dean in canvas will always, you know, have some connections stored in our databases between Jeremy Dean's annotations and the references that he highlighted. Even if of course disappeared. Even I forgot the third part of the question was in this jump back. Of course disappears or the annotated content is removed that you know just says on the web, all my annotations and the reference are connected to my account, even if the web page disappears the web page changes. Now, if the course is deleted and annotated content is removed. Currently, there's no way for me to view those annotations. But that's, you know, up next in our, our roadmap is the place where Jeremy Dean canvas can go and view all his annotations in canvas at an institution across documents across courses, and there it wouldn't be affected by whether that course disappears, or whether that content is changed they would be the annotations I created with their initial reference. And the LMS changes that's a lot more problematic so don't change your LMS. There's a lot of other reasons probably. I know everybody's doing that but Yeah, that would that would that would be disruptive. That's that's an eventuality we haven't yet built for you know what happens when Plymouth State changes from blackboard to canvas. The annotations aren't lost but how Martha birdish and blackboard at Plymouth State and Martha birdish at canvas and Plymouth State are connected. The problem may be solved by that same idea of just having a way to link multiple accounts, but currently, if if Plymouth State was switching to canvas like today, all the annotations and blackboard would just sort of still be there. And just for Kate, it doesn't it works with press books but it's a different integration and LMS integration so if your university has press books. You know, the hypothesis can be activated and press books natively. But again that's a situation where, you know, it was, I think a Kate that said, you know, you're going to be Kate Ellis and and press books with hypothesis and then different Kate Ellis in canvas with with with hypothesis but again, like the other questions. We're going to, you know, part of our vision for how annotation would work on the on the web and the different contexts in which one might annotate as a professional as a, you know, individual, you know, done as into the internet. Connecting those accounts, different LMS accounts, wild web accounts, LMS accounts, LMS accounts, WordPress accounts, LMS accounts, press book accounts, LMS accounts, scalar accounts, making some connectivity between all those is definitely part of the goal. Hey, so before we get to Cecilia's great question which could be really interesting and we could spend some time on BC Joseph's question and about non gradable assignments. So the way to give feedback would be as annotate so yeah the question to repeat it for everybody else is in a non gradable assignment with the instructor just have to give feedback as annotations, or in a separate channel. Right, it would have to be in in in an annotation, which would then have to be, you know, public to the group. So you wouldn't want to give a piece of, you know, private feedback in that context and if you want to do private feedback and more extended feedback feedback that might not be relevant to the group yeah you do that in a separate channel. For now, I would love to be honest, Joseph, to have a way, I mean, we've thought about this a couple different ways one might be a way for you to respond to a student in a kind of direct message way where you annotate and it's really just for their eyes. That's something we've thought about. So that may be something that would appear where you know your response to a student in an annotation wouldn't actually be viewable to everybody except for the student that you added if we do it like Twitter. So we've run out of questions in the Q&A but Cecilia brought up a great kind of topic in the chat there about suggestions on how instructors can prepare their class for using hypothesis for the first time. So Cecilia did provide a link that you'll see there about of a Jeremy you've you've done this a lot. Maybe you have some ideas. I do have a lot of ideas. And I think, you know, the number one idea is join the pilot program and then Michael and I will work intimately with your faculty to talk to that situation and have that chat but I mean, I think it's important to set up the why of why you're doing it, you know, like that this isn't just that there's a bigger stakes and just fulfilling an assignment even if you are grading, or if you're not grading especially like that this is for, you know, their edification that reading and annotating is important you can give them Billy Collins poem and say this is how you become a sort of owner of the material you're looking at how you become an active learner. I think there's also things you can do to talk about the etiquette of commentary, how to respond to each other civilly. There's some resources on our website for that that maybe somebody can drop in around sort of productive commentary practices in classrooms. The other thing that I think is important to talk about is just kind of the mechanics of annotation talking about like, you know, not highlighting a whole page highlighting some discrete piece of text that is that you're going to say something focused about this helping students decide when to respond to an existing annotation or creating their own annotations can be overlapping. But if you're really saying something in line with what somebody else has already said about an area of text, why not just reply it's going to make for a cleaner experience for everybody. A lot of it can be very disciplinary specific or skill level specific. As I said, some folks are using this for to practice paraphrase, which isn't particularly discursive but for students that are new to paraphrase it can be very powerful way to visualize the act of paraphrase how to make something say different words than adjacent text. So yeah, I would also you know I probably start off that conversation in the pilot cohort meeting by asking you know what are your goals for annotation what are the struggles you have with the learning goals that you're hoping to address through the mechanism of annotation. I could also activate your mic if you wanted to speak verbally, or if anyone else did raise your hand and not actually enable you to speak to, or happy to address any other questions so we've got one more in the q amp a here from Victoria. Any chance that annotation of images and graphs might be on your development plan. It's not in the near term, but we've already done some work on image annotation to. So there's already some groundwork laid for us to release that but you know once you get into the just the text annotation and the LMS and all the possibilities here. It's going to be a struggle for resources to decide what to do next if we move to image annotation or do we you know create an export mechanism that allows us to start writing from their annotations in a kind of paper like way. So, but it's definitely always been on the roadmap and one of our goals to annotate images and video actually. I'm just putting in a link to have a schedule a session with you. Oh, great. Yeah, Jeremy. Because I think a lot of people might want to do that. But if you want to give a little spiel about kind of how to get in touch and what you might do with folks on in a in a more, you know, a meeting with just just you and them and their people from their school that might help. Well, I just put my contact information on the screen here and you can also see the link to this slide deck again. There's links to the grading tutorial the grading knowledge base on individuals slides above so do grab this if you want an easy way to get back to find out how to set up the grading and Moodle. Reach out no matter what the context is right now we have a big push to get folks using LMS app to get folks in the pilot program. I'm very focused on signing and launching these pilots at schools and I think it's a great way to start your involvement but no matter who you are where you are, I to a fault will never turn down a conversation with an educator about annotation. So just go ahead and reach out no matter what your context even if you're some rogue, uninstitutional, uninstitutionally affiliated or multi institutionally affiliated lover of annotation. I'd love to chat. So, looks like folks may have run out of questions. No need to keep anybody longer if you do want to take off, but we're also happy to stick around till the top of the hour. 10 o'clock Pacific. I miss anything you want to share. I will just say to Michael's credit that you know we have a great. One of the things that our users often read about is our support responsiveness. As I said we're educators by training. Almost all of us. And Michael, you know, man's our support channel and is very, very responsive even after hours. So you may be in touch with Michael if you run into an issue and he will surely do everything you can to help you out. Yeah, I think, I mean, I did want to say if folks are. I guess you should talk to Jeremy under for any reason, but if you have strictly like support related stuff and you want to go right to your support channel. You can email support at hypothesis, you can also schedule a screen share with me and I'm just going to drop the link for scheduling screen shares with me. And into the chat but you know it's a good it's a good thing to start I think with Jeremy but if you're just like how do I what was that link for that campus file picker thing again. Feel free to write directly to me at support. And especially questions around, you know, getting the app installed if you're figuring out how to what if you're the LMS admin or working with an LMS admin like will be a great resource and also if you already have the app installed, but you're trying to figure out how to turn the grading on or how the grading is working with your courses. And that will be a great resource as well.