 This is our sort of general purpose education section session leading into some specific presentations from professors and others that have used hypothesis in the classroom. So I'm gonna give a very broad introduction to how hypothesis and collaborative annotation plays a role in the education space and then I'm joined up here by three practitioners, professors, faculty members at actually local universities, all of them who have come to sort of tell their stories, share their use cases. So annotation, this is and I apologize if this is a little redundant at times with Jeff's wonderful workshop, if you guys weren't that one as well, but I'll try to be new for those that that were there and also give some of that background on education, hypothesis in education that Jeff did a wonderful job of. Annotation is not a new technology, right, especially not in the education space. People have been using students and scholars have been writing in books since at least the invention of the book. I'm a trained as an English professor like Gardner and I always encourage my students to write in the margins of their books. I actually made it a tradition of handing out Billy Collins's Ode to Marginalia every semester to try to encourage them to do that, to aid in comprehension to help them begin to think critically about the material. But of course as course readings move online we lose that fundamental ability to write in the margins. We lose a fundamental learning practice. And web annotation and the work that hypothesis do, especially in education, is really about bringing that back to the digital environment, making it work across platforms so that each publisher is not producing their own annotation tool, and also adding some exciting affordances from the digital age. So we talk a lot about standards at IAnnotate and hypothesis played a major role along the W3C in getting standards through for web annotation. The thing that excites me about standards, you know, as a former English professor, English teacher, is that that unit that I was always encouraging my students to master, the sort of grabbing of a piece of text and saying something smart about it, saying something significant about it, is now sort of a basic piece of the infrastructure of the web. So I think it's particularly exciting the web standard for those of us in education. So this is what annotation can look like online today compared to that sort of analog vision before. This is moving kind of slowly. This is old school book. Gardner had all those wonderful images of old school books. And then this is what it can look like today. A single document online can have all these multiple levels of annotation. And specifically in education Jeff gave us tons of examples, and there's a wonderful resource that he created about all the different types of annotation assignments that one could have in a class. But teachers and students can have private notes on top of a document. A group of teachers can work together behind the scenes to talk about the teaching of the same document. Public annotations might be on that document if it's one that's in the public domain. And any number of private groups can be created on top of that document for any number of courses within a discipline or across disciplines that are using that same resource. The vast majority of the use of hypothesis is in the classroom. And the vast majority of that is in private groups in the classroom. So the numbers you see here are really almost, you know, very much largely contributed by student annotations. And there are three things that based on feedback from students and teachers that we think students and those in the education profession find so compelling about annotation. The first is hypothesis makes reading visible. As I said, I used to encourage my students to annotate every semester. But it was a black box to me, right? I never saw those annotations. I did teach high school for a while and I knew teachers that would actually make students sort of open their books and show them to the teacher to get to show that they, you know, highlighted or somewhere. But as I think Gardner said, or both I think Gardner and Jeff echoed this, right? We're not teaching them to annotate. We're just sort of saying do it and it's going to, you know, be useful to you. We promise you. And then largely evaluate it based on other artifacts of their learning, you know, the final product like a paper or something like that. But annotation can make that whole process visible to student and to teachers so we can really start to drill into the smaller pieces of the literacy practice of close reading and critical thinking. This is also nothing new again that that annotation is a tool for close reading. But again, hypothesis and other collaborative annotation tools are bringing that into the online reading environment to encourage kids, students to not, you know, be skimming. And you might be, you know, there's evidence that students that are reading online are retaining less and engaging less. But this is a way to kind of re-institute that focused engagement that close reading requires online. And then finally, and this has been said in other presentations too, hypothesis and other collaborative annotation tools make reading social. And it's definitely one of the number one pieces of feedback that we get from students every year when we survey them that they talked about how they loved hypothesis because they learned from their peers. And this is something that education researchers have known for a long time, that knowledge is produced and acquired through a collaborative sort of communal process. And social annotation enacts that. So I'm going to go quickly through some classroom examples. Actually, link, this presentation is linked from the main agenda document. And on this slide, you can link to the great resource that Jeff created for us and that everybody in our workshop contributed to, which is just a massive list of ways annotation can be used in the classroom, different types of assignments. But I'll show you a few here for those that weren't, that aren't familiar with annotation and education and those that weren't in Jeff's session. The bread and butter is definitely a collaborative close reading of course documents. So this is a PDF in a science class, a New Mexico state, and all the students in the class are reading it together, starting threads of conversation, answering each other's questions in the margins. One way I like to think about this is for those that are familiar with education space, the sort of go-to tool of the learning management system is the discussion forum. Well, this is like the discussion forum 2.0, right? And like other applications of hypothesis, it brings the conversation back to the text itself rather than that conversation happening on Twitter, away from the origin of that conversation or that conversation happening at the bottom of a document like a blog post or a newspaper article. We're bringing the conversation back to the text itself, which again, as an English professor, is really one of the number one skills that we tried to teach students. So it can be on PDFs. It can also be on the web. This is from an edX course, collaboratively annotating Wikipedia. But there are other use cases as well. This is a teacher at St. Louis University who had students just using annotation for commentary on each other's blogs. And so throughout the semester, they were writing blogs and commenting on each other's blogs. He was commenting on their blogs. So it can be used for peer review and for teacher feedback. In this case, it was public teacher feedback. They were all kind of discussing the themes and topics of the course. But it can also be used for one to one feedback in the sense that a teacher is just giving private comments on student papers. So one thing I just want to pause and acknowledge here is to think about, even though that collaborative annotation piece, the sort of discussion forum use case is a dominant one, and the one that we focus a lot of our development on, as I'll say in a second, the potential for annotation and education to be an essential piece of the infrastructure of learning and of scholarship for the professors that are doing the teaching is huge. So you can imagine that using the same tool for personal note-taking on top of your reading, whether it's for a course or independent research, both for students and the content before as part of their scholarship, whether it's the primary sources or the secondary sources, all those private notes used by the same tool for the teacher to gloss a text for students or for students to have a discussion on top of. The same tool for students to get feedback from teachers on their writing assignments. The same tool for students to get feedback from their peers. And I think when you think of all the different independent use cases within education, that annotation is going to become something more in a learning sense than the management sense of the LMS, but as essential to the infrastructure of education as something like the learning management system, just because it has so many different use cases at all levels. And then there's some out there use cases in the classroom. This is from the Perseids Project at Tufts, and it's, you can see the annotations, you should be able to see the annotations are rather formulaic. They're not critical close readings. They are sort of, you know, taxonomic formulas and that these were just harvested by the open API and hypothesis to create the real product of the course, which was network relationships of Greek characters and places and characters. So that's a sort of very extensive use in the sort of digital humanities case of hypothesis. A couple of extensions I want to mention that are important. Lots of folks have mentioned John Udell's lab tools that he's created, the prototypes that he's created. Anybody who's using a hypothesis and extensive in the classroom has relied on those tools. I don't have a slide for them, but big shout out to John for creating those tools that help teachers see and leverage the annotations that they and their students are making in the classroom. But I want to give a shout out to Vodong Chan at the University of Minnesota, who has taken what he calls an un-LMS approach and sort of enacted what I was describing, where annotation is really the infrastructure of his course. You can see here, they're making annotations on the course site on top of readings, getting notifications thanks to John's help in Slack, and then those notifications and annotations are getting streamed on the site. So really, annotation runs across everything that the students are doing in the course. Also, Remy Kalir and Francisco Perez at the University of Colorado at Denver. Remy has also been mentioned before, and you should definitely check out his crowd layers tool. The power of collaborative annotation, the first-hand power of collaborative annotation is seeing those student annotations on a document and being there as the teacher and responding to them and seeing that conversation. But there's a lot of power in taking a step back and having a kind of distant reading of that close reading through the kind of data that Remy and Francisco's tool surfaces. We can learn, I think we're really just at the tip of the iceberg here, to learn how students are reading, how students are annotating based on visualization of data that comes from annotation activity. And then finally, and by the way, these slides are all stolen from the aforementioned people. This one from Steele Wagstaff, who's here and may be talking about this a little bit more tomorrow, but Steele always asks a great question or often asks a great question when he presents about annotation, what can an annotation contain? So it can really be, that's one of the great things for teachers about hypothesis, especially, is that there's no prescription about what that annotation can contain. I think you saw earlier an image of the guy Dwight from the office. That was for an assignment where the teacher had the students annotating memes. You had to create a meme and annotate with a meme, right? You could annotate with a definition of word. Hypothesis is not prescribing what goes in that text box, right? It could be something that has an MLA citation like Jeff's assignments if you saw that in the previous section. It can contain anything and indeed it can contain not only rich multimedia stuff, but also quizzes and things like that. This is an example of an H5P interactive quiz that's dropped into the margin for students to respond to. So we're again really just at the beginning of learning about all the possible ways that how annotation can be used, not just by people, but also by applications. So our product development lately has been very focused on the LMS, an integration through the LTI standard from IMS. And we launched our LMS app in December with single sign-on. So that really, if you've ever taught in the classroom and big shout out to folks like Jeff who've been using it for years, you always had to sign up for an account yourself, create a private group, have your students sign up for an account, join that private group manually, and then find out a way to activate Hypothesis, which is often going to involve talking about Chrome extensions or bookmarklets, which not everybody uses in their daily practice, and then talking about how Safari sucks and you really need to get Chrome. So that can eat up the first week of a course and the LMS app sort of takes care of that. You really get right down to the good work of commenting on a text and collaborating with your classmates from the beginning. So it automates the process of onboarding through single sign-on. And one thing that's coming that we're working on now is gradebook integration. And I do want to pause for a second. This has come up with Jeff and with Gardner as well, and elsewhere. And just talk a little bit more about what we're thinking about in terms of gradebook integration, because I can imagine Gardner, though he practices good faith in the age of sort of instant shaming. But also there's a lot of conversation on Twitter about, you know, grading and anti-grading to talk a little bit more about what we mean here. And I think it's a little different from the surveillance mode that a lot of people see it as. But I think there's an important debate to have around this stuff. But for one, gradebook integration has been a top feature request from teachers, real-life teachers in the classroom with need, that I'm not going to ignore or dismiss because they're practitioners and I value their feedback. For my part, I see this kind of, you know, surveillance. And I think Gardner really said it quite nicely in his talk. There is a kind of surveillance that is really about attention, I think he said. I'm losing my place here. But about human attention. And that's really what the focus of our, you know, gradebook integration is. It's not necessarily about the assessment. You can see here in the campus example that even even bigger part of the assessment piece is the comment piece, right? The ability to add private feedback. We lack right now and Jeff had a great suggestion in the previous session around the sort of idea of a one-to-one annotation, the way that I might be able to give feedback on an annotation to a student without it being shared with classmates. We could develop that technology, that additional, you know, feature. But here's a place where you can actually have that feedback. You can comment on the reading process. You can comment on how closely, how actively the student's reading. You can comment on how, you know, they're treating their classmates and their replies. So I think it's very valuable for drilling into the practice of annotation, the practice of reading, the practice of critical thinking, the practice of collaboration around knowledge production. And starting to develop literacies around all those pieces. And also, as I think Caitlin said on Twitter this morning, you know, in terms of data collection, right? It's not just the fact of data collection, but is who has access to it and how it's leveraged. And in our opinion, you know, the teachers and students should be foremost in accessing and leveraging that content for their teaching and learning purposes. In that direction, we are doing some early experiments with the IMS Caliper. Again, John Udell taking lead on doing some sort of Hello World experiments with the IMS Caliper standard. So this is a standard for data in education, so that universities can grab data from all different types of tools and put them in one place and develop learning analytics and learn more about, you know, how performance in different ways relates to success in classes. So this is again something that universities that we're partnering with have asked for. And we're doing some early experiments there. But just a reminder, as if you needed one based on, you know, everything you've seen here and that you'll see tomorrow. Hypothesis is not just another ed tech tool, right? It's super important to everybody who's working at hypothesis and to me that this is a tool that doesn't end after a course ends, right? That is something that a student can take from course to course, whether or not it's part of an assignment and take beyond a campus or from one campus to another campus and indeed beyond and use their annotation skills as practitioners and engaged citizens on the web. Although this is like the most awkward part of the slide, right? Part of a larger mission, business development. Couldn't figure out a way how to like not like really good guys. But we also have to have a sustainability plan. So we're working on that. There's a really healthy ecosystem of annotation tools for the education space. We have representatives from Annotation Studio here sort of. Lacuna was supposed to be here. They're not. Peruzal was here last year. Digo was here several years ago. So, you know, there's a lot of great tools out there and a lot of them are really focused on the marketplace of education. I think hypothesis is distinct in that we're one of the tools here that really isn't just about education. There's lots of people here who didn't come to the education session who may or may not even be involved at a university or a school who are interested in annotation. So a lot of these other ones are really much more focused on the education marketplace and don't go beyond that use case. We have a pilot program right now. It is centered around the LMS app and launching the LMS app at scale at universities and schools. But it's not just about the LMS app. We're working right now in a partnership with Muhlenberg and Davidson College. That's really around the non-LMS use of hypothesis. And we really try to tailor these pilots to be useful to whatever the school is and whatever their needs are. So some of them are more based around research and things like that. Here are some of the pilot schools. You'll notice that you will notice that the University of Mary Washington is not on here. Virginia Commonwealth is not on here. So you may, if you're coming from a university like the University of Wisconsin and you're here and you want to talk and add your name to this list, we'll happily reduce the size of the logos and include you here. But in all seriousness, we just launched this in December which was kind of an awkward time to launch. It's now spring when people are finally taking a breath from the school year and so it's a great time to talk to us about launching a pilot in the fall. All right. I'm going to stop. That's my story but it's always best to hear from teachers about their use of hypothesis. So we're going to turn now to talk to our panel which includes three practitioners of annotation in the classroom. Hunter Hoskins here on my right who's been using hypothesis for a long time. I think we met at Digped Labs several years ago, shared a cab. Jesse Matthews who's at George Mason who has been using hypothesis for a very long time and Chris Carina who's a little new to hypothesis but conducting some really important research with a group of researchers. So let's talk to them. I've sort of formulated four questions for them to frame our conversation and the first one is really just to hear about one particular way that you've implemented the tool in the classroom and let's start with Chris and come this way. In my classroom, I teach freshman composition. I teach two semesters of it and I was playing around with it in the fall. It's not the first digital annotation tool that I have used in my previous life. I was a high school teacher as well. So I did use some digital annotation tools. The first one was comment and I'm not even sure that exists anymore. It's a French company but it's one that I stumbled across back when Crocodocs and those guys were just barely in the space. And then I moved away from it as teachers often do. We kind of lost track of it. So when I had the opportunity to work with some professors across six different institutions, they said hey, let's try this tool for annotation and see what happens. So we did a little research study that we're still kind of looking at student annotations to see what we can see, what happens when we introduce this with students. And primarily I was using it for close reading and rhetorical analysis of text in an argument class which is a second semester composition course. I teach a general education required literature course. 50% of the students in my class put this off to their final semester of their senior year. None of them are English majors so just to say the motivation level in this course is pretty low. One of the learning outcomes for this course is that students need to read for nuance and meaning and detect literary devices operating in a text. So I decided to use hypothesis to give them a means by which to demonstrate their skill at this. So I used it embedded in an LMS in black board in groups, in private groups. And what I have found is that students adapt to it very quickly. They do their best work in the course in these annotations. And then when I move away from the poetry unit and into fiction and drama, the level of reading drops off noticeably. So the one thing that I hear from students is that the part of the course they like best is the annotation part. And it's always followed by, I didn't think I would like poetry that much. So I consider that a win. Hi. So I think the way I teach first year writing and I've been doing that for exclusively for my academic career. And for the last several years, I'm sure like a lot of composition teachers in writing studies have been using texts such as Graf and Birkenstein's Moves that Matter in Academic Writing. So the challenge in teaching a book like that or using that book to help students get acclimated to the academic voice is it's still a book. So they see it as not something that's organic but something that they have to do because here's a teacher. So one of the things that I've used hypothesis for and found a lot of success with it is having students find as they read across the web, not just in my class but for all kinds of academic classes or even on their own, annotate where they see these moves in real life. And so and then we can tag it. Well, so like Graf and Birkenstein talks about three types of openings. So the standard view, a single author type of move in an ongoing debate. So I asked them when you come across these, annotate them and tag it. And that's been a great tool for me to show that it's not something in a book that it's actually authentic. These are authentic moves that writers really use not because it's in a book but because they're the right moves to make. Great. The second prompt was to share a success story but I also want to allow people, I gave them a little heads up about this to answer Dan Gilmour's question from earlier, like something that surprised you, what most surprised you from your experience of annotation or from a particular student and I, let's go again this way. I think the thing that surprised me most was after students use the tool for a little while, they become, especially as they become used to using it as a conversation about the text on the text. They become a little bit more open to saying things like wow, I didn't see it that way. This has changed the way that I'm looking at this. So that was a nice surprise for me because students are really self conscious about being wrong. And it is, even though I've used it as a private group space, it is still a public classroom space and in some reflections because we did a little bit of reflection as part of our protocol, most students said seeing other annotations helped me read better, helped me understand this in ways I didn't before. I wasn't expecting as I went through their annotations that they would actually say that publicly. So that was a surprise for me. It's almost like seeing the process of making meaning and annotation makes one realize that that process exists. Right, they see that as a meaning making process and that they can kind of change their understandings based on the understandings and it's less ephemeral for them as discussion. Right, they can go back and keep looking at it and keep retracing those moves. So this semester I gave my students one assignment where I didn't put the title of the poem or the author and one of the selections was a Bob Dylan song. So it took until the very, very end of the annotation process for some wise student to figure out that it was in fact a Bob Dylan song. And the only reason the student caught it was they were talking about it and her parent, her mom overheard her and she goes, oh, that's, you know, I'm like, so. First I felt really old and then second I thought they are not Googling the lines, right, because that would have popped, this is a super famous song so it would have popped up. Now what I'm teaching drama or fiction, mostly drama, I know there, I can tell the schmoops, you know, sparks notes kind of stuff that's entering into, this is a fully online course into their discussions, but the fact that they did not pick up on what that song was through Googling was to me a big surprise and I thought, well this is great because it means they're staying within the context of the text which is what I wanted them to do. And to be clear, am I right that you use the hypothesis tool with your poetry unit and then you don't use it with the other units, right? Right, don't. So why do you think that is, why do they stay with the text just because they're being directed to do so by the tool? You know, I think I agree with Chris that there is a more intimate social nature about the hypothesis annotation interface. I also use the Blackboard discussion forum which I hate and I'm going to start shifting away from that and doing more hypothesis annotation merely because the look, but there's just a different feel in those annotations and I don't know what else to ascribe it to. Yeah, I've found it is remarkable what happens when my students feel like they actually don't feel like they have ownership over their own work but also really commenting on each other's work. So I use it for a lot for peer review and what I find and what I meant by ownership is they take, I found stuff today that I didn't even realize that students had done last semester commenting on each other's work because they're doing it so often and it was wonderful for me to see this today on and as an aside for that it also helps me see what they're picking up because how are they talking to each other? So they were using terms like oh this is a great example of exigence which was a term that we've been working with this and so it's fun for me to see how they're talking to each other but it's authentic. It's again it's not me saying hey why don't, they're doing a lot of public writing on the web so one of the comments that I saw today was your color scheme here is kind of difficult for me to read and I can say that but they just oh it's professors haven't been telling me to do this but when one of their peers there's a different level of respect that's happening there and I think it in many ways has more impact and that's I was surprised continually surprised at how often they take ownership over their own stuff like that unless we sound like we're an advertisement for hypothesis or collaboration how about something that you struggled with or struggles that came up and your use of collaborative annotation in the classroom we'll start with Chris again. I would say that there were there were two struggles one of them was ensuring that students were making comments in our groups rather than in public because every time they change a device they have to change their settings they didn't always understand how to enable the the interface in a new space so that's certainly one thing that I saw that was that was interesting of course the impermanence of the web I have an entire article full of as Jeff mentioned or Jeffrey mentioned earlier orphan comments that I can't really you know I have the text in other forms so I can match but it's been kind of difficult for me to do that and think through because I was kind of looking at the threads the conversations trying to trace those as part of the research and I can't look at the original anymore because it's gone so thanks Huffington Post for changing your web structure so where I teach when you teach online you have to use blackboard and a fully online course students actually take this course by interacting with your blackboard stuff so if you don't do something with it they think that they don't have to do it so I do grade the hypothesis annotations and having a grade book integration would be really useful for me but I have to give a shout out to John Udell is he here? I think he might have stepped out but yes anyway those extensions he created just simply for the volume of annotations so you can see a single student view was made a big difference in my course this semester but yeah I agree also with Chris about the public group kind of thing students get a little lost with that at the beginning and again grade book integration would be terrific to have I will just quickly interject that the LMS app kind of eliminates the public group so that all annotations go directly to the private one so we can talk about North Virginia Community College the challenge there is that we don't own our LMS system so it's a state-level thing for me the challenge is also the opportunity we're taught and we're always told that millennials and the young people know native technology users but I find that most of my students don't have never really added in the browser extension for example and so this gives me an opportunity to teach a little bit about the web and how it works and so there is there is a little bit of a hurdle but I think that hurdle is a real great opportunity to give a little bit of an overview of how the web works and it's good to know there's browser extensions we only have five minutes left so I actually want to turn and see if there are some questions I did have one other question I asked our panelists that we can finish with but if you want to ask a teacher about their use of hypothesis or collaboration here's a good chance let's use the mics now it is okay so I work at Springer and we're thinking of putting this in some of our textbooks I think originally we were thinking of it as just for students just taking their own notes while they study do you think they do that or do they really only use it when it's sort of a collaborative discussion type thing as opposed to you know you're reading you're just taking notes yep I did have a student who I have several non-traditional students so one of my older students said oh yeah I use it all the time in fact he didn't he had the opposite problem where he started taking his annotations in the group so I get to see what the students really put into the group so I think some will use it I think it's something that certainly would be would have to be introduced mindfully I don't know in my course my students are determined not to read the textbook and most of them don't even buy it sometimes but I would think that in other courses that are really sort of content and textbook based that could be tell me later that they've used hypothesis for their own group work and so on and I think part of I hope maybe I try to introduce things to them that I actually use and explain why I use it and how it benefits me and hypothesis is something that I use daily as much as I'm on the web I've got hypothesis activated to kind of show how I use it in conjunction with Zotero or some other functionality and so I've had a few students say thanks I've used this for my you know group project that I had for you know whatever and so I have found that they will do that yeah there was one particular student at American that I was able to continue to take notes with hypothesis groups for his notes for his other courses that was really neat to see any other questions step up to the mic please I'm just curious about whether you've encountered any issues with lack like do the students I presume are mostly using this on the laptops or desktops and you know the lack of mobile support how important do you think that is for this type of student work that was actually Jeremy's last question was about hopes and mobile integration is one of mine I do have students who exclusively use their phones for even composition they will write entire papers on their phones which to me sounds like a horrific nightmare but they they live on the phone you know I had a student say I finally got a tablet a used tablet so that I can do some of the stuff on blackboard because we're switching from blackboard to canvas and one of the reasons why as I understand it the the state decided to do that was because of the better mobile integration so that is a huge thing for at least our student population I would agree I'll just share briefly that we have had feedback that again we have to focus on mobile optimization regardless of the context but I have gotten feedback from professors that are using hypothesis in the LMS that because it's natively active there it's still not totally optimized alright so I don't see anybody else stepping up to the mic but I'll give one last chance but if not Chris you got your chance to say your hope for the future so since I asked you to do your homework I'll let you sound off Jesse do you have some hopes for annotation in the future actually just hopes for my own use of it how about that expand I would say I at American University I used to think of when I was there I would think of Julian Bond who was teaching there at the time and I thought how awesome would it be to to come across a pdf that somebody like he had annotated and I see that as a wonderful potential to if all of us who are current and alive today working we're annotating all the time and then two, three generations down the line we've got this incredible record of how people have read and thought that's kind of this dream that I've been dreaming excellent so we've got an afternoon of other instructors and professors sharing use case from education thanks to our practitioners here for sharing and we'll be around