 So welcome to liquid margins episode annotation unbound social reading for any subject I'm Nate Angel from hypothesis. This is kind of a special episode because we're welcoming folks from basically all over Rutgers and it might sound like that's just a focus on a single institution, but Rutgers is one of the largest universities in the world. And it has so many multiple schools and, you know, campuses and things going on, we really thought it would be exciting to kind of try to get a picture of usage in social annotation across Rutgers. So we invited a number of people here today, I'm going to be handing over the reins to my colleague Jeremy Dean in just a second and he'll introduce these good folks and start the conversation with them in just a second. Thanks Nate. It's great to be here. I'm super excited. I've been working with folks at Rutgers for many years now. Some of our guests today were some of the first users of hypothesis at Rutgers. So we go way back and I'm excited to have this conversation here today. I just wanted to start off by getting to know a little bit about you guys as educators outside of the social annotation of it all. So I'd just like to go around and just tell us a little bit about yourself as a as a as a teacher, what your philosophy is, you know, briefly and we'll get started there. Sylvia, you want to start off? Hi, my name is Sylvia Muller. Let's see. I've been teaching at Rutgers since 1998. I spent a chunk of time as a non tenure track faculty member, but during that time, not tenure track faculty member, but during that time I was also doing work as pedagogical support for faculty that were transitioning to online and hybrid forms of classrooms. So, you know, it's it's I've seen this from both sides of the fence. I've I came from an undergraduate degree at Sarah Lawrence College, which was which is a tiny, small liberal arts college. Every class is heavily invested in independent work with the student working directly with the faculty member and all the classes are tiny seminars. And I did my junior year abroad at Rutgers University to be able to take as many classes in film history as I wanted and was confronted with the fact that what I thought was college teaching and learning was not obviously the dominant pattern at Rutgers. I got through it then by just kind of pretending that I was in a tiny seminar, which I'm sure endeared me to my faculty to the teachers and the other students. But when I started teaching here, it really because I knew of some of the advantages of that kind of close intimate work with material. I've always been trying to get back to high touch kinds of activities, because I've I really strongly believe that there was a wonderful set of research studies. Oh, I'm going to get the title slightly wrong, Christie, can you correct me? The Invisible Universe? No. Oh, they were done at the Cambridge basically it's that students don't come to us as empty vessels waiting to be filled. They come with their own set of ideas, their own mental models for how the world works. And if you never directly talk to them about how they think about the material you're showing them or the processes that you're trying to explain, then you really aren't able to do much in terms of changing their mind. Because what the research shows is that they will kind of conform to what you're asking them to do for a while, but it doesn't create lasting impressions. And you know, one of the things that I'd like to think is that if you go through a class with me is that it doesn't just end at the end of those 14 or 15 weeks. So the hypothesis tool has really, when I saw the tool for the first time, it was so clear to me that this was something that would really match up with a lot of the things that I was always trying to accomplish. And yeah, so I have a lot to say. It's good stuff. That's great. So I love that. I love the idea of hypothesis as a way to bring high touch interaction to larger courses. Christy, why don't we go next? And I think Sylvia reframed my question a little bit, like tell us a little bit about your teacher and then yourself as a teacher. And then how does hypothesis fit into that philosophy? Sure. So my background is actually in K-12 education. I spent a few years teaching high school history before I moved to Rutgers University, where I was an instructional designer. And I also have taught online at the university level for NJIT and for Rutgers Camden. So Sylvia, I've seen hypothesis as an instructional designer working with faculty, and I've also seen it as an instructor myself. And I personally was interested in using it when I was designing online classes, because I had this weird conflict of, I'm an instructional designer and I really advocate for online classes to be planned from day one. And we want everything laid out and everything should be ready to go in these asynchronous formats. But at the same time, I don't want to be the sage on the stage that's just giving my top down information to my students. As Sylvia was saying, my students come with a wealth of experiences. And I was trying to look for ways that I could have them bring their experiences to the online class. So whether they're talking about and sharing and connecting their experiences from other classes, what they've learned, what they have, you know, just experienced in their own life. So I'm teaching a gender and technology class right now. And so a lot of their own lived experiences are really relevant to to the material. So I want them to be experts in my class as well. And a hypothesis has been a really great way to do that. That's awesome. Thanks for that. And just to clarify real quick, Christie's teaching online. Sylvia, do you have a synchronous meeting as well in your class? Are you also fully online? Well, my preferred modality is fully in the classroom because I need a challenge in the past couple of years. But yes, I've been teaching fully online since 2004 and hybrid since 2008. So right now I'm teaching hybrid. OK. Yeah. All right, Rachel, tell us a little bit about your teaching philosophy and how hypothesis fits in. Sorry, you're on mute right now. So I'm a nurse and I began teaching nursing in 2005. Then let's see, I came to Rutgers and I am the director of their pre-licensure program. So I deal a lot with curriculum and we look at, you know, what are we teaching the students and what what are some different ways that we can help them to learn that information? And one thing that I like about social annotation is that it really supports universal universal learning and design principles. Universal design for learning principles. And it really helps to create that inclusive environment. Like what's been previously said that our students, you know, they come with their own life experiences and that impacts how they view the material that you're giving to them, how they learn it. So it's important that with with the inclusivity of social annotation, it gives students multiple means of engagement, of interaction, of expression. And it it lets us know as the faculty, you know, we actually learn more from the students sometimes than we're we're teaching to them. It also puts the students in the driver's seat. So the students instead of becoming passive, you know, receptacles for the knowledge, they become the creators of the knowledge and the learning with social annotation. Another connection with nursing specifically is that, you know, in nursing, we talk a lot about critical thinking and clinical judgment. And, you know, our nurses really have to be able to think on their feet and make complex decisions. Social annotation fits in nicely. So there there is a popular nursing theory. It's a nursing model for clinical judgment that supports thinking. And it's by Christine Tanner. It's called the clinical judgment model. And it focuses on noticing. Interpreting, then responding and reflecting. So when you look at social annotation, that's exactly what we're asking the students to do. We're asking them to take a look at this article, notice what's important to you. Then interpret that. So they're going to think about, OK, well, I like this part of the article, but why do I like it? Why is it important to me? Is it important to something I've learned before, something we're learning in this class, something that I've seen in clinical or maybe even something in their personal lives that they that they can bring to the table for everyone in class to share, but in a safe space. So social annotation creates a safe space because sometimes in nursing, we talk about sensitive topics and you'll ask a question in class and the room is silent. But when you put it into social annotation by giving them an article to look at, they they feel safer because it's an online environment. They feel safer to say what they're thinking. So they also get to not only interpret, but they get to respond. You know, they're they're telling you, OK, these are my thoughts. And then they can reflect. They reflect on the room thoughts through and they reflect on their classmates thoughts by conversing, replying in the annotation aspect. So I really like social annotation and it's used for nursing. That's awesome, Rachel. Thanks for that. I want to follow up with you specifically about that model for clinical judgment that that you were referring to, because I think that could be something cool to share elsewhere in our community of practitioners. Let's use that as a springboard to go back to Sylvia and Christie to tell us a little bit about how how social annotation specifically connects to your disciplines. One of the really neat things about this group, one of the really neat things about what's going on at Rutgers with hypothesis is it's a really vast diversity of disciplines that are using hypothesis. I'd have to give a shout out because I thought this was really cool. For the past couple of semesters, it's been a Caribbean studies course that's the most heavily annotated course, not that, you know, volume matters necessarily, but there's quite a diversity. We heard about the connection to nursing and Christie, you're coming from sort of a gender studies place and Sylvia from social informatics. So tell us a little bit about, you know, how that particulars of that discipline, what you're trying to get students to do and how social annotation helps with that, maybe starting with Christie. Sure. So the students in my class are often coming from a lot of different majors, like some of them are gender studies majors, but gender studies a lot of times is not a primary major at Rutgers, Camden. They have we have a lot of minors or double majors. So for me, the gender studies class really overlaps with so many other disciplines and the social annotation allows the students to bring their experience from these other disciplines into the class. So students have talked about what they've learned in their anthropology classes. I think that annotation this week had a criminal justice major was talking about how what she's learned in criminal justice applies to the the gender and technology things we were discussing with like algorithms and facial recognition. So it's really like I kind of said before, this is a little bit repetitive, but just an experience to let those overlaps with other subjects kind of creep in in a more natural way. And also we do kind of start the class off with more like heavy theory gender studies and it helps the students understand that better because they're seeing what each other is thinking. So they often will just ask a question of like, what do they mean by this? And other students will respond to them. Well, I think that the author is saying this or I'm interpreting it as why. And most of the time they answer each other and I don't have to say anything in the end, besides like, yeah, you got it. So it really helps them also, you know, get through the more dense material. That's awesome. Sylvia, how does it connect to informatics? Well, a lot of students that come to the Information Technology and Informatics program are kind of under a bit of misunderstanding about what the nature of the program is about. So we are going to teach them things about information technology. But the field is really centered around a social sciences approach to understanding all the different impacts that happen when you put human behavior, human practices in contact with rich uses of information technology. So a lot of the times for especially the social informatics class, I'm having to explain disciplinary differences, which are abstract. And frankly, if you're new to the field, they're weird. There's also the goal that I have for that class is to introduce them to the kinds of research that social informatics scholars do. And the whole sequence of assignments is designed to introduce them to, OK, if you want to develop better tech, if you want to implement tech in a more effective way, there is research that has some answers for you and you need to be able to look at that research and interpret it successfully. And because this is a 200 level class, it's mostly sophomores. Sometimes I get a junior or two. Most of them haven't read research studies and there's a lot of questions. So sometimes what I'm asking them, basically what I'm asking them to do is to go through the article and use the annotations and the annotations can be anything from a bringing in personal experiences to I don't understand what the heck they mean by prosopography. What you know, what is that term? So and I tell them that I'm going through and I'm looking at what they're they're annotating and I'm using those opportunities to answer them. Sometimes like what Christy was saying, they answer each other and all I have to do is kind of go, yeah, you know. But sometimes they really do need my guidance because they're reading in a genre. They're reading a type of writing that it's not just decoding the language. It's also decoding a whole super set of structures, which I think would be pretty much impossible. I have not I have not had a ton of success with the research study analysis assignment until I figured out how to interleave it with the annotation exercises. So it gives them enough confidence in I'm successfully identifying the pieces and what's going on in this research to then be able to write about it and present about it for themselves. So so yeah, I see this. I've done a lot of different things in this class. I've been teaching it since 2004, but the rewrite the last couple of rewrites over the lockdown because we were forced online and I had to make better use of it. Really pushed it to, oh, you know, something clicked into place. And it's not that everything is perfect, but suddenly we are able to do things or get the students to learn how to do things in in this 15 weeks that we couldn't get them to do before successfully. So that was a big deal for me, at least I was really excited. Thanks, Sylvia. So as I was saying earlier, a lot of you guys are old school in terms of your use of hypothesis at Rutgers. I was looking at some data this morning and I was seeing some of the first usage was wasn't 2019. Not sure if that was some of you guys. I know that the School of Communication and Information was one of the first to start to play around with the tool. Shout out to Veronica or more. I don't know if she's still there and that Christie was the early there at Camden and herself and others, they're using the tool. And Rachel, I know you've been using it for many semesters. And I'll just say that just really quickly to sort of impress upon the idea that of what grew out of those early experiments, you know, over 10,000 students at Rutgers since 2019 have used hypothesis, which is a pretty, pretty healthy number. So my question is, how has your use of social annotation evolved over that time of when you first discovered the tool and you might have assigned social annotation assignments a certain way and where you are now? Has there been any growth or sophistication in how you're assigning the tool? And I will just grant the fact that there was a pretty major global disruption in the middle of that that probably affected any kind of linear progression of like how one would use the tool and challenge the use of the tool. So you could also mention, you know, how a push to remote if you were synchronous like Sylvia before forced you into new ways of using the tool. But just talk a little bit about how your relationship with hypothesis and how you use in the classroom has evolved and let's go back to Rachel. Interestingly enough, I started using or I first was introduced to hypothesis right when the pandemic started. Rockers, Camden offered a certificate in online learning and Christie was one of the teachers and she introduced me to social annotation. And I actually got to participate in it as a student. So I saw it from the other side and then all these ideas started popping into my head like, oh, wow, this has so many uses for me for nursing. I can use that for my students. So that's how I was introduced to it. And initially, you know, I would assign an article, give the students expectations like this many annotations, this many responses. And then over time, it has evolved. I have a rubric now, I don't just use it as an outside of class activity. I pull it into class. So we we use it as an in class activity. One of the big things I look for, like I used to assign articles on, you know, most of the topics that we were covering weekly. But I try to get to those more sensitive issues now because that's when I noticed the students really open up because we can't get that back and forth in class conversation wise on sensitive topics. So like when we talk about things like addiction and equity and diversity, inclusion, LGBTQ issues in health care, students kind of they'll shut down. They'll listen to what you're teaching them, right? But they're not really diving deep into the knowledge or not engaging, engaging with it, which they're able to do with social annotation. Also in another class, evidence, I teach a research and evidence based practice course. You know, what Sylvie was saying, how she used it so the students could be introduced to like really reading research. That's I had students do that as well. So I would divide the article into components and divide them in the groups and make it a group activity and tell them, you know, this is what you're looking for. And this is how to dissect an article. And this is, you know, for this type of evidence. Now let's do it with another type of evidence and it introduces them to different sources and getting more comfortable in the reading. So, you know, we could look at a research study. We could look at a clinical practice guideline or a position statement. It supports advocacy. And the students start thinking about, oh, well, yeah, I'm reading this and now I want to do this and be an advocate for my patients. It's awesome. Christy, how has your practice evolved? Sounds like you were the the beginning of the use of the tool for for Rachel herself. That is awesome. Yeah. So I first used it in fall 2019. I taught the gender and technology class online asynchronously that fall. And then honestly, this is my the next time I've taught the classes currently because COVID hit. I was doing instructional design support and there was not a lot of extra time to be teaching. So this is the second time I've been using hypothesis in the asynchronous online class. I actually haven't really made changes with how I used it from that first run because it works really well. My class, my online class tends to be pretty small. It's about 25 students. And the way I set it up was I just wanted the students to feel free to get in there and have a conversation without having pressure of specific expectations. So I was trying to avoid creating kind of the sometimes, you know, in online classes, the discussion board can be a little bit more like stilted with the post and reply. And I wanted this to be more informal. So I basically give the students the instruction of you need to go in and annotate. Here are some examples of what a, you know, a substantial annotation is. You can ask a question. You can reply to a classmate about answering a question. You can talk about your own experience and how it connects to that. You know, they have a couple of bullets of like what a good annotation is like, not just like I agree. And I have no requirements of how many annotations they have to post. Just the assignment each week is annotate the readings and it's pass-fail. You annotate, you get credit. And if you don't annotate, you don't get credit. So that system worked well for me. I have some students that go in and really mark the document up and some other students go in and just say a few words here or there. So, but in the end, I feel that the goal of them, you know, really digging into the reading and bringing their experiences to the reading has been met. And that's why I didn't really make any changes to what I was doing. So we'll see how it goes this semester. Thanks, Kristi. Sylvia, how's your practice with social annotation evolved? Oh my, well, I've been using it every semester since I think I want to say it was before 2019, honestly, because I think I was using it. I know I was using it prior to the Canvas integration because the Canvas integration did make things a lot easier. I was looking just at Canvas stats for 2019, so it's very possible. Yeah, yeah. So the first couple of rounds were not successful because we had a lot of technical issues. The Camden integration made part of it easier. What I found I had to do was sort of fiddle with it a little bit because initially I was giving it some grade weight, but not enough. So I had too high a percentage of the students were kind of blowing it off because it felt like too much for not enough. And that drove me crazy because I was like, you'll get so much more out of this if you just do this. So the lockdown actually gave me a language to talk about that because I was like, whenever we are synchronous, it's costing everybody a lot in terms of resources to be able to all be together at the same time. And I don't want to waste time dragging things out of you. So part of what, and I actually wrote this up as a page in the Getting Started section of the online and the hybrid courses, which explained like, this is what I think we're doing with hypothesis. Feel free to disagree with me, but this is why I do things the way I do them. And I wanna make those, if we're in the classroom or if we are on the Zoom meeting, I only have you for an hour and 20 minutes and I need to get as much value out of that hour and 20 minutes as possible. So it's not like, I don't wanna be your task master where I'm just smacking you to keep you moving, but what you will find is if you've done at least a little bit of the reading and you've pre-verbalized some of the things you're thinking about the reading, then you're gonna be much more willing to come in and talk. And after the first, I think the first two semesters teaching during the pandemic, it suddenly struck me that something that was missing because we had so many just logistic issues with getting everybody together that we weren't always doing a good job of following up. So I turned the assignment, which I had been doing kind of in two phases, which is you get some of the points for doing the initial set of annotations and then you get some of the points for the participating and the group activities based on those annotations in the classroom. The last year, I've been doing it as a three phase. So you get each activity is worth 30 points out of 1,000. You get 10 points for doing the initial set of annotations. You get 10 points for doing a good job in the classroom meeting. And then each week, I'm giving them a follow-up thing to do. And I've been using, I've found the follow-ups to be very flexible. So like the first phase annotations is always the same. Just go in, look at what we're reading this week. I'm gonna be discussing, give me five things. I kind of give them a little bit of a ballpark because they're undergraduates. They want a word count. And then, but for the follow-up phase, I can do things like, for example, I expanded the use from social informatics into information visualization. So we've been giving them Tufti, which is dense and chewy and lots of examples. So the Tufti is kind of like the bedrock, but there's lots of other things that we're giving them to look at. So in the follow-up phases, I've been saying things like, okay, go look at the examples in the other reading that I didn't have you do annotations on and pull those into, you know, as examples of what Tufti is talking about in this thing. Or sometimes I'll ask them to do something in their discussion group in the classroom and then report out in an additional annotation back to the original reading. There's, I've really been very happy. There's a lot of different changes I can ring with that depending on what I need them to be doing that particular week to kind of solidify the work that we did together in the classroom. So yeah, but it took me a few goes to figure out what was the right number? Like how much weight is too much weight? And now the issue we're working on is, well, it obviously I need them to be doing some of the cognitive work in the class and that happens in the classroom, but we still have students that are stretched mighty thin with extra jobs, people getting sick, stuff like that. Thank God that's starting to die back a little. So I'm working on trying to figure out what's a good attendance policy because, you know, I need you there for some of this. I maybe don't need you there as much for some of this, but striking the right balance. But yeah, once they kind of get the sense that, oh, Sylvia really is reading these. Oh, Sylvia is really responding every time I have a question. Then that opens up all kinds of possibilities. So there have been some really delightful conversations that have erupted in these annotations, especially with the InfoViz students this year, because there are all kinds of directions you can move off in. So yeah, I hope that's not too long an answer, but it really has been a journey over the last few years with this. No, that was a great answer. There's so many different points in there that I wanted to highlight, but I really love the idea. You're going to say it better than I'm going to recap, but about, you know, making the synchronous time is precious, make it count. And this is a tool that kind of can help make your synchronous time really count. Well, I'll tell you that of all the reasons I've given students for why I like using this tool in my classes, that's the one that really seems to click with at least this group of undergraduates because I know how much some of them are struggling to be there in the classroom, paying attention, you know, the whole time that we spend in intellectual development is a precious resource. And, you know, anything that helps us get and preserve that focus for that time, I think they really do appreciate because so much of their college experience has been disrupted by forces that none of us can control. So prioritizing that focus I think really seems, that seemed to be the tipping point for some of us. It's a real lesson, right? I've been on both sides of it, right? If you show up to teach a class and it's clear that people haven't done the reading, you know, it's less productive, right? And I've been on the other side of that where I didn't do the reading and I wasn't contributing to the productivity of that group time. And it happens in the workplace too, right? You show up to a meeting without an agenda. It's probably not gonna be as focused. And so that's a real life skill, I think, to sort of, you know, put some investment into something before you and then develop that investment, you know, collaboratively. So I love that idea. So Christy this week tweeted a lovely anecdote from a student. And so my next question is around the student response to the tool. And I'll just wanna read Christy's quote. A student literally just told me this week about hypothesis quote, I really enjoy annotation, the annotation portion and reading my peers comments. The annotations helped me to understand the reading and get a chance to see other comments about everyone who was thinking. So we'll start with you, Christy. And then Rachel and then Sylvia, just tell us a little bit about how your students have responded to social annotation as a classroom tool. So I just wanna actually know about that quote too, that that was completely un, like I did not prompt the student for that. I really, I just emailed all my students this week to check in and be like, how are things going? How can I better support you? Very open-ended. So the student really just brought up annotation on her own. But hypothesis, the other time I ran this course was by far the most popular for what I was using and it wasn't the only toy I was using and the most popular assignment. I actually asked specific questions about it in both the midterm evaluation and the end of course evaluations and students very largely, I don't remember exactly the numbers but really felt that the annotation helped them learn the most out of all of the assignments that we were doing in the class and I'm running the midterm evaluations again next week. So we'll see what they have to say this semester obviously at least it's helping one student so far. But yeah, I mean, from my perspective I feel like they really take to it and they get something out of it and based on the feedback I have gotten that is true from the student perspective as well. And Rachel of course, you're a student turned teacher user of the tool. So you know something about the student perspective. Yes, interestingly that Christie had said a student was giving social annotation hypothesis accolades. I had a student last semester so at the end of the fall semester when they filled out their course evaluation, they said that I created a safe space for them and it really touched my heart first of all but I was like, oh wow, that's my goal and I'm glad that you're seeing that and you're feeling that it's hard to talk about some things and we all know how much more comfortable people are to talk when they're typing. So the social annotation when we're talking about this more difficult topics, the safety that they feel and really saying how they feel is important. To Christie said something that made me think of something I wanted to say, but it's left my mind. So I'll let you know if that thought comes back but I do also notice with social annotation, I've had students say to me, I really liked that article that you assigned this week and they actually, they have fun with it. So it's not like a choreo, this is I got to get this off the checklist. I got to go do my social annotation assignment. They actually have fun with it and enjoy it. I feel like so much of what you guys are saying is it's warming my heart and I feel like I could have, yeah, it's great. Sylvia, how have your students responded to social annotation? Pretty well. One of the things that, I know this is gonna sound strange. So I also do work in educational research for the computer science department from time to time. And I also, one of my previous lives at Wreckers was working with the implementation and the building of tools for very large enrollment STEM discipline classes. And one of my big, I think thing that they got a little sick of me talking about was not every F is the same. So I look at DFW rates, I think a little differently which is kind of like Tolstoy. It's like every happy family is happy but every unhappy family is unhappy in their own way. So one of the things that I really have been paying attention to is who's falling off the back end of the bus with what I'm doing in my classes. And one of the things that I think, and again, I wish I could quantify this more precisely. It really to me is a bit of a tone thing. But once I kind of convince them that no, I really am interested in what you have to say and I want you to feel like you can ask questions even if you think they're dumb and all that stuff. Speaking of dumb questions, we actually in one of the Tufti readings earlier this semester, Tufti shows an example of a stereo opticon view. And so in the annotations, the students were like, I don't see what he's talking about. I don't see what he's talking about. So when we got to, I mean, this whole conversation erupted like those magic images that you're stare at and like another whole picture pops up. Nobody was getting it. I was like, guys, number one, very few people can see this without the special viewer. And number two, you're viewing it on a PDF instead of the original book. It's not gonna work. But we had a fun time with that. Anyway, but once you kind of convinced people that this is a safe space, they felt like this is something I can participate in. And if I can keep you with me for more of the course, if I can keep you with me through some stuff, which is maybe gonna make sense a little later, but for right now, I just need you to get through some of the basics with me. That seems to be keeping more people in the class for longer. So I don't think that the number of A's and B's has skyrocketed, but frankly those are the students that were already doing pretty well using everything. And they like hypothesis. I would totally say that I've had a few comments. It seems to indicate that this is something that they prefer to just reading alone. But the thing that I have been noticing is that students that I'm pretty sure, given the circumstances, especially in the last few semesters, would have been Fs have managed to kind of stick with the course a little longer because there is a way for them to participate, for them to build learning, even if they're not completely aware that they're doing it. So that to me was for this past year especially, it was a big win because one of the big concerns I had going into this was college is expensive. And I know that some of you are coming, I didn't say this to them directly, but I mean, talking with colleagues, it's like I know that some of my students are coming with damn near insuperable obstacles to being able to focus on their learning. And my worry was that we were gonna see like this bottoming out and that bottoming out, like the bottom half of the class just doesn't make it across the finish line. And that obviously is going to be largely around socioeconomic fault lines that we have in a public university. So this to me is, I don't wanna oversell it. I mean, like I'm sure that there were other, that there may have been other ways to have accomplished this, but at this moment, this tool, I think was really a lifeline for at least, my classes are generally about 25 to 35 students. So typically in a class that size, I'm looking at five to seven that are weaker in terms of their preparation. And really it's keeping four or five of those students out of maybe seven on the bus, on the boat, on the transportation system moving forward. So I would love to see some, I know you guys probably do quantitative analysis, but it would be interesting to hook that up with what the ultimate course outcomes are because I haven't done this myself because I've been changing the way I use hypothesis over and over, but now with that, I feel like we're really settling into a more solid, consistent pattern. That's something that I'd like to really check out because at least anecdotally, that seems to me what I'm seeing in the classes is that it's keeping more of the students with me longer. That's amazing. I would love to, I love that anecdote, but I would love to substantiate it with a study in some way. So we are looking for folks that are interested, I don't know if you teach multiple sections, Silvia, and so it seems complicated, right? If you really feel like this is helping struggling students, like why deprive one section of it, but that would be a way that we might be able to prove some of what you're saying, but we certainly heard that elsewhere. So we're gonna turn and try to foreground some things from the chat. I will say this bring a request, several requests for your assignments that you teach with to be shared and Christie's already dropped that in the chat and maybe Silvia and Rachel, if you guys have assignments that you'd be willing to share, we can follow up with folks if you don't have a direct link right now. But there's also one question that I wanted to surface around and we covered this a little bit. It sounded like Christie and you shared your assignment, but you're a little more laissez faire in terms of what you're asking since you give them examples and then Silvia, it sounded like pretty sophisticated in terms of like first pass is this, second pass is this and Rachel, you talked about a specific assignment with that one clinical model. So the question is what specific guidelines do you... Well, there's one that's specifically around specific guidelines for the safe space that you mentioned, Rachel, and how you cultivate that. And I think it's contained within the question that that's not just hypothesis. You've also obviously created a culture of that in your course more broadly for students to feel that comfortable, but how do you set up the assignments? Let's start with that. So one of the things is I have to explain what is hypothesis because the students, many of them never used it before. So that's first and foremost, but also in nursing, students wanna know exactly, okay, what do you expect from me? What do I have to do? And I also teach larger classes. So I don't teach huge classes, but I would consider 60 students to be a larger class. So they need some kind of structure and some kind of parameter. So I do assign a rubric. And in the rubric, I have like three areas that I look at and they can get a total of 10 points. So they start out at the top. They start out with their full 10 points and the rubric basically says, this is why you might get a point deducted. And I look at substantive entries is one topic, then grammar and spelling is another one and then participation is the third. But when we talk about substantive, what it really means is that, whatever the student is annotating, they're adding meaning to the conversation. So I give them some examples, like reflect on the reading. What do you think? Pose a question. Is something in there really making you think and maybe you don't understand it? Connect the reading to what we're learning in class or your life or what you learned in another class. So they'll get a point taken off if it's not substantive. Like Christie said, they can't just say, I agree. And then for grammar and spelling, I'd like to see it have good grammar and spelling, but I also in that column, I put that you can use emojis. You can use texting language. I want you to convey emotion. So conveying emotion is important and you won't get points taken off for doing those types of things. And then for participation, I have to be a little more structured with nursing. So I'll tell them, I want to see a minimum of two annotations and one response to a classmate, but to encourage participation and that discourse that we get with social annotation. I say, but you'll get one extra point if you actually have more than your required annotations. That's good. And I actually want to fold a question into this one since you, you contained it there, Rachel, which is how do you set it up and to grade or not to grade? Are you grading them or how are you grading them? And Christie, you've already sort of answered some of this, but I did want to, I mean, you can repeat a little bit of how you set up the students for using hypothesis and you mentioned that you're sort of a pass fail greater for this assignment. But I remind, tell me if you do what Rachel does, which is encourage replies to try to encourage that discursive aspect or yeah, tell us a little more about how you set up your assignment. So like I said, it's just my assignments pass fail and they get credit for any type of annotation. So I actually, I linked to my specific instructions in the chat. I tell them that they can reply to a classmate. That's really all of the kind of encouragement I give them. And I do get in the beginning of the semester, my students are like, but how many do you mean? And like, it's really, I mean as many as you would like to do. So in the first week, they're a little like hesitant about that, but I found that I really haven't had to kind of push them to reply to classmates. I have some students that some weeks go in and they only post replies to people. And then other times people will have a mix. Some students don't really reply as much and just annotate their own thoughts really heavily. So I really get, I found that I really get a mix even without providing them a lot of guidance. I, there's always a pretty rich conversation going on often with like multiple back and forths or even more than one student like, you know, replying to one annotation. So yeah, I haven't, I don't like have an issue with using a rubric or anything. I just didn't want the students to get very hung up on getting a specific grade. I wanted their score around this to just be encouragement to go into the readings and do the assignment as a kind of informal conversation. So that was my reasoning there. And it's worked pretty well for, you know, my specific course. So anything more about your height, how you tee things up and whether you're great or not? Oh, I'm sorry, was that for me? Yeah, just, I've got a cat crying on the landing. So I want to close the door. Yeah, so I teach undergraduates and they're wonderful. You know, I wouldn't run the assignment the same way for a graduate class. I have used this kind of tool in graduate courses and, you know, I don't do it the same way, but I get tired of having the how many annotations conversations. So what I did at what I do, what I started doing is at the start of the semester, I said there are going to be X number of hypothesis activities this semester. The first phase is always the same five annotations. I give them like, these are all acceptable at kinds of annotations, you can reply, you can ask questions, you can comment, you can bring in additional things. I generally don't pay much attention to grammar and spelling in that instruction because I know that when I'm actually grading them, I'm not gonna have the time. So, you know, for that, for my purposes, it's like, okay, you know, I'm gonna care about your grammar in other places, but not here. And the, I go through, and this is the part that kind of happened by accident, but I don't know if you can see these. So I kind of developed for myself a little spreadsheet that I print out X number of copies from. And in the prep time before I walk into the classroom or start the Zoom meeting, I would sit down and I would go through all the annotations for the reading that we were gonna be discussing. And I fire back comments and I answer questions and da, da, da. So the students know that I am looking at this, you know, within six hours before I'm walking into the classroom. So, and of all the grading and as anybody who's had a class with me can tell you, I am sometimes very late with turning in grades. I am never late with that part of the feedback because the way I see it is the whole point of this is the high touch is that they feel like they're getting heard and they're being seen. So if I don't deliver on that, I've just cut the legs out from under my efforts. So yeah, so I do, actually this one is pretty good, but I do little notes for myself like this person only posted four out of five annotations. The first phase is always the same five annotations. The second phase is always, what are you doing with me in the classroom? And then the third phase is always the follow-up which changes from week to week. So I was trying to reduce the complexity because what I was finding for some students was, I don't wanna have a conversation about how many annotations and talking about this many annotations for this reading and this many annotations. That was just bogging everybody down. I do think that would be a useful strategy in other settings. So let me be real clear about that. But in my particular case, it was just needless complexity. So the first phase is always the same. The third phase gets released to them. And sometimes my favorite thing to do, although I can't do it every week, is I will post direct questions back to them based on what I am seeing in their annotations. And then their follow-up phase is to write a 150, 200 word reply to the question that I've left for them. And is that question, Sylvia, happening in an annotation? Yeah, yeah. Okay, so this is, yeah, you are very high touch and it's very impressive and your anecdote about coming from St. Lawrence, Rutgers is wonderful. So you are annotated. I did wanna ask that question to others. Like, and again, one of the neat things about hypothesis is it's used in a lot of different ways, a lot of different courses and in many ways there's no right and wrong. But I'm just curious, Rachel and Christy, do you annotate as well or do you let that be the student space and stand back and watch a follow-up in other ways to annotate or not to annotate alongside the students is the question. Go for Christy, Rachel, sorry again. So it depends on how busy I am at the time. So sometimes I'll jump in and I'll annotate. Other times I don't. So yes and no, but I do enjoy the annotation part of it. With such large classes, sometimes it's hard to do that. So I divide the students usually into groups for an annotation assignment. An LMS system makes it easy to do that. And once they're divided in the groups that I can kind of go in and I have their small group that it's easier for me to make a comment and I'm not making a comment on 60 individuals as opposed to, oh, I see what you're talking about within your group and maybe ask them another question or add to their conversation. Yeah. That's great. Christy? So I usually don't go and participate in the annotations mostly because they don't get the notifications from hypothesis and I want to make sure that they see what I'm saying. So I will either reply to them in like the private Canvas comments or I will make like a broader comment to the class in an announcement or something like that. So I tend to stay hands-off in annotations but make sure they know I'm reading them in some way. Yeah. Yeah, just to clarify, the annotations that I post back to them are not the grade. Right. I have actually some boilerplate that I work with. So the first thing that goes back in the comments is this is to acknowledge that you did the five annotations or you weren't completely successful with the five annotations. The second paragraph is this is what you need to do for the follow-up and the third paragraph is if the follow-up is done and it gives the due dates. So I try to keep the conversation. I try to keep the conversation in the annotations kind of like what I do with them in the classroom. And I'll say this because as I'm getting older and the anecdotes are piling up in my brain, responding back to them in the annotations means that I have imparted that funny little story but I didn't take up class time to do it. So I kind of dropped it where I thought it would do the most good and then I can kind of let it go. So yeah, but it's a balancing act. It definitely is a balancing act. I could go on all afternoon with this conversation. It's been amazing. So many different pieces to jump off from. I know people have other places to go and we've already pushed past our time and I see Franny and Nate coming to tell us to wrap it up. So I just wanna thank you all for a great conversation and for your long-term collaboration around social annotation and teaching and learning which has really been a privilege to be a part of. Thank you. Thank you for having me. Yeah, thank you so much. Really enjoyed it.