 Welcome, everyone, to Liquid Margins. This is called Back to School Engaging Faculty with Annotation. And this is our sixth Liquid Margins. So I just want to thank everybody for joining us today. Right now I'd like to introduce today's guests. We have Betsy Berry. She's the Executive Director of the Center for Advanced, for the Advancement of Teaching at Wake Forest University. And Christine Moskel. She's an instructional designer from Colgate University. And Jeremy Dean, he's our VP of Education Hypothesis. He's going to be our moderator today. So at this point, I'd like to turn it over to the guests and let you say something about yourself or just say hello, whatever you like. So hi, I'm Betsy. I'm really happy to be here. And as Franny said, you know, I'm the Executive Director of the Teaching Center at Wake Forest. But I also teach my background is in philosophy and religious studies. And so I've used hypothesis in my own courses as well as working with faculty here at Wake Forest to think about integrating hypothesis into their classes. And I'm Christine Moskel. I'm an instructional designer for connected learning. Actually, that's my focus here at Colgate. I work in what's called our Learning and Applied Innovation Group in ITS, but we also work very closely with our Center for Teaching and Learning. I actually do not have a background in instructional design. I have a PhD in natural resource management and found, I taught as an adjunct at Colgate and then found my way to instructional design as a different way to support teaching and learning here at Colgate. So I've been supporting hypothesis for a few years now. And I wish I had known about it when I was teaching, because I definitely would have used it, but it's exciting to support other people who are using it in their courses. Great. Thank you. Jeremy, are you on? Yes. Hi. Yeah, I'm Jeremy Dean, Vice President of Education at Hypothesis. I've been working in collaborative annotation for eight or nine years, two different companies. And before that I was a, I got a PhD in English at UT. So I'm also a, well, I'm a lapsed academic, a rogue academic. I'm excited to talk to you guys today about what it's like to go back to school right now. It kind of feels like it has a whole different meaning in this day and age. So my first question to the panelists is a pretty complicated one. And I mean it, you know, genuinely, but how are you doing? Christine, how are you doing right now? With all that's going on. Yeah. Yeah, surprisingly very well. But, um, I don't know. It's this weird situation where Colgate is fully reopening. We're a small school of 3000 students and they're expecting 2800 of them to return in less than two weeks. So I guess I sort of feel like I'm riding on a bus. That's driving off a cliff and depending on how injured I get will depend on like all these other factors in place. So that's kind of how I feel. But right now I'm like enjoying the ride before we get to the cliff. That's how I'm feeling. But I kind of feel like there is going to be a cliff. You know, and I don't know how deep, how, how big the fall is going to be yet. Yeah. Let's see what's your metaphor for how you're feeling. I know I like to use my, my friend Josh Eiler's metaphor of how Indiana Jones where, you know, many scenes where there's one thing that he escapes and then the next thing all of a sudden comes up and now he has another thing he has to do and that I feel like since March, um, you know, no number of things have come up. We've resolved that. And then it's like now time for the next thing. We've even started talking about spring at wake forest as well. Um, just to start planning. And it's just, it's a kind of ceaseless amount of work. And I will say one good thing about wake forest is wake forest really cares about teaching and thinking about teaching. So I have been very heavily involved in the university planning for what we're doing at the, at high levels, which has been a ton of work and it's, and all of you at all of your institutions know how much work your administrators are putting into this and getting faculty engaged as well. And so I'm tired. But I also think the things that we have been able to do at wake, there have been moments of inspiration that sort of keep me going. Um, but I definitely, since it's, we're entering month six now essentially of just nonstop work. I think a lot of us are just feeling like we need, we normally have summers to take a little bit of a break and regroup and strategically plan and we didn't have that. Um, so, so I think, yeah, I don't know. Good metaphor for just being tired and exhausted, but we're a week away from the students coming back and all the different modalities and, and supporting faculty with that. But I think they're ready. I do think they may not think they're ready, but I definitely think our faculty are ready. So, um, so I'm a little anxious and hopeful and also just hope and I can, you know, take one week off someday soon. Maybe, maybe next summer. I'm not sure. Right. Uh, what, what is the plan at wake? Uh, Christine said most kids, most students are coming back. Yeah. So we have, um, so they're just like many schools sort of, there's a discussion of the academic plan and then there's also the residential plan. And so, um, we invited all students back residentially. Um, and not all are coming back, but we also have flipped our singles versus doubles and also preserved a number of rooms for quarantine and isolation. So we also, um, uh, rented out other apartment complexes in Winston Salem so that students can have singles. So a number of students are coming back, but, but in a de-densified campus and then our classes, uh, over 50% of them are fully online, uh, about 10 to 12% are fully in person. So like labs and studio art, et cetera. And then there are some that we're doing blended, but we're not doing high flex. We're doing blended where there's, uh, you know, they made me once every so often in person, but the rest is fully online. So those are our, our three broad modalities and most of the seats because our big classes are fully online are actually online in terms of our coursework. And so that's a huge, that's still a huge increase of in terms of remote teaching and learning for the students there or. Uh, that's compared to normal. Yeah. Yeah. Absolutely. In fact, Wake Forest has, um, traditionally only had online courses in the summer and usually like 10 courses. So 10 courses a year online, typically. And now it's most of the courses are online. So both our faculty and, and we actually surveyed our faculty. I think we found that 10% of our faculty had taught online before the spring, and that includes at other institutions. So we had to go from zero to, you know, it was a big, a big ask of our faculty, but they're amazing. And they, they put in the work this summer to learn and to think, but most of our faculty and students had never experienced remote learning or online education. From afar, it looks like you guys have been doing a great job there. Christine, are students going to be in face to face courses or are people going to be there? Residentially, but taking a lot of courses remotely. My sense is that the majority of faculty are teaching in person. Um, they're going to try to, um, I want to say like, it was like about a third of faculty though, said that they would teach fully remotely. Um, and online learning here is brand new. Colgate, we're a small residential liberal arts college where the face to face interactions and teaching is very sacred. And I think we have like one edX course that's taught every year. And that's it. So this is all new. But so I want to say that face to face preserving that face to face element, socially distanced, of course, I think Colgate's working really hard to make it happen. Although I went to campus today, just on a bike ride with my daughter and there are tents being set up all outside. So there's going to be a lot of outdoor classrooms set up here. Interesting. Interesting. Um, when you guys were talking about metaphors, I was in my mind, I had that, uh, Simpson's, uh, image of like, I think side show Bob is like walking and he, like steps on a rake and he's like, like walks and he steps on another rake and he just keeps, it's like a field of rakes. Um, I know that's not as glamorous as, uh, as Indiana Jones, but same idea. Um, so, uh, tell me what it's like to talk about and we'll get there to, uh, to talk about annotations specifically, but what's it like to talk to. Instructors about technology right now, uh, just generally, uh, and introducing new technologies and helping instructors that may not really have, uh, utilized a lot of technology, maybe even not the LMS. Like just generally what's it like to talk about technology with, with instructors and Betsy, maybe you can start us off. Yeah. I think, um, one thing I've been noticing and we've been talking about here at Wake is the difference between talking with them right now about it versus talking with them. Two months ago and that right now, I think they're in a sort of panic of, I just got to get this planned and put together. And so no new, no new information, please. And that's smart. I actually think on their part, but I would say that, um, earlier this summer, they were really eager to learn it, particularly if we could connect it to what Christine was saying about how this tool is actually going to help you preserve that thing that you love so much about in person teaching, but that you had trouble with in the spring, because we just stuck to the things that were just quick and easy. And so, um, many of them got very excited about that. And so it's really a framing as you all know, those of you that introduced technology framing it as, and I have a wonderful, you know, we have a wonderful team of colleagues that I work with at Wake Forest, who did the same that you've talked to before. I know. And, um, so helping them think through these is not, um, obstacles, but actually reinforcing the thing that they want to do, uh, was wonderful and helping them think about how you might be able to bring this back to your in person classes after this moment. Got them really excited too, because they didn't think it was wasted effort to learn the tool. Christine, what about you? I just, I think it's a lot different than in the spring, because I feel like the spring kind of had this element of this is crisis mode. We're going through a lot of trauma right now. So I feel like that still sort of is lingering in the air, but people, I think, saw this coming. So that, that has made it a little bit easier. Like people realize like we're stuck in this together, like let's say, I just have to say in working with faculty, I just have so much empathy for them. And it's really hard to be empathetic when they're freaking out about something not working. But there just have been so many, for me personally, um, when sometimes when I interact with faculty, they almost feel like they, like we'll make self deprecating comments. Like, I don't know technology at all. And I'm like, well, you figured out how to get into the zoom meeting with me. You know, so, so trying to like boost their confidence, like you might not feel like you're technologically savvy, but you're learning. I'm here to help you. Let's take this slow. We'll figure it out. So I guess that's kind of been the framing for me for how I'm approaching faculty is just. Having a ton of empathy for them. Cause I can only imagine that the pressure is really high. On multiple levels. So, yeah, absolutely. And so Christine, right back at you, like, what is the role of, uh, of annotation, cloud of annotation or social reading in the sort of set of tools and practices. Um, that you are working with instructors to, um, to take on at this moment. I've really been framing. The use of annotation and hypothesis. As a way to build community with your students. Um, because. You know, especially for courses that are meeting like asynchronously. Um, students might not have that synchronous real time interaction. So hypothesis. The mart creates can opens up a community if you are a space for community to develop in the margins of a text, which I just think is really powerful. And I think is something that instructors. I've had good luck. I think peaking people's interest in the tool by framing it in that way. Because when we're not meeting. It just. Like we need more social connection in this day and age. So I think that's a real, that's been a really powerful message to me. Of course there's the deep. It, it helps with deep reading. It helps students see. Like make their thinking visible to their peers and see the instructor. Like that's all important too. But that hasn't really been what I've led with. I've led with this is another tool to create community. A scholarly community. If you will, among your students. That's what it has been for me. Yeah, I love that. Qualified with the scholarly community and sort of training students to think about themselves as a, as a scholarly community. Betsy, what about you? How have you been talking about collaborative annotation with instructors there at Wake? Yeah, I think that helping them see this as a place where you can build community is certainly a part of, of our conversations. Again. Many of our courses at Wake forest are discussion based, not all of them, but many of them. Are sort of traditional liberal arts experience. And so helping them see that there are tools that can allow that to persist outside of the classroom is really important. And I think it's also kind of a, a wonderful alternative for the faculty who are like, I really don't want to do a discussion board to say that here are things here are ways that you can have discussions that are actually pegged to the text in a way that's really powerful. And I also think that, you know, we were having slow, but some uptake even before March of hypothesis because so many of our faculty, myself included care so much about reading and texts. We're very humanities heavy institution. And so this was a just immediately clear to them when you just show them. And that's the key. Hopefully we can talk about that too. Is when you show them how it works, instead of just explaining it to them. And actually having them do it, you know, practice it. I think made them realize almost immediately. Oh, I see how this can be helpful in my course. And so we've had a lot of people interested in using it. That's great. Why don't you pick up where you suggested going next by saying, like, you know, what are the sort of techniques of introducing it? You mentioned hands on and showing them like talk about it. And I think it's really important to be able to, you know, what are the sort of techniques of introducing that you mentioned hands on and showing them like talk about. And we didn't, some schools also hired just a bunch of instructional designers contract ones and we didn't want to do that. So we created a, I don't like this language, but train the trainer model is a quick way to explain it that basically we worked with 66 peer faculty leaders. And we had a learning community with them where we, we taught them things and then they went out and had groups of 15 faculty in their departments and they had learning communities that they ran for them. So it was a sort of scaled way to get to everyone. And I think what was really important about that model back to this question of showing faculty is that we actually ran it like an online course for two weeks. And it was really important to do that so that the faculty got to experience what it was like to be a student in an online course. And so we used hypothesis just because I love it and think it's easy. And they loved it. Like using it to just talk with one another, reading pedagogy articles and thinking about teaching and learning. And then we didn't require them to do this in their learning communities, but most of them adopted hypothesis for their learning communities. And so then all the faculty got to see how it worked there. And so now so many faculty are adopting it for their courses. Just because we did this kind of distributed model of we're going to have to enroll you in a course where you actually are a student using hypothesis and then you're going to enroll your colleagues in a course where they use hypothesis. So it worked out really well. It was not an intentional plan to seed hypothesis, but it has really worked, worked really well for us. That's really cool. I just love the idea that, that a tool is useful to an instructor as part of their professional work, not just in terms of their, you know, teaching part of that, or it's also part of their scholarly work as well. Yeah, I love that. Very cool. Christine, how have you guys been introducing, you know, in a more practical way of faculty to hypothesis or to annotation? Well, I'll also give a little bit of context about just introducing all tools to Colgate, all tools to Colgate faculty, and then I'll position hypothesis there, but we're kind of taking like a, we're just presenting a menu of things for faculty to refer to. And we kind of, they like create their own learning pathway. I wouldn't formally call it that, but that's sort of been like, we're just going to put a whole bunch of different support on the table and you pick and choose what you have the bandwidth to do. So I've been offering workshops on hypothesis all summer long. Like two a week actually for most of the summertime. And again, we have 300 faculty here at Colgate. So if I get five people, that seems like a lot. So in my workshops, I definitely start off with a discussion of just reading in your courses. And I always ask faculty like go back to the spring. And did you notice any change in your students engagement with the reading before and after the pivot to online learning? And I have definitely had a spectrum to like students, readings went up engagement with readings went up because they had nothing better to do to like, I have no idea what was going on with students because there was just so much trauma in my class that like, whether they did the reading or not didn't matter. So I just had the whole spectrum of answers, but that's meant to really start us off in a discussion and reflection of what's the role of readings in our course. And now that we're mostly like online or in this hybrid, mode, how do we, how can we know that students are engaged in the readings that we give them? So then I'm like, Oh, well, here's this tool hypothesis that, and I do, I actually have not this book, but I have a book that's been heavily annotated, like from grad school that I show up and I talk about, I show a picture of a Colgate classroom where the students are sitting really close, they have their readings out. And I'm like, because many faculty expect that students bring the printed readings or the books to class. And I'm like, you can't see what students wrote in the margins of the text. So I actually on Zoom, I like show, hold the book up. And so just meant to like frame up like this is the learning environment that we find ourselves in. And then I introduce hypothesis. I talk, I spend time talking about kind of the pedagogy of web-based collaboration and what the affordances are of a tool like hypothesis. So I mentioned like making student thinking visible is one, one segue, I guess I give to introduce showing them the tool. And then also this fill up candies idea of linking thinking, the idea that within the margins of a text or something you can't do in a hard copy text is provide a hyperlink to another article on the web. So I talk about linking thinking as something that another affordance of web-based annotation is students can be linking to other articles and resources on the web. And that that also gives them a new way to demonstrate there, that they're making connections between what they're reading and all their resources out there like connected learning, right? So I kind of tell that story. And then I introduce hypothesis by just showing example pages online that have been annotated. So I refer to the marginal syllabus. People, they work closely with hypothesis, but the teacher professional development program where educators use hypothesis to annotate articles about equity and education. So I just use, use the, a few readings from the marginal syllabus project just to give people like the 10,000 foot view of this is what the tool looks like. Here's what, here's an annotated page. And then I kind of pivot into, here's some instructional uses of the tool, continuing to show example web pages that have been annotated as part of assignments in higher ed. So that's kind of the arc of the story that I tell in my workshops, but that the workshops again have been the primary way that I've been engaging faculty with hypothesis. That's awesome. I'd like to sit in on one of those. And I'm definitely going to steal that. Send you with a zoom link. Definitely. And I like, I'm going to steal that linking thinking idea. That's really cool. Christine, I want to stick with you just for a second and ask, are there any particular disciplines where you see this tool getting annotation, getting picked up more than others or particular types of courses? Um, I would say it's been pretty evenly distributed. I have, um, Alice and Colazar is a friend of mine. She was on, she's in geology here at Colgate. She was on liquid margins a few weeks ago. And also, um, um, Jennifer Blake Mamoud, she's a postdoc in, um, biology here at Colgate. So though they're kind of our science, um, annotators, if you will, but I've also had, um, faculty who heavily use hypothesis in, um, Spanish. Having students annotate actually annotate in Spanish. Um, history. Um, ge, uh, I said geology, but also geography. So I guess I've had, we're an interdisciplinary liberal arts college. So you, it makes sense that there's a distribution across all the disciplines, but I really have, I would not say that one discipline has had a higher rate of adoption than another. It's been pretty even here. You didn't even mention English, which is the obvious one. But, uh, yeah, that's great that, uh, it's such a wide, uh, interdisciplinary, um, span there. What, what about you, Betsy? And also just the question about what, what types of courses in addition to what disciplines, if there are a lot of smaller courses or some of the bigger courses are using it as well. Yeah. And I, and I don't have as much of my finger on the pulse of this as much as Kyle. And I know he's here. So he can chat about this too. But I, I definitely think before pandemic, we were targeting a humanities just that made a lot of sense that they were so, uh, um, interested in texts and also some in some ways doing less technology in their courses. So it made some good sense to start here. But I think actually because of our peer learning communities being across the board, I know for a fact, just off of the top of my head that there are folks in all the disciplines and all the major divisions of the university that are thinking about ways they can use it. I think they'll use it somewhat differently, which you would expect in different disciplines, but, um, but I do see that, that, um, that I was, I was intrigued and incited that many folks and disciplines that were not text, that were not humanities based were just as excited and saw the, the value of it as well. Um, and that most of our faculty assigned to us, you know, even if it's in a science class, they assigned a textbook. So, um, so it seemed to work well for them. What about pushback? Have you had any resistance to, uh, hypothesis or collaborative annotation? Christine is already unmuted. She's ready for that question. We can go to you, Christine and give, give Betsy a second. Um, so I had an interesting interaction with a faculty member during one of my workshops. Where I just have to frame this as I think it was stemming from just this larger place of almost trauma. Like the, the way that I have taught, like I didn't sign up to do this online, anything. And here I am stuck with this. So, so I just want to, again, my empathy hat is on. Um, but during my workshop, they just started making comments about how. Unfortunate. It is that students are reading texts online now. And that they were very much, I think if they were in, um, language, like a, like a romance language, um, or our department of romance languages here at Colgate. So I think the actual physical act of reading a hard copy of a text was a sacred practice to them as a person and to their discipline. So it was, it was, it was, it was, it was, it was, it was, it was, it was just, it was, it was, it was, it was, it was, it was really important to them as a person and to their discipline. So it was one of these, like in the middle of my workshop, luckily I didn't have too many people and I was able to engage in this discussion, but, um, the pushback was more related to just like, what does it mean to be a reader online? That's so I, I love that you use the language of sacred practice, uh, especially as a religious studies and philosophy scholar, cause. And I will tell you that I actually do see, read like behind me, as being a sacred practice, like in all the ways, all the baggage that comes with that phrase of what we mean by that. And so I actually think, but it's also, there are other sacred practices in these inhumanities disciplines too that are tied to discussing ideas and texts. And so I think saying, yeah, that's great, but there's other things you can do. One of the conversations or struggles I had when I first started using this and talking with other faculty was, I wanted my students to read hard copies, again, not necessarily rationally. So this is related to being a sacred practice. But I loved the discussion so much that I was like, well, that outweighs the value of that in this case. And so I even told some students to buy the book as well. So use this for the discussion, but buy the book because you will have it in your library and that's really important to me. And of course, they probably rolled their eyes. I mean, like I'm not stuck in, I'm still in my old school ways, but I do think reminding them of the other side of the benefits could be helpful there. But I totally am not surprised when you said that. I haven't experienced that, but I'm not surprised that you had some faculty who were just very sad at the loss, grieving the loss of the hard copy book. That's what I, and I also think that they also expressed this sense of we're all zoomed out and like we're just like glued to our computers all the time. And like, so I think they were struggling with like, I don't want to be complicit in having my, giving my students more screen time to do this reading. Yeah, that's really interesting. Yeah. So that Jeremy, I hope that provides an answer to your question around pushback. It does. And I would just add that, you know, like I too, I don't have such a lovely bookshelf as you do back there Betsy, but I also, you know, consider books sacred. If I was still in the classroom, I wouldn't want students to be buying the books. I think it's hard to quantify it, which maybe you just find, but I want my kids, students get lost in the book in some physical way, like go forget time underneath a tree on campus or something like that as you get lost in the book. But in terms of sacred practices, you know, there, you know, what there can't be an either or, right? It's not like, you know, texts are online, right? And it's not just the books themselves, they're physical books that are sacred, right? There are a lot of practices that surround the books. And it's true that when books move online, a lot of those practices become harder, like annotation and it's, and so I guess I'm just thinking that, you know, you need to preserve some of the analog sacred skills in the new environment. And that's one way to think about this is that the books are going there, you know, and your students are reading online. You're not going to be able to fight against that, especially not now maybe, right? And so why not focus on some of the sort of corollary aspects of what we consider to be our suite of sacred practices in academia and try to make some of those as authentic and vigorous as they can be online as we move into these new era, you know? This is great. I want to pause here and see with Franny or Nate. Actually, I'll first turn it to you, Betsy and Christine. Just see, is there anything that you wanted to share as you were thinking about this conversation this morning that I haven't been able to elicit in my questions? And then, or is, or... I mean, I can say more, but I think I'm gonna wait for the, I'll just let's see if the, I'm sure the questions will raise the things that I will add and if it doesn't come up, I'll say something before we end, yeah. Fair enough. Franny or Nate, has there been any questions in the chat that we want to surface for our panelists? Yeah, there have actually. And you've already sort of touched on this idea of community, but Terry Elliott wanted to know, asked a question and then answered it as well, but other people ask it. What does community mean exactly in terms of social reading? And Terry's answer, but yours might be different, is that it extends to other networks. This idea that it's not just the community and the margins, but a larger community. So I was thinking of when you were talking about linking, adding a link to go somewhere else out onto the web, but can you talk about that a little bit? Well, I'll follow up. And I think I can tie these two pieces together, I'm hoping the community piece and then the linking. And I actually, I'm glad, Franny, you raised Terry's question because I saw that earlier and I was like, oh, I want to clarify what I mean by building community. And what I want to say is just based on my own experience, using hypothesis to annotate as part of the marginal syllabus, for example. But one thing that I think is important to reflect on is, if you're a student in a course and you don't know all the other students yet, or if you want to dive into annotation as part of an organized experience on the web, like the marginal syllabus project, it's very, you're very vulnerable to like add a comment in the margin of a text and you don't really know who the people are who are going to read your comment. And so at first, like you have to almost get over that hump of fear. Like how are the readers of this text going to think about my annotation, about my comment? But I have found related to building the community, then when people start replying to your comment, it's like really invigorating, I guess, and like exciting that, oh, I, like something that I, that came to mind at this point in the reading also resonated something similar for someone else. So then I feel like that starts to build connections and links, I guess, between a group of people who find themselves in the margin of a text. And that from, if you can leverage that, I think that that can help build community around like shared interests, right? And of course, students are there, they might be there because they have to check the box. This is a Gen Ed course, but hoping to like use hypothesis as a way to surface people's interests that relate to the reading and then that relate to each other and then that maybe if people start replying to each other in the margin of the text that that interaction is what starts to build community and connection between people. And then of course, if you're linking out to other communities on the web, that also helps strengthen and build networks. So that's what I wanted to say there. Yeah, that's great. And I want to kind of ask the follow up because there was another question about, so in terms of that community, you know, we're building community, it's like a safe place and everything. So then how does assessment, how do things like assessment and grading figure into that, especially if it is, you know, a required course. So those things can seem to be at odds. At among the faculty that I've worked with who have used hypothesis in their courses, they don't tend to assign a grade like each annotation gets a grade. They view, they kind of position annotation or engagement in annotation as part of the participation grade overall. Like you're going to use, this tool is there for you to participate in these discussions in this class. So it's, I've seen it been used as just an assessment, if you will, of students' participation in a course. So I have, I think it's, I think the question was really well put because it depends on what your purposes are for using it. And I actually, part of the reason I love hypothesis so much is the more that I've learned about teaching and learning and the more I've thought about my own teaching and what I really care about for my students, the more I realize how central the outcome of learning how to read is. I mean, it's the most important goal in my course and any of my courses is that they learn how to read and that they read in a certain way. They read in a certain way. Of course, that means I have to teach them how to read. And so in my courses now, learning how to read is the central outcome, which means that it's also the thing I assess the most and do, and I do it, I do specs grading and so I'm not doing it like eight, 90% or whatever, but I do have a rubric for how to raise certain kinds of questions about a text, et cetera. And so I actually, I don't want to overwhelm them because I do think it can serve a role for community as well, but for me and my course, it's actually hypothesis can serve as an assessment tool. And of course, there's lots of practicing that they do and I say, okay, like, so this is the way you ask the, you notice this in the text, but maybe you wanna notice something else here and here's another way you can ask the question differently or extended or connected to other things in the text. So I think in special cases, you can assess reading, but I don't think it's like just assessing did you, I mean, I do have, if it's in my participation grade, it's like, did you do some of them? But typically what I did was, as I said, there would be four annotations that would be assessed throughout the semester and they knew in advance and then they would practice beforehand and I would give them feedback, but those four I would assess with a rubric and so they would obviously put more time into reading and giving better annotations, but it wasn't really community, they weren't communicating to each other necessarily, although they did get credit for that, but for me, it was really them engaging with the text almost like the sort of this could be a tool even if they couldn't see each other's annotations, I could use it that way too. So yeah, so I think I hope that gives you all some ideas of how assessment could work without it being so stifling that they would stifle their communication, but it was still really important for me to do and that's why I did it. I wanna make one comment and then follow up with you Betsy about something you said, you know, one thing I've thought about, I haven't thought this out fully, but you know, if you know, you always see something like collaboration is like a 21st century skill, right? Or something like collaboration is something that we should be teaching students and instead of the more sort of harder skills from previous sort of rubrics that have been designed for what student outcomes should be. And I think if collaboration is a skill we're trying to teach in an outcome that we want to achieve, I've thought about this and I've never put into practice, I haven't been in a classroom that like how somebody is responding to classmates with their in a threaded conversation, right? How they are building on other people's ideas. Like the very idea, I hesitate to put to make it, you know, analytic to say like building community is something that can be assessed, but community skills and collaboration skills may be something that can be assessed or at least addressed, right? How a student is interacting with classmates, how they're building knowledge with classmates can be addressed through the visibility as Christine has been saying of hypothesis and the way it makes reading and thinking visible. But Betsy, I wanted to follow up with a very specific thing that you said about reading, right? This is something I've been thinking a lot about lately and it's a little tangential perhaps. So I think, you know, talking about reading and introducing reading is essential to college experience. And so probably would resonate with your instructors to frame it as, you know, reading being so vital. But, you know, just to not to push back but to hear a little more about it, like your students are literate when they arrive at your course, right? And so what do you mean when you say teaching, reading? They need to learn to read. What if some of those, you mentioned a rubric, which I'd love to see at some point, but like, you know, what does it mean to learn to read for your course, for your discipline, even though they probably know how to read in the kind of fundamental sense. There are other literacies they're developing, like, can you be more specific about that? Yeah, so I think, and it depends on discipline. So I noticed too, when I've taught, because my PhDs in religion I do, but I'm an ethicist by training. So I really read like a philosopher. And so I can, I'm gonna look up some things that I can share with you too about participation in philosophy classes that can be helpful. But for me, it's really learning how to read for argument and to see assumptions and to see connections between ideas. And so it's not just like, I literally read the words, but what's, and in the literature class, it would be other things that you're seeing behind the words. In a philosophy class, it's understanding motivation for arguments. It's understanding the structure of a text. So it's really, and I mean, I sort of nerd out on this. Maybe my students are like, oh, please no. But once they start to figure it out, they're like, oh, wow. And they can take that with them when they're reading a newspaper and they can read other texts that aren't actually texts, that also are arguments. So a podcast or something along those lines. It doesn't actually have to be a text text, but helping them learn how to, quote unquote, read for argument is really what I mean along those lines. And I could say more about that, but I don't wanna bore all your, all your watchers or those who are watching now. I'm just curious. When you do that though, when you say reading for argument, do you have like three or four things underneath that that you would normally say, like, these are some different ways to read for argument. Like you said, assumption. Like find assumption or find things like that. Yeah, so the first thing is that they can summarize an argument. So that actually, I think a lot of us just assume that that's the baseline and easy. They are terrible at that. I mean, and not they, we, we all are terrible at actually understanding somebody's argument and being able to summarize. So I don't actually think that's a low level skill if you're actually holding them accountable for really understanding the argument of what is being read. So that's one of my outcomes in my rubric is like, are you accurately understanding what reasons support what conclusions, et cetera? So, so here's what they say. Why do they say it? What are their reasons? That's like the, that's, I think the first step is, so it's not just summarizing literally, I'm quoting what they said, but why do you, why did they say that? Which other part of the text actually supports this part of the text, et cetera. And then what's not said that supports that argument. And then also, and this is for me part of reading is do you agree or not? Do you agree with their argument? And so in my rubric, they actually have to, they actually do evaluate the text. So it's all students are required to learn how to evaluate the text in the margins of their reading. So do you agree with this argument or not? And why, you know? So being able to say why. And then another one I often have them do because for me, synthesis is really important just in their life skills is how is what you've read here, whether you agree or disagree related to something else in your life or something else you've read elsewhere. And so they have to bring in a quote and that's the linkage I think that Christine was talking about. So those are the four very broad when I do hypothesis rubrics, those four things. But certainly if I were teaching like an intro critical thinking class or I taught intro writing before, it was very argument heavy. I spend much more time talking about the details of argument structure and doing assignments on those lines. But I think broadly, if they do those four things, if they can get good at seeing arguments, seeing what's missing from the arguments, being able to evaluate them well and connected to other things that they've read, I am super happy that they have, that's all that matters in my courses in many ways that they can do those things. Well, if they can do those things then I think we would probably have a different society. So I hope that that carries on beyond the classroom because that seems to be some of the essential problems in America right now that you're attacking. Christine, you've also talked about framing this in terms of reading and showing that book, right? You wanna talk more about types of reading, different specific sorts of literacy? I mean, I don't really engage in like different types of reading. When I show up the book, it's really meant to be an exercise and like students might not be able to like have physical texts in front of them to annotate right now. But one thing that's been on my mind that maybe loops into some of the things we've been talking about earlier is actually when I'm doing my workshop, I actually call attention to the practice of annotation and whether that is something that faculty members even talk about at the beginning of a class. Like I was thinking back to my own education and I don't know, like I can't remember actually ever being explicitly told, like I expect you to write on the margins of the text and to highlight and this is the types of comments that you should be making in the margins of the text. So I've actually been starting to include that as a point of like reflection among the faculty in my workshops of like, do you even discuss annotation with your students? Well, like regardless of the reading, like do you talk to them about engaging? How should they be engaging with the reading that's in front of them? Whether it's online or like printed out in front of them. So I think I, yeah, I just, I mean, Jeremiah, I'd be interested in your perspective as an English. Well, I mean, as anybody who's sitting on one of my webinars knows, I used to hand out Billy Collins's marginalia at the very class. So I made it pretty explicit even though I didn't have the visible-ness to it, but there was something I want to follow up with you about, well, yeah, what's out of 10 faculty members that you've talked to, how many have made a practice in the past? Do you think of being explicit about that? I mean, when I pitched that question to the group and I'm like looking at their Zoom videos, I do feel like they're pausing and like they're put on their thinking face, like, oh, so I don't exactly know, but just judging their reactions on the Zoom screen, I feel like it is something that they might not actually do in their classes. So I've just been trying to like, I don't know, again, like go back to basics, if you will, like what is it that we expect our students to be annotating in the margins of a text? Like what kinds of thoughts would be there? And again, this is where I talk about, well, we want them to make connections. Like this chapter reminded me of what we talked about in class last week, you know, or this chapter reminded me of the front page of an article in the New York Times today. So like just being, so yeah, so I just, I really go back to talking about like, what does it mean for students in your course to be annotating? Like what kinds of thoughts do you want them to be having in as they're doing the reading and then how can they make that explicit invisible to you and to each other? There is, I'm sorry I've been like scrambling like trying to find, I'm gonna post a bunch of links in the chat just in a second, but I think there are some disciplinary resources out there that for if you're talking with specifically in the humanities, of course, and not necessarily in others, but on how to read for history. One of my friends from Rice Kaleb McDaniel has a nice blog post on how to read for history. There are a couple of things in how to read for philosophy that colleagues have done. And actually one of the things I'm gonna share is not about reading, it was from earlier about how can I participate and it's like a hundred ways in which a student can participate in a conversation in a philosophy class and it totally applies to when you're reading and annotating in hypothesis. But there's also this like old 1950s book that is so outdated and terrible but I still love it by Mortimer Adler called How to Read a Book. And it has, and it just sort of, it's just this lovely thing that to me as a nerd who loves thinking about reading that I even had my students read sections of, like this is really important. There are different ways of reading and you're gonna in different disciplines they're gonna expect you to read differently. And in a philosophy class, I may ask you to read two pages but to spend two hours on those two pages in a history you're gonna read 300 pages in half an hour. And how do they, they don't, they've never been taught that. So hypothesis can be a real tool to help them learn those disciplinary differences of reading as well. It's just so exciting. And I think getting faculty to realize this is a potential for them to make their reading better which is what they often complain about is the students aren't reading well. And so if they're not doing something well maybe we should teach them how to do it better, right? So I'll post some of these links in here that you all can look at but I just wanted to say that too. Yeah, that'd be amazing. I would love to read those how to reads. And I've been thinking a lot about that and talking a lot about that with the faculty I'm talking to. I'm imagining that it's about time for us to wrap up and respect people's time to move on with the rest of the day. I have to admit, I don't know what I'm gonna do with the rest of the day because this is such a wonderful conversation. It's kind of can't be beat. I'm just gonna go back to something far less interesting for the rest of my afternoon. But I have enjoyed this thoroughly. I've learned a lot and I'm really thankful for both you guys sharing your ideas and I just hang in there. You're doing great work and be well. Yeah, and I wanna thank everyone. And I did have another question. It was actually my own but I'm just gonna throw it out there like maybe as an exercise people could have students when they're at home if they actually have any hard copy books at home they could pick one up and start to annotate it in it and see what the difference is between doing that alone in a vacuum and then coming back and doing that in the community online. So just an idea. But yes, thank you. This was an amazing show.