 Hello, and welcome to Liquid Margins. Our guest today, Remy Kahlir, is now going to say a few words, and then we'll turn it over to Jeremy Dean after that. Welcome, everyone. It's really a pleasure to have you join us again. My name is Remy Kahlir. Good afternoon, good morning, whenever you may be watching this. And again, thanks to at least the 44 people who have joined us live right now. It's a pleasure to join Hypothesis as their inaugural scholar in residence. I'm excited to talk about our initial projects, our vision. And I just want to thank my many friends and colleagues who I've been working with at Hypothesis for a number of years now for creating an opportunity to advance consequential research that can really support both educators and of course students, particularly in the current context that we're in. And so I'll leave it there for the moment. It's good to be in conversation with Jeremy and all of you through the chat. Welcome to our webinar this morning. Hey Remy. How you doing? You know, we're here. We're in it. When's the last time you went to campus? I picked up some books actually. I picked up some books about annotation, but you're all right here in August and that will probably be the only time I'll physically be on campus most of this academic year. Well, that's a nice nice nice real scholarly background that you have there. I feel like this could be like the CNN interview with the scholar, public scholar. Nice to talk to you today. It's been a long journey, man. We've been in conversation for for years now and I was trying to reflect and maybe I have a terrible memory if you can help me remember like, when did we first connect what was the context. I know it was early on in my time at Hypothesis I barely started I believe. So I began at the University of Colorado in Denver in 2015 and I think that just sometime in my first year there, maybe like the very early ish 2016 maybe you and I were in some kind of webinar together maybe it was a very educator focused conversation. It might have been something like the teachers teaching teacher session or something maybe through the National Writing Project. In any case, I think we both realized that we had a keen interest in collaborative learning, and how people read, write and make sense of texts and ideas together. And pretty shortly after that conversation we said we should get in touch and begin those conversations. That was also that I guess spring winter spring semester of 2016 was the first time that I began to really experiment as an educator with Hypothesis. And that really began our journey together it's been a very productive and pretty, pretty robust four years now. Yeah, and so first Hypothesis was of interest to you for teaching. Absolutely. I mean not immediately for research and can you talk about the sort of first experiences with using in the classroom. Absolutely, you know I began, you know, like I think many of us as someone who was curious about learning technologies that's part of my disciplinary training but also for many people in higher ed, where and how I was teaching I was teaching online or I was teaching hybrid classes. And as is typical of many courses, students are reading together, they're making sense of texts together. And in my training as a learning scientist and in my training as somebody who designs and implements curricula, particularly for other educators. I was very curious about how a medium like Hypothesis a tool like that the practices around social annotation could augment that work. And as I began to experiment again in my own practice. I began to also read literature about social annotation and collaborative annotation there's of course a robust literature that we might talk about today. I began to think creatively and I said also think pretty critically about how this could be incorporated into a variety of course designs, and pretty quickly, my interests in Hypothesis specifically but social annotation more generally, kind of translated into some research work and we actually began our first kind of informal pilot research project, pretty early on in 2016 there was a National Science Foundation fellowship that was specifically looking at how to use learning analytics in online, and kind of digital rich environments and I was named one of those fellows, and I began kind of my first then scholarly interests in researching how our learners in digital online spaces, marking up texts, applying to one another, talking to each other through social annotation. And then how do we begin to understand patterns and discourse and knowledge construction in those kinds of processes. That's great and at that time, what did the landscape of the research look like I mean when you when you started thinking about these, what was out well. You know it's interesting Jeremy because I don't think I was as aware of the breadth of the literature at that time in maybe 2016 or 17 I'm certainly a little bit more fluid in that in that landscape of literature now. I can appreciate. There has been decades now actually 20 plus years of research on digital annotation or annotation occurring in online web based computer supported spaces, and notably enough hypothesis is perhaps the tool that has been researched the within just the last four or five years, Nate Angel are you're of course your hypothesis colleague you know our dear friend recently sent me an article from 2016, where an educator was using hypothesis in their class, surveying their students, and sharing information about how individuals are collaborating, which is again also say that there's now been quite a few years worth of pretty dedicated research across disciplines really interdisciplinary research about how hypothesis as a form of social annotation does support student learning. But again, there's really a decades now of social annotation research research about this type of collaborative learning in online spaces but no tool in my reading of the literature has actually been used as extensively, and now researched as extensively as as hypothesis hasn't I think that's really a testament to the organization, a commitment to open technologies, the kind of flexibility with which now hypothesis can be used across a variety of learning environments, and then just some of those social and technical affordances that really do productively aid students and their learning. So a little bit about in terms of some of the research that's out there, what kind of questions people are asking, what kind of angles they're taking in that research. Some of the conclusions if they seem that they become, I don't know, sort of ratified as claims that we can make about social annotation. Absolutely. So there's been a lot of research about how students perceive how they value the use of these tools. And so whether it again it's hypothesis or a variety of other social annotation tools, some of which are created for a single research project and then kind of again, researchers are very curious about how do students like us find, you know, comfort, you know, comfort in using these kinds of tools and so that for me broadly falls under a category of student perceptions there's a lot of research about student perceptions of uses uses of social annotation and I will say that I've also contributed with a dear friend and colleague, while Pablo Alperin and his, his research lab at Simon Fraser University up in Vancouver. Some of that research and there are some benefits and there are some drawbacks to how those student perception surveys are, are designed that is to say though that largely students do perceive value in their use of these kinds of tools. They like using social annotation, they find it aids their learning in productive ways. It helps them for example to see other people's perspectives to share resources to interpret or elaborate on certain points in a text again whatever that particular text may be. And again it can and there is some evidence that social annotation can also contribute productively to a sense of community a sense of feeling connected to peers in a class so that's one very large body of research about social annotation. Actually there's another really interesting body of research about social annotation that concerns language learning and things like reading comprehension again for centuries annotation even when written by hand has been used to translate words, provide an interpretation of a word or an explanation of some term. And of course, social annotation tools have adapted those century old practices and there's actually quite a lot of literature in the language learning space around the use of digital annotation tools to support student learning sense a whole other area of research that's pretty pretty exciting and productive. I would also say that there's another really robust area of literature here around how students use social annotation to then either construct their own knowledge by making knowledge from their conversations with one other or or make meaning. And there's some, there's a little bit of a synonymous interpretation there between meaning making and knowledge construction and I don't want to necessarily get too caught up in the weeds of that but a lot of researchers again across disciplines have studied how students together. When they're reading a text and then discussing that text through annotation, they're learning from one another's interpretations, they're extending lines of logic, they're working through perhaps conflict, they're finding consensus and negotiating around key points or key ideas and all be seen as knowledge construction practices that students engage in when they read together and then also when they annotate together that's another area of social annotation research. I've rambled a bit so I just want to say one more thing and then I'll just kind of, you know, say that there's actually been less research and I think this is a real opportunity, particularly for hypothesis and others to really kind of continue to leave the hypothesis, there's actually been less research connecting what educators are doing with what students then learn through annotation. There have been some nice kind of descriptive cases of educators actually particularly in K 12 settings around how they may design a high school English class to support annotation practices. There have been some useful kind of descriptions of educator practice, but from an empirical perspective there's actually a very small and really I would say underdeveloped research literature about saying, here's how as an educator, I'm designing for collaborative and social annotation. Here then are the kinds of activities and practices that students are participating in, and then pairing very intentional pedagogy, particularly in digital and online spaces, with now what we know to be the productive student learning practices that are supported by social annotation. That's really cool. I mean, I think actually over the past few weeks we've, we've talked on this topic, and I hadn't totally gotten what you were talking about until just now, and I really appreciate it in terms of, you know, obviously there's the one question of is, is social annotation efficacious, in various ways, but I love the focus on the practical in that last category, especially as it's something we might be working on together in the coming months and years, in terms of really focusing on what are the types of activities because that's just something that, even from a claim, you can hand off to an instructor and say, here are some best practices in terms of actually doing this. I have to ask you one question though, because it's come up and we haven't had a chance to fully have this argument yet but like social annotation, or collaborative annotation you prefer social annotation I do become in the habit of using collaborative annotation but of course I'm the vendor and you're the scholar so why is it social annotation and why should it be or what's the difference. So that's a great question Jeremy, I see collaboration or collaborative learning as a description of practice. Many things can be collaborative learning right and there are many ways that we can participate in collaborative learning, whether we're building with Legos, we're solving a math problem together, we're reading a text together. There are many ways that I might understand how learners collaborate with one another, and also how an educator and a learner might collaborate in a variety of ways. And to me, collaborative learning references from the research perspective, some important work that's been done over quite a few decades now around what can be called computer supported collaborative learning and some really important advances and how we understand the nature of collaboration, particularly in digital and online spaces as supported by things like computer technology, and that is even as distinct from cooperative tasks, or other kinds of individual uses of technology. So I approach collaborative learning in a particular way of understanding practice and what learners are doing together. That being said, now I also understand social annotation as a specific, we might call it a genre of learning technology. And here's why I understand social annotation as a genre of learning technology. There has been great precedent in annotation, of course, being a technology that's been associated with, of course, manuscripts, books, again, all kinds of web browsers, you know the very first web browser had annotation built into it right, and there are of course many social technologies. Again, hypothesis really I think being the vanguard of how one designs and develops and shares an open social annotation technology but again there are others that have been used in a variety of spaces and including education contexts. And so I understand, and I think that the the literature is really kind of coming together at least in the last few years to use the term social annotation as the descriptor of the genre, irrespective of the tool that's being used and I would give an example there was a paper that was just published this year in the book, Computers and Education, by a team from a university in Hong Kong, and they designed their own, you know, entirely unique digital social annotation tool. And in their description of this technology, they actually give it an abbreviation so and social annotation, which is interesting because there's a pretty broad literature now they're using social annotation just know the letters SA as the abbreviations all just to say that there's a kind of a shorthand now of social annotation being referred to as this kind of genre that is specific to certain technologies. Another really nice current lit review in the journal information and learning sciences has a nice again review of some social annotation practices and technologies, looking back over the last few years of research. So that's how I distinguish social annotation. To me, it's a genre. It's associated with specific technologies, and then it enables collaborative learning, and it enables a whole repertoire of collaborative learning. Again, maybe in an online context in a hybrid context with this type of learner with this type of teaching practice, etc. But we can continue to debate that Jeremy if you want to keep going back and forth. So but is it is it then possible for there to be something that is a social annotation but not necessarily collaborative. I think that's interesting right the questions like how does one begin to define collaborative learning in a way that is then useful in regards to my own research, for example, I looked last year at an example of hypothesis annotation to try and better understand the context of collaboration, particularly in the context of open learning so this is a study that some of my grad students and I analyze data, wrote it up. And in short, there's been I think some ambiguity around for example what does collaboration mean, what does collaborative learning mean in the context of something like open education and open learning that term collaboration often gets thrown around as a descriptor. I think it's more of an adjective, and it's used to really specify particular practices. And so I began to identify these very specific group level. Again, kind of group level epistemic kind of knowledge oriented moves that social annotation enabled. And so I don't want to suggest that again it's always synonymous, but I think that it's very useful as you're suggesting Jeremy to be very careful, particularly for talking about research. And to be able to create really empirical claims to say, if we're saying that social annotation does enable collaborative learning. What are those qualities of collaborative learning what are those particular practices of collaborative learning, and what are those look like particularly among groups of learners. That's great. Thanks for that Rami. So this is very exciting that you are now hypothesis is first scholar in residence. And I do want to just give a shout out because we've always worked collaboratively with a lot of other folks and there's a great group of people who are attending today who many are doing their own research. And this role is meant to, you know, enable all of us as a community of folks interested in this space to work together and build together but tell us a little bit about this scholar in residence program and and what your your goals are for for the program. Well, again, I just I really want to thank hypothesis for, you know, as an as an organization supporting this new emergent and to some degree still being defined, you know, adventure that being said I want to make it very clear that, you know, I've been involved in a variety of initiatives and research practice partnerships over the years and I think it's really important to emphasize that that positions should not be associated with an individual. I'm honored to be the inaugural scholar in residence, and I fully anticipate that in future academic years. Other scholars from a variety of disciplines will sit, maybe not in this particular chair, but in their own share in their own residents. However, this current context shakes out and help to lead this work for it and so actually that helps to define part of my role now. One of the big goals for my involvement as the inaugural scholar in residence is to help hypothesis develop research capacity that then will support other scholars, other universities and other programs and advancing their work. I just want to be really clear that this is not Rami's agenda. This is very much a collaboration that has allowing hypothesis as an organization to now not only be I think believe the vanguard technology is enabling student learning with social annotation. It's a vanguard organization that is pushing the field forward the field of research about how social annotation in a variety of forms and in a variety of learning environments can support students and educators and institutions. And so I will see it as a mark of success in this year. If I can better facilitate the future research of other scholars. So what does that mean that means doing a few things this year. For example, surveying the field and getting a strong sense of like what's the state of the art in social annotation and so one of the concrete goals that we have in the beginning of this year this this academic year is to publish a white paper that really brings together and a concise, easily understood, and a very kind of like audience friendly way for a variety of communities to communicate. What do we know about social annotation and how is it really supporting student success. And that will be a very concrete deliverable that we will be able to share, again with a variety of stakeholders to help, again, educators, the research community technology leaders to really make further investments, whether that is an investment in a research project, or an investment in courses and teaching to use this these these technologies and social practices that's one thing. And part of my job in this scholar residence capacity is to help other scholars and so that means developing things like an easily accessible knowledge base of research about social annotation, where my other researchers turned to to get easy access to many, many resources pertinent to social annotation were, for example, putting together right now and open Zotero bibliography so that anybody else who's writing about social annotation can go there, figure out what they need to know in terms of the literature, figure out how their study perhaps relates to the broader field and use that as again an easy scaffold for their own scholarly work. We're going to be putting together some publicly accessible research instruments, these may be survey tools, these may be interview tools, these may be again things that other researchers can essentially pick up, and then bring into their context as a way to again create more robust, usefully empirically grounded research, and then some consistency among the field. So that's going to be another thing that we can do. And then the others to then begin to initiate and lead research projects with partner institutions and of course, I don't think it's going to be a surprise about the 52 people, at least on our webinar right now, that there is an incredibly robust annotated coalition of institutions K 12 and higher ed, who are using social annotation in their courses in some way. And we look forward and really this is an invitation to say, if you're a researcher at annotated partner institutions. And if you come from whatever disciplinary background, whatever field if you're curious about whatever particular theoretical orientations or methodologies, whatever your curiosities may be about social annotation and learning. How can we hypothesis how can we through this new research initiative, support you and your work. Let's begin that conversation and let's develop the tools and the processes to make that research possible. That was awesome. Especially that last part I know I have a backlog of people who have been asking me, I think I sent you something this morning where somebody was asking for a list of research. And I was talking to somebody on the phone yesterday who said, you know, we're interested as we continue using hypothesis to start doing some research. And I said, Well, we have something in place to help you out there. I do think they can correct me if I'm wrong, but do you think you need to put a little emphasis on the ed. So it's annotate ed, it's not like annotated. They got anti ed to sort of anyway, we can investigate that later, but one of the things I really appreciate about the way that you and Nate and all of us have put this together and I think it's largely been led by, by your philosophy in this part, is the way that the research program that hypothesis is set up with. I guess some ethical guard rails, or just the real attention to the ethics of research around transparency and openness can you just talk a little bit about some of the values that you try to you're trying to instill from the start about how this research should be done in a way that is most helpful to students, teachers, the public. I thought that was a really cool aspect of the program and is articulated in the in the blog that people can can go and check out as well. Absolutely. Again, I see that Nate's sharing in the chat now I would highly recommend it people you know look into that you know I guess I could start this from a few different perspectives I could say one could consider that there has been ongoing sometimes controversial conversation around how technology organizations engage in research, maybe it's exploitative, maybe it's not partner driven, maybe it really doesn't respect again the privacy of students for example that that's a whole conversation that some people may be familiar with. I could also approach this conversation by saying that, again, I'm a former classroom teacher, and I still teach day in and day out, I have incredible students some of them are actually I believe in the webinar right now. And as an educator, I would never want to in any way compromise my relationship to students particularly in the context of research. And I very much value those types of relationships and so again when one introduces research into teaching learning context, one needs to be very clear about how one does that again in an ethical and a kind of partner driven way. And I think that again that will resonate with many, many researchers. I could also to appreciate or excuse me approach this conversation by saying that I officially wear a hat as my School of Education's representative on my university's social sciences IRB committee. I am the CU Denver School of that representative on the University of Colorado's panel s. So I've been trained pretty extensively and how to engage in ethical human subjects research that respects all participants that minimizes risk that really make sure that if we are engaging even if they're over the age of 18 even if they're of course adults that we do so in ways that really do ensure the highest standards for for research and so those are three different ways that I could approach this conversation. All of which is to say that again hypothesis as an organization has always been committed to and again if folks on the webinar now are not familiar with this I would also highly encourage you to check out the kind of core values that has always grounded hypothesis as an organization but notions of transparency notions of integrity have always been very core to the organization and so it strikes me as entirely consistent then with the vision of hypothesis to then continue to engage now in formal research around social annotation in whether it's K-12 or university contexts that abides by the highest standards for ethical human subjects research and then of course that is also driven by the needs of partners by the opinions and the expertise of those partners and that really responds to local problems of practice. And again that vision is articulated in our early our research vision that we published earlier this week. I appreciate that Rami and I think it's true that Dan Whaley and everybody from the started hypothesis has sort of really tried to be very aware of ethics as we move forward in this space but it's also true that it comes from partners keeping us in check reminding us of those values. And so I thank you for that I think the blog post and just your approach to this partnership and the project that's before us is incredibly helpful in that regard. Well Jeremy you're thankful but I would also want to emphasize that you know not everybody and not every organization can take this term I should say and I don't want to kind of go off the rails too much here but I will say that I think that we're seeing right now in this particular moment for all kinds of reasons a lot of very cold like values watching where organizations are making claims about their work that perhaps don't align with previous actions and I'll just leave it there and people can read into that what they want to read into that. Again, for an organization like hypothesis that has a very clear track record from its founding about what it values in terms of again the development of technology, the sharing of knowledge. It just I think again it extends that kind of lineage of work to then conduct research in this way, and I would also add to that that another core commitment to our future research is one of open scholarship and open research. And again, our research vision document our blog post has that articulated but I think that perhaps some people who at least I can I can see on our on our attendees list are pretty familiar with processes that support the open production of knowledge. Aspects for example of dissemination of open data if possible or open research tools or certainly making accessible open publications, and we are very much in this partnership, aligning our work to supporting those open scholarly and research processes. Thanks for me. I think with that I want to because we have such a distinguished crew of attendees. And there's been a great conversation in the chat that I'm sure Nate and Franny are moderating. I sort of want to open it up to others to share their own work or ask Rami questions. And I guess I'll just say like well those questions and comments come in I am just so personally just honored that that I recognize so many friends and colleagues in the attendees list today that's really to me. There are a few things that bring smiles to my face at least these days. And I just really want to say, if you can hear me and you can watch this and you know me. Thank you so much for joining us that really means that means a lot to this whole project and process. I'm going to, I did collect a bunch of questions for the chat from the chat, really great great questions and a couple from the Q&A and so I'm just going to ask you as many of those as we can get to in the time we have remaining and if we go over and folks can hang out and Rami you can hang out. I'm hanging. I'm here. This is great. Okay, great. My question is from and just, I'm sorry if I get anyone's name wrong, but Phillip Piety. That's how I'm going to pronounce it. How much of social annotation research involves web web artifacts, compared to traditional texts like textbooks. That's a nice question, because one could suggest that some flavors of social annotation have existed, you know, in print books, you know, in, you know, in Victorian England, you know, people would write in books and then pass them along to their neighbors, as a way of sharing thoughts with each other and that was of course happening hundreds of years ago. So, you know, one could interpret social annotation a variety of ways, specific to our work in this research agenda. Much of the social annotation research is of course social annotation of digital resources. And that's a key distinguishing feature of how one might define that again genre of learning technology. Think of social annotation as a genre, whatever that tool may be. We're talking about the annotation of open textbooks, or of blog posts, or of course scholarly articles or book chapters and so predominantly then because it sounds like the question was how much of predominantly the social annotation research over the last few decades has looked at then the annotation of and the interactions or the conversations associated with documents that are living online documents that are digital in some way. I hope that's a useful response. I'm picturing like that was, you know, the prototype for texting your neighbor, right in the book and get it. That's right. Oh, absolutely. And there's really well documented history that it's pretty fascinating actually. It's really great. Okay. Next question is from Johnny already is web annotation from social annotation. That's a really good point Johnny I would actually say that yes web annotation for me as a slightly even broader umbrella, because there have been some pretty notable examples of web annotation. And again as I mentioned briefly the first browser mosaic had web annotation built in so that readers of whatever folks are reading on their browser. And decades ago now could annotate the web. And there again actually there's a incredible list and I think that hypothesis may still maintain it I've at least seen it a few times of this incredible lineage of web annotation technologies. And so in this case we're talking specifically about technologies that are not only enabling an annotation of a document on the web, but then these social and interactive practices. They're allowing for things like a reply for one person to another. There's aspects for example of perhaps resource resource sharing, and there's, again in the context particularly of teaching and learning ways in which these social annotation activities are designed and facilitated for social engagement, and for students to learn from one another and to learn from their texts that's a really important point I really actually, first of all, great for that. Thank you so much. It's great to have you here. Thank you so much for the question. Yes. I'm thinking about again a kind of genre of social annotation technologies, while understanding that there is a very rich and long lineage now of technologies that can essentially mark up the web. Can I follow up with a question that from my former colleague and friend Jim Brown in the Q&A here. You mentioned the history of annotation in the attempt it relates to this conversation about web annotation. You mentioned the history of annotation in the attempt to build annotation into mosaic. How much research do you see happening about the history of web annotation and how contemporary tools fit with that history and I would add or don't fit with that history. Yeah, that's a that's a great question actually. You know, there hasn't been. Let me. Yeah, there's quite a bit there. There's been some really notable examples of, I would say, not not an empirical study, but perhaps scholarship from a variety of disciplines about forms of digital annotation or annotation on the web. And kind of turning around and taking a book off my shelf here. There's this great book called book was there that was written that has really lovely examples of like early web annotation and kind of digital forms of annotation that were maybe not only social in any respect. There's not been as much of that research, honestly. And, you know, much of the research is again driven by uses in particular in this case teaching learning contexts, which leads to more grounding in the social annotation space. I hope that I hope that makes sense. I want to add one thing to that, which I think is an interesting way that I've talked about this with some users who sometimes grapple, struggle to understand how hypothesis works, especially with the browser extension. And I don't know if it totally relates Jim, but it's a difference between, you know, annotation as a tool, web annotation as a tool or platform, something that's a destination that you have to go to, you know, it contains the text it contains the library on the text, or some tool that contains functionality and is more distributed in nature, and can be used in various platforms or locations across across the web, but I would love to see research from Jim in that regard. Great digital scholar there for any any other questions that you wanted to surface. Yeah, there are a bunch more actually. This is from Karen LeBonte. And again, I'm sorry if I mispronounce your last name. Does collaborative annotation imply articulated shared purpose. So I, I, I'd like to actually hear more from Karen about what what she may mean by articulated shared purpose. I'll answer from a few again a few different perspectives. Many, let me see this as an educator for example I would not maybe have my students just open up a text and start reading it. Why are we reading this text, you know, what's the purpose are we studying rhetorical moves we're looking at about how do we use or we're learning about disciplinary methods. Again, I think in many course contexts, for example, professors provide a reason for why a certain text is read and then a reason for why students might discuss that text including the fact that maybe not all texts need to be socially annotated. Some may, some may not. So my hope actually maybe more of a hope is that when students and teachers when individuals engage in social annotation practices they do so with a particular purpose in mind, and that that type of engagement with the text and engagement among readers is done so for a particular reason. Again, that reason may be meaning making that reason may be critique. So that's helpful but I'd like to I'd actually I'd like to know a little bit more about what that Karen how you are approaching understanding that that concept and I would also say that there are some other really amazing, again, colleagues friends and researchers, you know, in the webinar today who may have some other really useful things to say about that. I think in the chat. Karen, so that you answered it. So she says, Oh, you've done it. So, I think that means that's answered. This next question is from Bodong Chen, who also wants to say thanks to the inspiring work Rami, and so excited about this partnership. That's great. Okay, and then asked, What are your thoughts on possible roles social annotation can play informal classrooms in this post truth era we are in. That's an awesome question. I'd love to hear the answer to that. Yeah, that's a that's a really. So, so great to have you here. Thanks so much. I appreciate comments of course. Some folks know that my dear friend and colleague on tarot Garcia and I have a forthcoming book titled annotation coming from the MIT press. We addressed the challenges of the post truth moment in the conclusion of our book. And we talk a bit about how annotation can be leveraged for social good. And I don't want to suggest in any way that annotation is a silver bullet and in fact we say so. There are a number of initiatives now, particularly in civic media spaces that are beginning to identify some useful strategies around fact checking around the ways in which people can engage productively in conversation. And have also in some cases actually engaged in things like crowdsourced legislation, developing ways to actually have civic conversations together, and then inform actual civic change and we again we articulate a few of those examples in our book. So I don't in any way, particularly because so much of what can occur particularly in the social media space is of course, actually quite dangerous and misinformation disinformation or rather right. I do not want to in any way suggest that somehow social annotation is going to, you know, create a more robust democracy. I do want to suggest though that that social annotation practices in certain context can contribute and may be useful as a kind of broader toolkit in finding ways of perhaps making more viable. Very useful conversations and again that might come from media fact checking, and we already see that in the media. Again, whether or not you may trust source a versus source be after many large speeches or events, transcripts of comments from politicians who are often annotated as a form of fact checking, which is for me one example of the way in which there's some promise around social annotation contributing to a more again informed, you know, again productive public discourse. But I don't want to again pin the hopes of a more of our democracy on social annotation. It's a tool, and it can be useful I hope I'm that's helpful in this in this current context. I just say as a, well, I don't know if it's a follow up question or not, but I mean, I think about how it in this is not proven but a lot of people put this out there that it increases critical thinking skills. So I think that would be a subset of what you're saying, probably, but can you talk a little bit about that. Absolutely. And I think that we've seen some really, you know, useful examples of the annotation and this perhaps is where there's a useful distinction back to Johnny's question about these useful between web annotation, how people can mark up the web and have access to that, as opposed to social annotation where maybe everybody's invited to jump into the conversation. You are seeing again media outlets, pretty in my assessment reputable news organizations who annotate again transcripts documents legislation, where multiple experts are sharing for example, the evidence that backs up a claim. They're sharing related legislation or resources, their fact checking by pulling in other quotes or statements, but they're making available through their expert opinion that that information and this is where reading expert annotation actually is very productive for learning. It may also be productive in a civic space, but there is an important difference between someone participating in social annotation and someone who has access to expert annotation of a text. Both can aid people's learning, and both can really be useful. And so I think we are seeing now in domains beyond formal schooling, experts who are making their annotations available to help inform publics, for example. That's a great example of using digital resources and annotating them on the web and having access to that, compared to social annotation that maybe anybody can join or a class may be doing at some type of an activity. I hope that's helpful for any. Yeah, definitely. Thank you. This was, this is from Kristen Whitmore. And she wants to know what you believe is the biggest advantage to social annotation and then what is still slash, what is still slash if there is one, the biggest challenge. That's awesome. So really I'll see Kristen's current current student of mine and so maybe one of the biggest benefits is that that kind of curiosity leads to these kinds of connections. So here's, here's one of the things I really value Kristen and others is that social annotation makes thinking visible. And my thinking is not always precise, my thinking can be flawed, my thinking can have blind spots of biases my thinking may simply be wrong, for example. And so one of the things that I really value about social annotations that makes people thinking visible to others. It can lead to all kinds of potential change all kinds of aspects again of learning where Kristen if I have access to your thinking, your ideas may challenge mine. If I have a serious blind spot in regards to my understanding of an idea or some other real bias. You may be able to notice that in my rough draft thinking made visible by social annotation, and then you can engage me in a conversation around that. And again, I know there are a lot of actually like learning scientists in the call right now and a lot of folks with some pretty serious training on learning. When I think about notions of distributed cognition of cognition that is stretched across multiple texts, multiple people, and that really exists beyond the mind of Ramey. So my thinking limited in my head is really rather insufficient when compared to the thinking that exists when I'm having a conversation with you Kristen and with Johnny and Bodong and Jim and many of the Karen and all the other folks were chiming in now. Our thinking together has far more potential to be consequential particularly teaching learning context and social annotation makes that distributed cognition, viable and pliable, and it allows us to work with one another, and our texts in very consequential ways. And that's one of the real exciting things about social annotation. As for its challenges, many of the challenges that currently exist with, for example, aspects of talking online. Do we know if individuals are present, how much might people participate, how do we invite people to share those thoughts, even if it's in a private space or a semi public space. There was a related Twitter conversation last week about what makes for really effective online discussion and particularly in the context of things like conventional learning management system discussion boards and people talked a lot about aspects of access and equity, who participates whose voices are welcome whose voices may be hidden, even in asynchronous online discussion. And I think that many of those dynamics are also present in social annotation. If people feel comfortable having a presence in this kind of medium, can they share in a way that reflects again, their rough draft thinking, you know with others that can be a real challenge from a teaching learning standpoint Kristen, and I hope that that's that's useful. Again, thank you so much for coming. It's really, really a pleasure to have you here today. Yeah, and I just want to say, we are past the end point, but we'll keep going, but I want to invite people invite people to leave it's not anymore on but you know if you have to go go but if you'd love to stay stay we'll answer a few more questions and we'll just keep going and the recording will be available either later today or on Monday. So look for that on our website and because you were here today you will also get an email from us with a link to the recording and to other artifacts from the show. And I just want to for those of you who do need to leave. Thank you so much for being here and everybody else. Thank you for being here to and again, hang out if you'd like. So there are a few more questions but I think I'll do my, my bookend here, and just say thank you to everyone for joining liquid margins today. And thank you to our guests, Rami, and thank you to Jeremy Dean for moderating. I just want to add one more thing which is as Rami mentioned before. This is the beginning of something. Rami is our first scholar in residence and part of Rami's work as a scholar in residence is going to be support other scholars working on annotation, and we're going to be supporting projects and supporting other individuals and groups doing research. So, for everybody to join today, you know please be in touch. Please reach out to us. We want to start having conversations and collaborations, because this is a program that is now launched, and we want to support all of you all in this space. Yeah, absolutely. Yeah, but I'm happy to hang out so let's just again keep the conversations going. So, this one is from Marty Lu. Are there subgenres of media manuscript video annotation with particular practices and forms that are discipline or domain specific? It's so it's so nice to have you here and to reconnect. You know there has been some really important, you know research around discipline specific uses of social annotation. And again, you might turn to for example composition and literacy studies has always been a home for annotation. We're also seeing some I think really important studies that have come out of the sciences now. We're seeing some really useful work around how, for example, the annotation of primary source scientific literature through a project like the science in the classroom initiative has really strengthened for example undergraduate students learning of discipline specific methods vocabulary, etc. And so I absolutely actually as part of this research effort. I'm hoping that scholars with deep expertise and their respective disciplines are eager to find ways of applying social annotation insights and practices to their course contexts to discipline specific projects. And I'm not doing that for. So that's my, that's my brief answer there but I'm, I'm actually very much heartened by discipline specific research and uses of social annotation, particularly now in online learning contexts. It kind of seems like there's really any discipline that can be. They can use social annotation, I think, even math, I mean the math show. Absolutely. Yes, absolutely. I just call him Johnny. What I just call him Johnny now because I've seen his name sometimes I feel like we know each other. What advice would you give to a teacher who struggles to get their students to engage in uptake to really explore ideas together. What structure would you give to social annotation activities to avoid the isolated one off annotations to which nobody replies. Well, so, you know, here's the thing Johnny, you know, I've got things that have worked in my teaching and in my courses and get having one of my current doc students Kristen and Nicole maybe she should answer this question and see what she says. I also want to say that my own pedagogical preferences may not work for all educators and so, you know, I could say things like well maybe you shouldn't great annotations or maybe you should specify how many or maybe you should see the text with these kinds of questions. I actually would want to say Johnny is there has been an incredibly robust archive of these liquid margins webinars and if I'm not mistaken, this is episode 11 and the 10, that's the case this the 10 prior episodes. There's some really again even disciplines specific examples of educators who are really I think honing their craft finding ways of engaging their students and so I guess this is my, you know, LeVar Burton, you know, you don't have to take my word for a moment to say, some things that have worked for me over the past number of years of social annotation when it comes to engaging students but again it might not work for you or in your course context and so maybe finding an educator whose course design or whose discipline is a little bit more aligned with maybe your particular classroom context may be able to provide those really practical strategies that are going to help your students to fully embrace and engage with social annotation. That's a whole community of us. Certainly is it's growing. Do you want to keep going with a couple more questions. Oh, I'm great I love this and the fact that folks are still around is fantastic let's just keep rolling and people can keep chiming in and yeah absolutely great. This is from Terry Elliott. It's an easy one to run. Social institutions and behaviors are arising from the use of hypothesis for example what social expectations are raised by reciprocation as an institutional value. Great to have you here Terry and thanks. Thanks so much for joining you know I'll just speak from my perspective in regards to researchers to say that there are again now I can say over 100 hundred make you know quite a few institutions that are part of this Ed they get that right Jeremy make annotate Ed coalition. And when it comes to then research we really want our research initiatives to be driven by partner driven collaboration and commitments so that means that again I don't have like a Back pocket you know like set of burning research questions that I would like to research in context a or in context to be rather as we move these collaborative research projects forward. We really want to listen to researchers at a particular institution we want to listen to the problems of practice there that are germane to a particular set of classrooms we want to know how social annotation is or is not perhaps working effectively in this context and then from those institutional and contextual dynamics we will shape our research to be again responsive and of course respectful to those particular research partnerships. And so I just want to be really clear that when it comes to this new again research vision and you know the forthcoming projects, we really want to be responsive to context and to partnership as a guiding values. Excellent. And then here's another one from Philip piety. When social annotation, when are you left out a word that when our social annotation tools. Wait a minute, when social annotation tools are used for assessment. So I think when are they used for assessment. That's a great question you know I've actually been going back and forth a little bit with someone who's who's still on right now and said I actually recommend that you connect with Daniel Hickey he's a professor and learning scientist at Indiana University with deep expertise in assessment and has really been developing I think some pretty important insights about various assessment practices and then the particular role that social annotation may use so the first one I would recommend is from this conversation, you know, connect with Dan, and we can facilitate that connection if that's welcome and useful as well. But I would say this, you know, even in my own teaching but also reading of the literature. First, I would want to be curious about what aspects of assessment one is for example looking at there are for example in my courses and I've there are some important examples of this peer to peer feedback or peer to peer assessment has been kind of used alongside social annotation. And so of course having peers read one another's work, comment upon it, provide aspects of feedback to improve the quality of writing. Again, whether that's in argumentation use of evidence, whatever that may be rhetorical moves social annotation can again help to facilitate some of those practices. There's been less work that is, for example, looking at social annotation as a form of like summative assessment. But again, that's perhaps an open area of inquiry that could certainly be looked at. But I think that, again, I would want to begin that conversation by getting a sense of what are the assessment, perhaps practices and desired outcomes. And then what is then the role of social annotation in supporting that process. Yeah, and thank you Johnny senior in the chat there and definitely connect with Dan, he could certainly deepen the conversation I think some really useful ways. It looks like we've got two more. This one is from Karen borowski, and will be introducing hypothesis very soon and I'm really excited about it yay. So what's your difference in how effective annotation is with undergrads versus grads. And then he also wants to know that first year students in particular. Yeah, that's a good question. You know, the first thing I would do is I would direct you to looking at, again, there's some important literature about undergraduate learning and graduate learning although studies are often not. I've not come across a study yet, although again if someone's aware that please send it my way that has compared those two. There is quite a bit of research that concerns graduate students for a variety of reasons, again a lot of research around social notation at least recently has been done in teacher education context and so grad students in those courses social annotation is being used to study that there of course is certainly an important research being done in, you know, at the undergraduate level. I was again really honored to join. Again, want to follow up rent and his colleagues up at Simon Frazier and we just did a ongoing set of studies but the first papers out that looks at undergraduate students and their again perceptions of social notation for learning and so, you know, saying that's more or less effective in undergraduates versus graduate students. I don't want to say that it's more effective for Group A and it's less effective for Group B. I think that studies that involve graduate students and studies that involve undergraduate graduate students both show that social notation does again productively aid learning again perhaps in slightly different ways given again maybe the context of the class. But if you're working specifically with, as you say first year students, I think there's a lot that a lot of research that I would say that does support. Broadly speaking the use of social notation with, for example, college freshmen. The questions just keep coming so. So this is from Juan Pablo Alperin. What do you see as the role of social annotation for encouraging better civil discourse kind of touched on that already but is there a secondary goal of social annotation with the primary being learning and community building, or is the learning secondary and better civil civil discourse the primary. That's awesome one I first of all great to have you here and so so much appreciate the question. You know this is where I see to back to an earlier response social annotation as a genre, and it enables a whole variety of practices again for many of us as educators as researchers of teaching and learning. We can say that social annotation does enable productive collaborative learning. Social annotation of course enables many, many other things. And so we could point to, for example, a project like climate feedback, which if folks are unfamiliar is a group of scientists volunteer scientists from around the world, who essentially pure review the online journalism is published concerning climate change. These scientists will go in and they will publicly annotate using social annotation tools like actually hypothesis specifically these news media reports, and then essentially provide that reporting with a great, you know, with an evaluation of how scientifically valid the reporting is. There have been some really interesting outcomes from their work and so I think that's a great example of seeing social annotation, enabling more scientifically accurate journalism, which of course we might say has a broad kind of social good. And this is why I and actually I should also mention briefly that there's a new related effort come right now coming out around coven 19 research and reporting as well with scientists also publicly annotating for more informed early civic and public discourse research about coven 19 in our current pandemic. So that's to say one and others that I am. I've always been really curious about social annotation because it can afford so many of these kinds of practices in again a whole host of domains. Perhaps I see it as maybe primary because of my work in education and as an educational researcher and as somebody who wants to design for and research learning, but maybe a climate scientist volunteer who peer reviews the news would say that the primary purpose of social annotation is to hold journalists accountable to, you know, a high standard of journalism in order to, you know, make reporting about climate change, you know, more informed and effective so maybe it depends upon upon your role and your and your professional responsibilities. That's great. Yeah, thanks Juan so much great to have you here. That makes me think that we should probably do a show with like people who teach climate science, you know, kind of talk about all those issues that would be very interesting. And so lastly, this is the last thing it's not really a question so much. It's from to go all Saloni and largely it is a question, but it's something that will probably is out beyond our scope here but how would social annotation be used as a research tool so not research into it but how would we use it as a research tool. And it is, you know, if you have something to say about that that would be super helpful. Yeah, you know, I would just say like very briefly, there's been I think some really important developments in this regard and the example I would point you towards is the annotation for transparent inquiry initiative, which is essentially using hypothesis social annotation as a kind of backbone to making research evidence more open and accessible in broadly social sciences research. I believe we include a link to let me just quickly double check here, annotation we do. To go to our, our new research vision blog post, we actually include a link to the annotation for transparent inquiry project that again uses hypothesis and this is I think an important distinction between. Here's an initiative that is essentially using the power of, we might say now web annotation through hypothesis to make research in this case on a whole variety of social science topics again, more accessible. Their name is, you know, annotation for transparent inquiry. Whereas this research agenda and initiative is looking at how social annotation as a practice can support aspects of again teaching learning shared knowledge construction, etc. I hope that's a useful distinction. And again, there's a link to that resource in our blog post. That was excellent. And I wanted just to say, again, thank you, Ray me, you're an amazing guest today, of course, I expected that you would be. And I want to thank everybody else for being here. And we're going to end this now but again the recording will be on our site and you will get an email with a link to it. I'm super excited to watch it myself because even though I've been here, you know, we're not, I've been paying attention to the chat which was also amazing and thanks for all the great questions and the links those will also be available. And with that, would you like to say a few words of parting. I just want to say thanks so much. You know, again, we have quite a few people here we still have a few folks hanging around. I just really want to appreciate the fact that this is a community, you know, driven initiative. The folks who are here, the folks asking questions of course it looks a hypothesis. You know, none of this would be possible without people the technology is very exciting. The hypothesis that it enables are also again very exciting but this is very much about the people who make this happen and a lot of those who are here today. I am just honored to call a lot of them friends and colleagues. And so thank you so much for just being involved in this work and again contributing to the broad community it wouldn't it wouldn't happen without you. That's great. Okay, thank you. Thank you everyone, and we'll see you next time.