 Hey, greetings folks. This is Nate Angel from Hypothesis. But I'd like to welcome Rami Khalir, who also happens to be in Colorado. It seems like we have a real Colorado showing in the house today. Rami is acting this year as the inaugural scholar in residence of Hypothesis. He is an educator focused on teacher education at the University of Colorado, Denver, which happens to be right on the same campus with the Metropolitan State folks. So they're their neighbors practically, and I know they're preparing for a big snowstorm. So they're probably anxious about all that. But Rami is going to be moderating the next segment of the show, which focuses on social annotation in science. And so I will kick it over to Rami and let him take away and introduce the next part of the show. Well, greetings, Nate. Thank you. We are now moving into episode 20 of liquid margins, which is focusing on science education and the use of social annotation to support both science education, student learning in classrooms, and also research and research about science education. We have three really incredible guests who are joining us today, and so we'll turn it over to them in just a minute here. Welcome to episode 20 of liquid margins titled Making Sense of Science with Social Annotation. I am just thrilled to host and moderate this episode again. My name is Rami Kalir. Day to day I am an assistant professor of learning design and technology at the University of Colorado in Denver, and I've had really, I think the highlight of my professional career has been serving during this current academic year as a scholar in residence and hypothesis and working with a variety of colleagues and institutions to support the use of social annotation across disciplines and in both creative and consequential ways. We're joined today by an excellent panel of science educators and science researchers, and I'll introduce them in just a moment. But, Nate, if you do want to advance to the next slide, this is just a general reminder that if you do use the chat in today's Zoom webinar, please make sure that you're communicating with our entire community. And so toggle to all participants and attendees as you as you take those notes and share them in our Zoom chat. So thanks for that. You can also again follow that little link there for more information. And as a kind of preview to the next episode will be episode 21 of liquid margins. This will be scheduled for the 19th of March. That's just in a week called Northern Associations. And that's going to be about social learning in Canadian higher education. So that's coming up next Friday. So today's guests, I really have the pleasure of introducing and learning from and I hope that we all enjoy the wisdom that will be shared by the following folks. Dr. Melissa McCartney is assistant professor of biological sciences at Florida International University. She also serves as the director of research at science in the classroom. Dr. Aaron McKenney is assistant professor of applied ecology and also serves as the director of undergraduate programs at North Carolina State University, and she is joined by her colleague also at North Carolina State University, Dr. Aaron McKenney who is an associate teaching professor of biological sciences. I have had the pleasure of reading scholarship by these really esteemed science education researchers for really actually a number of years now in some cases, learning quite a lot about how social annotation has supported science education. And I'm really excited to hear from them about how hypothesis in particular but social annotation more generally can support science education learning in the classroom, and also science education research. And so that is our panel today and so welcome to all of you. If we might begin with just some general introductions. I was hoping to hear from the three are going to our three esteemed panelists. How did you first like learn about social annotation generally or perhaps hypothesis more specifically and why, since you're all science researchers, science educators, why did this thing called social annotation kind of strike you as kind of consequential for the work that you do given your professional expertise. Anyone's welcome to begin. So I learned about social annotation and hypothesis specifically through the open pedagogy incubator, which is an amazing program held through the NC State libraries. They participate in their inaugural cohort and learn about all sorts of different open educational resources and ways to really engage students in, in fully accessing content, right so lowering accessibility barriers, but also in taking ownership of co creating learning barriers, which I think a lot of times these annotations are. And then, I think, of course, it was kind of perfect timing with the incubator to be thinking about ways to, to lower accessibility and to increase inclusivity at a time where we're all remote and to build community online. Yeah, I can answer to. So I probably I think I started working with hypothesis probably in 2014. And we were actually approached. Someone named Jeremy and blanking on his last name right now. I'm sorry, like the Jeremy Dean or our car. Yes, yes, yes. Yes, he came and found us and we had some really great brainstorming sessions with him. And we kind of use annotation or hypothesis in two ways. One, because of the way it kind of the back end of the software makes it very easy to pull in to our to our website. And then the second reason is kind of what Aaron was saying like it, it gives people ownership and we started using it as a professional development training for graduate students who are looking to get more into science outreach and science communication. We use the annotation tools a way to help kind of break down, you know, like hardcore science papers, so that they would be more easily accessible to the public. So we kind of did it in parallel with the training on how do you, you know, how do you break down complicated science, kind of giving them background on that side of things and then hypothesis and annotations were really the delivery routes to make that all come together. I was also fortunate to be part of the open pedagogy open incubator with the library, and they are fantastic collaborators and I was brainstorming with them one afternoon on the whiteboard at one of our libraries. And I was talking about how I really wanted students to co create and read together and share their notes together and that was my goal for this, this assignment. And we've been doing reading scientific articles for a long time in that course and using different rubrics or frameworks. And then I had a Micah Vandergrist from the state library said, Carlos, you need hypothesis, and I remember stopping him and I said, yes, part of the activity is students identify the hypothesis in the papers. So you need to really look into hypothesis and have students annotate within a group, and he set me up and really helped explain how it could help. And it's been, it has snowballed from there and I've been learning from you and others, how we can really engage students. Thank you all for joining us today and sharing your expertise and wisdom. I'm wondering if you might kind of deepen your engagement with social annotation through perhaps a story or an example. You all wear multiple hats, both again as science educators, and also as researchers of science education and how a variety of learners are deepening their disciplinary engagement through social annotation. I'm wondering for folks who may typically associate annotation maybe with the humanities, or with personal marginalia and may not necessarily associate social annotation with the sciences. So people may have a hard time kind of like imagining what does that really look like and again because you're all working, whether it's in labs, or in courses, or in professional scientific education initiatives. What does this really look like from your perspective can you share a few some stories of us love to hear some of those from all from all of you. I can go first this time. So for me in in my work, I am very interested in sharing what scientists find with with everyone, you know the general public undergrads high school students whoever it is anybody who's really not a trained scientist. And one thing that is, you know has come up over and over again in our work on this is when you ask someone to read a scientific study the first thing that they stumble on is the language. You know specific, you know, jargon terms, scientists, some, sometimes we like to make up words, you know, like, like post translational things like that that make makes sense sense to us but they don't necessarily make sense to anyone else. So for us, annotation does many things but mostly it really breaks down that language barrier, because when you come across a word that's not intuitive, and you really, you know a lot of scientific words, you can't even really use context clues to find what they mean. So we've done a ton of work with the annotations and just defining vocabulary words as a way to just encourage people to keep reading, you know this word might trip you up and you're going to stop but here's that definition. So keep going. And that has been our in the sciences, our main, you know the real main benefit of annotations. I'm taking notes as fast as I can Melissa. I think in a complimentary way to that. In my courses where where I see you are are making science translatable and, you know, participatory for the public, right, or accessible to the public. In my classes, I also from day one, I teach ecology classes and ecology, you know we have these like basic tenants of biodiversity is important. And yet, there is not enough, in my opinion, focus on the diversity of humans is important. So from day one I try to really emphasize, I offer a lot of group work and group discussions whether it's small groups or full class and it is so important to me for for all of us to recognize the value of diverse perspectives that, you know we now there are multiple publications now that demonstrate that diverse humans produce better science. And so I think annotation provides a space for students to build confidence to call out you know what is this jargon, even within a stem class to become, you know to break down those barriers to sharing and discussing you know assigned readings in in class discussions but also to hear you know all these different readings. I'm thinking about you know scrolling through the Langston Hughes poem from the annotated workshop and and just seeing all those diverse perspectives and readings, and we can get the same diversity of perspectives and the same richness of discussion in a talk about a single published journal article. Right. So, yeah, those are some aspects that I'm really excited about. Yeah, and I can just following up on what you said I know in my own teaching and especially since we've gone online. I think students are much more willing to ask a question written, you know in an email or on some kind of forum in the chat box as they are out loud so can you bring up a great point, putting this stuff in into hypothesis or into an annotation format. You're going to catch more of those students asking questions who never really would have said it out loud or come to office hours or anything like that. Yes, and I think it also provides a space for the three than me principle that you know I try not to jump right in and answer students questions and wait for a few days, because then I find that you get enhanced, you know or increased peer to peer interactions answering questions for each other. And then if there is a lot of like, yeah I didn't get this either, then I can jump in and say, Oh, well actually here's a little bit of of interpretation to help you. But yeah, I, I love the community aspect and and Erin does a fantastic job creating community in her courses and really emphasizing diversity of ideas. I teach molecular biology classes very techniques based in a bio technology program that has undergraduate students and graduate students in the same class. I look at that more on the method side of microbial diversity, for example, a class I teach, and I'm focused on the methods so some of the papers that we look at are really dense. While we may have grad students that are folk are have already been reading papers, their fields may be textiles, this sign, maybe college of medicine. So, and on the other side I have undergraduates that are fantastic chemical engineers in their programs or biologists who may not feel comfortable in person, talking to the grad students because they are intimidated. And on hypothesis when we annotate papers together. I also try really hard not to start placing comments and answering questions, but what I've seen is there are more interactions between grad students and undergrads and I would like to quantify that at some point but my classes are smaller, which is good. And then the other thing that's really interesting is we we tried some papers where we actually had one of the authors one of the lead authors be part of the class, and they are. We had trouble reading those papers by informatics, and I told students it's okay to ask questions. And with, with hypothesis, I felt like I had to define to scientists and engineers, what annotation was. Once I came up with our definition of we are going to ask questions, we are going to clarify the text by adding another layer to make it more transparent to us and others, what the meaning is. And our idea was or our goal was to annotate ask questions linked to YouTube videos or other papers and explain what's going on in this complex scenario. And then once we have a set of annotations groups of three or four students are tasked with making the page notes. So we mentioned page notes earlier, and we use the page notes as a way of trying to summarize annotations in small groups, and that has been a lot of fun. Thank you for these these stories and perspectives and you know we've got questions and resources that are now flowing through the chat. And I wanted to also kind of open up in addition to you know your commentary about social annotation in a class context, and some of the pedagogy surrounding that you've also all embarked upon research research perhaps about what your own students have been learning and doing or again how others are engaging with scientific literature through social annotation. And it strikes me that not every educator, whatever their discipline may be is going to choose to start researching the processes by which their students are learning using a variety of new technologies and yet you all have. And I just be curious to hear again from everyone, you know what motivated you to also look at annotation from a research perspective. And what are some of the key insights the key findings that you are beginning to glean that you have identified in regards to the ways in which social annotation does support engagement with scientific again terminologies concepts student learning professional learning as well as science communication. Bring us into your research as well as again having previously shared a little bit about your teaching. Okay, Aaron do you want to go either way. You go ahead. So I guess, I've been thinking about, I've been learning about the ecology of education and the ecology of learning and thinking a lot about, you know, what about the core ecological concepts like diversity and functionality and you know, connectivity, right. I'm thinking about kind of food webs and maybe this is a food for thought web. And so those are some of the concepts and that are shaping questions that I'm asking. Again, very much from the perspective of, you know, hypothesis as a tool to increase student confidence, but also student community, particularly during this past year of remote learning. So, so I'm asking questions like, and some things that I found from from quantifying the usage data from my fall class. I found that you do see higher connectivity or more interaction via replies and threads, the more that students annotate. So the more you use it the more you might benefit. I've also seen, you know, across the board, no matter what the usage students overwhelmingly reported that they felt that annotation did increase their confidence and did increase their sense of belonging in class, which felt exciting. But that has kind of opened some, some additional roots for questioning like, Well, what, why don't we see, you know, a tighter correlation with a number of annotations. Carlos and I started thinking about imposter syndrome and, you know, well, what are the spaces where we might see like a false inflation of annotation that might not correlate to an increase in confidence. So now we're thinking, you know, oh, can we quantitatively identify, you know, usage behaviors that might help us pinpoint some sort of imposter syndrome and how could we help to address that. How can we help increase student comfort and confidence. So, yeah. I am, I'm stuck on this and I believe the libraries a little bit on open educational resources and and co-creating and I went through the pandemic summer of thinking about my teaching philosophy and thinking about how we teach courses because we teach lab based courses. And what what I enjoy from the experience and what I love about teaching and I love the equipment and the toys that they haven't had access to in a while. I also love having students create something and that really drives me and I think with annotations and summarizing and being able to understand complex terms, or being able to come up with a group of students came up with their tag system for what it is to learn about their undergrad research. It really emphasizes how creating community and empowering students to to do something. You have the self efficacy you agency comes to mind, helping students discover what they want to learn about them and not shy them off with a. You have an assignment you have to read this paper, and I've struggled with that because I want them to annotate but I don't want it to be a checkbox. And I think Aaron, and our experiences have really helped on the instructor side, comparing notes and learning about, okay, how can we do this better to not shy away some students and shy away is probably not the right word or encourage everyone to contribute in a way that's meaningful and where it's comparison free which may be green land and impossible but so that they can share ideas in a way that others can build on those and it's not a requirement that's a community. Okay, so I've come at this very different. And there's two different ways that I've done research with annotations. The first way is I do a lot of work with graduate students and professional development, specifically graduate students who don't want to necessarily stay in academia and they're looking for these quote unquote alternative careers outside of working in an academic lab. The way we use annotations with the grad students is like I mentioned before that training, teaching them how to how to translate complex science for a general audience. And it's always amazing to me that grad students don't recognize that they have these skills, because I think grad students go through training and for them is just normal. You know because they spend every day looking at these words and kind of translating to themselves in their head, and they don't see it as a skill. So we, we use the annotation process to show them know you really do have a skill. And you can really turn this scientific paper into something that the, the, you know, the rest of the world can use and benefit from. And then of what what Aaron was mentioning we do see confidence changes when when they go through this they do, you know, start to realize oh I can contribute to the, you know, the larger society. And they do increase some of their skill or communication skills, specifically, you know, learning how, you know what parts of the paper to annotate how to annotate what to include in the annotation, those kinds of things. And the other benefit to the annotation process is at the end of it they have like a tangible packets that they can take with them. You know on interviews or put in a portfolio, or something like that showing that that you know they've they've actually done this they came up with this product, and it's out there in the world, and you know and someone's using it. So that's kind of what I've been investigating on the professional development side of things. Then, as far as students reading other people's annotations we use that with first year undergraduates in intro biology courses. And, you know, you can't just give a freshman student of scientific paper and say here go read this it's really cool it's really interesting. You're going to know you're going to need to know this for the rest of your scientific career, and good luck. There's all kinds of research out there. Some of my own about barriers that students face when they go to read a scientific paper. What we've done with the annotations is design them specifically to start reducing those barriers so you know vocabulary is a big one methods is a big one. Because a lot of these students third their intro bio students they might not necessarily know what a Western blot is, or, you know, what, you know, how to how to titrate something. So we get really get into the methods section. So, and again that that shows the graduate students writing the annotations that they really do have valuable skills and the undergrads who then read them. It helps them, you know better understand what, why we need scientific papers how the scientific community uses scientific papers both to you know advanced science and to tend to teach others, and it eliminates a lot of those frustration barriers that would initially turn students off. I'm not sure I'm working on this now actually I'm not sure it increases their motivation and their confidence to keep reading PSL I think it does. I don't have the hard numbers on that yet. But that that is something that that we're working on right now. And another thing I think the annotations do with the undergrads who go on to read them is it helps them learn strategies for breaking down complex text. And they'll, you know, they'll start to realize Oh, all these words that I don't know are defined. In the future when I see a word I don't know maybe I should take the time to define it. And, you know, taking like, like chunks of text and writing it. You know in your own words or in simpler language, I think that's also strategy that they pick up just just seeing annotations and knowing that they're there. Well, thank you so much for this again really keen insight from again a research perspective on how students in science, again, and professionals and graduate students are learning and engaging with this practice. We're about to transition into some q amp a with again the many attendees who are here and as I'm looking at the chat there are a lot of questions that are coming through but I'm hoping that that I might just just kind of wrap up with one more question for the for the three of you. And I don't mean to pigeonhole annotation, since it's something that I care a lot about and study but it often is associated again with maybe the humanities of the social sciences or again, informal kind of personal reading. And I wonder if the three of you have recommendations specifically for other science educators who may hear about social annotation and kind of say, huh, that sounds interesting or maybe to reference Carlos's story or like hypothesis. Oh, you're talking about the hypothesis of a study, not a tool. And I'm just curious again, given your various areas of expertise and again your deep experience with social annotation. Do you have a recommendation for other science educators who again maybe curious or hearing about this for the first time, where and how might they begin this process of dipping their toes into the pedagogical affordances of social annotation. One of my advice is don't be afraid of the technology, your students will get it, even if you don't. And, and to be honest, I think that has worked in my favor because a few times I haven't known how to do something and they've showed me. That's great for them to know that they're also teaching me and I think it builds a little more of a rapport between us so don't I don't be afraid of the technology. That's excellent Melissa thanks. I think also, I've tried to build in practice time, either as part of my welcome letter like here's the syllabus. We are going to be using hypothesis so go ahead and open an account and then practice. So I'll have a link to the get started for students tutorial and then I've I've shown screenshots or recorded a brief video and zoom. I'm going to ask them to annotate the syllabus so it's a very low stakes and it helps, you know, make sure that we all understand what's required, and they have helped me to identify like, oops, I did not update that date from last time I taught this class. You know, before we're halfway into the semester, you know everybody's on the same literal page in the first week of class. And, and what I want to add to that now is in addition to introducing it in the welcome letter also carving out some time on day one, just to make sure that everybody has you know a guided 15 minutes like let's all go to the same page the same as we did in the annotated workshop earlier, you know I think that's so helpful. Yeah, and as Melissa said, it's hugely helpful to get the students on the same page but also to afford them opportunities to correct and enlighten me. It has changed, but every semester, and I've tried hypothesis and now for different classes, and different topics of papers or different types of assignments, and one thing I've done is don't be scared of the technology and also have the students ask questions. What I try to do now is when I was in person and I've just finished coach teaching this class with my better half. I would point at me and say, Okay, Dr. Goller, how do we use hypothesis and I will bring it up on the screen that we would start and this is what we're going to do we're going to annotate this paper and it's really neat, because now we can talk to everybody read this paper. And what I've done online is I've filmed screen tasks of me going over the assignment expectations and how it's going to work, and actually going through annotation and putting some adaptations in. And I've honestly in some of the videos, I've bumbled and then didn't do it within the right group or, and students get to see what's going on, and see that, oh yeah, I, there are so many options I can do I can bring in rich text and links. And I also tried to, as Aaron mentioned, emphasize it throughout the course our courses are typically eight weeks. So I, we have to be up and running really quickly. So, after we read a paper. I tried to summarize in a video summary and walk them through with my screen open with a hypothesis open show them how we went from a blank layer to this layer. And you just did an hour ago, they're full of annotations 140 annotations on this paper, and say you created this, and now let's make, let's make some sense out of this, what do we want to extract from your comments. Again, thank you this is just so for me I'll just speak for myself so informative and so again I just want to thank you know, all of you for joining today and I'm going to really just shut myself up now. I know that Nate hypothesis colleagues also Franny have been looking at the chat a little bit more closely than I have. And this is a time to turn things over to all of our attendees, and to elicit questions from those who joined us today. Some of the questions have already been asked in the chat, or folks want to drop those questions into the chat right now. Or we can also promote folks if they want to also share their questions, you know, by speaking it out. The floor is open. I think it's actually you guys have been doing a really good job of sort of addressing things both in chat and and live as we've been talking. I have noticed that a lot of folks have been centering around this idea of, you know, how to make sure learners of any age really are, you know, kind of able and empowered to actually use social annotation technology and you guys have been addressing that already. And there have been a lot of ideas floating around about, you know, using video, like Carlos was just talking about the demonstrations and so forth. And I just, I think it's some, I think that we should stress that it's never true that you can assume everybody is just going to be able to pick it up and use it easily with no problems that might be that way for some, but it's not going to be that way for everyone and so if you. I'm sure that you can all speak to how you've had to sort of support different levels of kind of ease and facility with social annotation in your classes and it could be that you have sort of pointers on how to, how to start the exercise of annotation off in ways that make it easier to figure out who needs more help and who's ready to go. And I think that he himself has a lot of practice in this area too. I don't need to necessarily kind of share my, my fumble legs in my, Dr. Carney, please. Yeah, I don't, I don't usually do annotations in a social group, which I'm realizing now that I'm listening to all of you, the grad students kind of do them on their own and then the undergrads read them on their own. That's a great answer for this, although I'm interested to hear what everybody else thinks. So, we use. I've been using two different things. So for one class, I change the articles every, every time I offered a class, and we just have one group, and we keep on annotating in that group, because the previous articles are still techniques and approaches that are used, and students can refer back to them, and students can actually see the examples of how one paper had 150 annotations. For other classes, I, I use the same two papers. So I create a new group every time. For students to annotate within that group, like yeast metabolic engineering 2021. And it's interesting because I see, I see the lack of an example in the yeast class at the beginning that could be helpful but I could share another paper or share some screen grabs. And one thing that I've, I've, we've always have little tech issues but I'm trying to get students to help each other out and I use student forums as a place to troubleshoot and ask questions. And once, once they realize they are annotating publicly and that's why they can't see the annotations or we should be using this link. And it has been really great because the students have understood how to have picked it up really quickly have helped each other out. I've had an undergrad who is now really committed to open science and really mad that papers behind paywalls. Issues sharing annotations between institutions to the point this student who graduated in December now wants to write an op ed article and I love that so from from troubles we can make, we can make learning and hopefully hopefully students learn in the process. So that's the next question come through so maybe I'll just pick it up from the chat if that's okay. We've got a question from that asks, you know, thanks a lot for the super into discussion, certainly echo that. Can you share a concrete assignment that you've that you've done in class again for those of us who maybe don't take this form of social peer to peer learning for granted. What does this really look like for your students in your in your class. And hypothesis for jigsaw exercises so I, I teach a class of 45 to 47 students in the fall. So in order to break that down into what I hope is more manageable sizes and and for the jigsaw I divide them into four expert groups. Each group gets assigned one paper and then I say you know. So all of the papers are available through our class private group. So they could potentially go and see all of the papers but they have to create five annotations, a minimum of five and that way it's a low stakes. I'm giving you credit for reading the paper that I'm expecting you to read to discuss in class so it seems, you know, fair to in a way to acknowledge their efforts. I encourage them, you know, I'm going to pick up from the annotated workshop like if the text is already highlighted don't create a new comment, reply in the thread, because I think that's a fantastic way to increase connectivity. And then I can compare to student behaviors from this fall and see like, does that actually increase you know thread length and the quality of our discussions. I think that really has helped again for them to clarify what the paper means and then if I give those expert groups 10 minutes to convene and come to a consensus. So something I want to add to that is maybe they will type a summary page and compose, you know, what's your one paragraph, or two paragraph review, right your TLDR for other groups, before they get shuffled into their jigsaws. I think we probably have time for maybe one or two more questions if, again, maybe may or front he has seen things come through the chat. Yeah, one thing I just actually wanted to bring out as I see it, folks are using the chat space to share articles that might be good starting places for various, you know, various scientific discussions and various disciplines and that's great and I can, I can understand why sharing them and zoom chat is maybe not the most efficient way to share resources like this. So I have an announcement to make that very soon will be unveiling companion site for liquid margins that specifically designed to allow educators like themselves to share common resources publicly about your practices. So for instance, if you, you know, let's say Carlos and Aaron have particular scientific articles that they think are great introductions for annotation exercises and their particular discipline, you know, you could share that with some even some additional information about how you use it as an assignment and so forth. Look for an announcement to that coming soon sorry that it's I wish that I could point you to it today but I it's not quite ready. Maybe that will maybe get us around this issue of trying to share, share a bunch of links and chat quickly before something ends. You know there's been a lot of a lot of talk about how the different disciplines might have different sort of needs and affordances right. And I think one suggestion that came out in the chat that was pretty interesting was the idea of instead of moving right to a scientific article, you can also use an exercise and annotating some sort of maybe mainstream media public treatment of science right. And so you can bring the scientific lens to a more popular work if you will, and have that really be an interesting annotation exercise, as opposed to diving deeply into a really rigorous scientific work which like and science in the classroom does so well, and I'm wondering if Aaron I saw you nodding as I was talking about that, do you have like particular experience directing students toward sort of more popular publications if you will. No, I was nodding in like I totally agree with you and I'm so excited that there might be a shared repository and I was thinking oh and then we can annotate them with like what we do right it's perfect it comes full circle. I. I give a blend. I, I don't like to label anything as like a classic paper or the paper and I think that's partially because I, I'm a microbiome researcher and you know that's a fairly new field and it's changing all the time as we learn more and as we are continuously by the microbes we study right. So I don't, I don't want to be sessile I don't want to anchor myself to anything specific. I. So I have often incorporated some of my own research into courses as case studies to give students a chance to challenge me in person if they have questions, you know to practice discourse with a published scientist. And to break down that hierarchy and power dynamic, hopefully, but I am even thinking like I also don't want to inadvertently hold my work as some golden standard because really it's not, you know it's. It's not where the nature of science is that it's dynamic that we are learning all the time that you know. Yeah, every day is a new Copernican revolution, right. I'm so, so, no, I don't have a today I have a go to paper but you know, by next semester, maybe they'll all be different. They'll all be different. Yeah. Sorry if that wasn't. No, no, it was really interesting take. Carlos. I have two random comments that somehow tried to connect here. So one is, I have teaching posts of research postdoc here at NC State Jason with him who has a ton of experience in bioinformatics and meta genomics said, I want some teaching students, can you help, can I help with this class and that was the best. Yes, I one of the best yeses I had last year, because he brought expertise and students felt comfortable. Maybe it was because we were using hypothesis usernames, but they felt comfortable saying I don't know what that means. I really appreciate that and these were science papers will change them up next semester as we try new projects. And with undergrads, I have the pleasure of working with student groups that are undergrad researchers and without the lab, we are lab experiences students for craving research experience. I was just amazed and happy and really impressed with what groups of students did once we said, Okay, let's find out everything we can about this weird bacterium. And it doesn't have to be science papers annotate wiki annotate public newsletters try to connect and students in that case made up their own code for tags and for for for classifying annotations. And I also did the same thing here and I was like, I'm not sure I feel comfortable calling them seminal papers or not. Students, they came up with their system and then they just ran with it or annotated with it, and I had fun. And in that case it was a combination of science papers posts, anything they could think with this key name. So with regard to selecting papers. It's very hard. And I, a lot of people ask me this question I do not have an easy answer. It's, it's difficult it's going to take time you're going to select some that don't work. And you know learn from what doesn't work on that and in, you know, keep selecting in the future. And a couple of things that I have found that usually always work is think more about the experimental design in the paper. If it's a like the simpler kind of experimental design the better. Because, you know, help students follow kind of from the question to the, the experimental design to the methods, you know, and those there's really elegant papers out there, I, and I would encourage you guys don't shy away from science and nature. They are shorter papers, but because they're short sometimes the experimental design within them is really beautiful. So I would look for the experimental design not so much, you know that the vocabulary words they're going to use or the length of the paper or the journal but but find a really beautiful experiment and start there. I mean, kind of bouncing off Aaron. I think it's great to select your own papers, and I would totally encourage that because research coming out of my lab shows that a lot of undergrads don't even know that we do research. They don't know research happens, especially I know you guys are at a huge campus like I am. A lot of our freshmen don't even know there's a biology department, which seems crazy to us but but students who commutes who get out of their car go to a lecture and then go back to their car they really don't understand that research is happening on campus. So I think it's totally great to use your own papers and use the papers from people in your department as a way to show students that this research really happens on campus, and they can be a part of it. And all you guys need help, you need volunteers. And you know it might help kind of the sense of community within the department about knowing what's going on knowing what people are doing and finding out a way to be more involved. I'm kind of thrilled at the kind of the various directions that this conversation is taken, particularly on this point about, you know, having researchers who are educators, then share with their students their own work to engage in a conversation through social variety of things that again you've all mentioned I just want to re voice this because to me this is just so important, you know, mentioning things like showing your students that research is happening on campus, and that you know you are also engaged in this kind of process of scientific inquiry. So, you know, to Aaron's point a few moments ago about also potentially troubling to some degree, the perceived expert novice teacher student power dynamic that can exist, particularly when for example students maybe reading, you know, primary source scientific literature, you know, conducted by and written by their own professors, but using social annotation as an entry point to then begin inquisitive conversations about the topics, the terms, the concept of the methods that to me just speaks to just the complexity of learning and the opportunity for social notation to deepen this shared shared experience. So thank you all so much for sharing from your experience. I believe that we're running out of time, and I hate to make that transition such a hard one. We're coming up against the hour here and typically these episodes run for about 45 minutes. I didn't know if there were any final concluding comments that our panelists wanted to share. I don't know if there was any final insight resource or question that you might want to leave us with or there was any other quick commentary or even a bit of housekeeping notes from any of our hypothesis colleagues, as we begin to wrap up today's today's liquid margins episode. Let's let's hear from the panelists first for sure. I had no idea. So many people were doing annotations so it's been cool to meet you guys and learn about what you're doing. And I feel like I'm like re energized to get back out there into annotating so thank you. I read Dr. McCartney's paper, and I really, really love that the hypothesis community has been so welcoming and vibrant and learning from from others and I really love that we were able to connect and share some thoughts and resources and when I had tech questions how can I get this. So it has been a really supportive community and I love learning from Erin and other fantastic educators what they are doing to encourage others to participate. And I think going back to my naive comment what do you mean hypothesis a couple of years ago at the library. Now, now I can rephrase that and say, okay, how can we deliver the guidelines or or deliver the charge of because I'm calling it charge now of we will annotate and make sense of this together because believe it or not, Carlos does not understand some of the text here. And I think that with 20 of us doing Google searches and being empowered to tag someone else, or ask questions or even annotate publicly on the authors article, we can get some answers. I absolutely love that Carlos and I would say, first broadly, this has been amazing. I, this has been another another flavor of I came to hypothesis because it seemed to make sense and it seemed really exciting. I've been doing things by intuition but once again I'm meeting people who are better established and have more experience and, and who just the chat, blowing up this whole time with so many inspiring questions and anecdotes and to be able to connect to this web of like motivated inquiry. Right, I mean, that's coming full circle to hypothesis and into what Carlos has just been saying about charging and empowering students to drive and and, you know, satisfy their own inquiries, and instead of, you know, leaning on us and and I'm there with you as an instructor I really try to model. Yeah, this part of the paper was super confusing to and how amazing to now have a space where we can document that, and then muddle through it together. I think that's, that's incredible. It's an incredible tool, it's an incredible revelation for students, you know, really democratizing science and education. I want to just for myself personally just who's just I've just been so inspired by this episode, but also on behalf of hypothesis as an organization and the broader social annotation community. I want to thank Dr. Melissa McCartney. I want to thank Dr. Aaron McKinney and also Dr. color for joining us today and sharing again your expertise and wisdom with us. And a big shout out and thanks to the OLC innovate both this online gathering and the broader OLC community for kind of hosting us within this broader event, and making this public and available to everyone. This has just been I think a really kind of edifying and inspiring conversation so just thank you all. I know that resources from again the chat and related this will be shared publicly through the hypothesis liquid margins page. They will also probably be shared out on Twitter and other social media channels soon. Please continue to stay connected to again the hypothesis community. And from the chat that actually it's episode 19 that's coming up next. So we're going back in time somehow, just somehow kind of apropos for this last, you know, year but in any case please do again remain engaged with the liquid margins webinar series as well as the broader hypothesis community. And my thanks to everyone who's joined us today. Stay healthy everyone. Please do take care.