 So welcome everybody. I'm Nate Angel from Hypothesis and today's episode of Liquid Margin's Annotating World Languages. I'm really excited to have some great guests here today with us, Georgia and Corey. And we're going to ask them to introduce themselves in just a second. But I wanted to welcome everybody to the show and invite everyone to participate as deeply as they want in the chat as we move along. So I'm hoping that, Georgia, you could start off by telling us a little bit about yourself and how you got interested in annotation and how you came to be here today. You'll need to unmute though also because you're muted in the background. Well, thank you. Hi everybody, Corey, Erin and Nate. Thanks a lot. It's good to be here. I'm associate professor of Spanish at St. Edwards and we have a population that's about 40% Hispanic and that really impacts the Spanish program. So that's something I think that'll be important later when we talk more about using hypothesis. I came to learn about hypothesis from the Office of Instructional Technology at our university that is kind of overseen by Rebecca Frost Davis and she sent out a call for people who wanted to pilot this. So I said, sure. And along with another colleague in Spanish actually, she also volunteered. We both used it and were really impressed with what we were able to do and the students had this reaction to it as well. So that's really how I came to get to know hypothesis. Great. That's so great to have you here, Georgia. And Rebecca is a great champion for collaborative annotation as well. So kudos to her. Corey, tell us a little bit about yourself. Who are you? Where are you calling in from and how did you get to know about hypothesis and start using it? I'm Corey. I'm the director of the Keck Center for Language Study at Coyote University, which I'd like to say is located on unceded land of the Haudenosaunee people. And my position here is I work with all the different language departments, although my specialty is in Spanish and some Portuguese or Portugno is probably more accurate of my Portuguese. But I do work with all the different language faculty. My interest in collaborative reading and annotation has been existed long before I knew about hypothesis. So I remember in grad school seeing a simple tool built by the folks at the Scholar's Lab at the University of Virginia called Prism. And it was a simple tool where you could choose three different colors to highlight a text as a group. It was really built on, it was really, it was a digital version of an analog activity that an instructor used to do where she would have students put a transparent projector transparency over a poem, a paper poem, and then they'd underline with different colors, different parts they found important. And that she would physically stack all of their transparencies together and put them on the projector and see, look, this is what we all thought. And so they basically built that into a digital version that you could, you know, highlight things. And then you could see the heat map and things about how people were annotating. So that was my first introduction. And I thought, this is great. I've been looking, I've been looking for something for years that could be more robust. And so it was in talking with our academic technology specialists here at Colgate. They're the ones who pointed out hypothesis as a tool and started using that. And I've used it in some classes, some language classes, also some culture classes, and also in a bilingual class that I pioneered. And that was really useful. So I've been really impressed with how hypothesis the amount of stuff that we can do together as a class and the way students have responded to using it, especially in the language classrooms. That's so, so interesting that it kind of built out of this analog kind of tradition, which I think, I mean, annotation has been around as long as there have been books to write in, right? And that's one of the points here. Well, okay, let me take this opportunity then to hand the baton over to my colleague, Erin Barker. Erin, could you say a little bit about yourself and lead the discussion from here. Thank you so much for coming. Yeah, thanks for having me. I'm Erin. I'm the customer success specialist for hypothesis. And as such, my job is to make sure that all of you know how to use it well and implement it in your courses. And then you're successful using it. My background with languages is actually I was a Spanish undergrad major at Southwestern University in Georgetown, Texas, studied abroad in Valdivia, Chile at Universidad Austral de Chile, which is my background currently. I thought I'd bring some culture into today's discussion and actually attended grad school in Spain multiple times. So that's a little bit of my background. I taught Spanish immersion in K-12 middle school. So language arts and social studies in Spanish and have some of my own annotation practices teaching reading in a world language or a second language for students. So that's a bit of my background. Nate, are you ready? Do you want to get started? Yeah, let's do it. Okay. Take it away. Well, so I wanted to start first Corey and Georgia talking about how your courses have changed in the time of COVID. We know that language courses really need a lot of interaction and are based a lot on our physical expressions and our facial expressions. So how have you changed your teaching style if you have gone remote or if you've gone hybrid? So Georgia or Corey, either of you can start. Sorry. Okay, Corey, I'll go ahead and start. Well, at St. Edwards, we are all online now. So that was mandated about two thirds of the way through the summer. I am not currently using hypothesis just because of the selection of courses I have, but I was using it in the spring and we had to kind of shut down in the spring after spring break. So I had already set up a few assignments with hypothesis and we were reading Mexican women courses and I used primary text, lección de cocina by Rosario Castellanos and it's a difficult text for students. So I think hypothesis helped to kind of break the reading down and scaffold it as well as allowing students to comment on an ongoing basis as we were moving through that text. And then I used it for an academic article, which they don't have a lot of experience reading in Spanish. But as I mentioned before, we have a probably 80% of the students in our upper division Spanish classes are Spanish speakers with a high level of proficiency. So I thought this is a good way to challenge them with this kind of language because they're not going to voluntarily read an academic article. So we used an article talking about the history of misogyny in different Spanish language texts, literature, Sor Juana was one of the texts that we were looking at and it was a really good experience for the students and for me because they felt a sense of accomplishment. They were able to read these articles whereas in the past when I tried to do something like that and nobody read that I had to end up summarizing the article for them, which is okay, but it's not the same. It's not the same experience for them. So being able to scaffold a reading, break it down and have them comment on it, I would have them comment on vocabulary that they had had to look up and give definitions of words that they looked up as well as trace the development of misogyny through the article and how the author was bringing that forward and critiquing that idea. So it turned out to be a really beneficial tool for the students and for me because it was satisfying to see that they were working with the reading, grappling with the difficult concepts and doing it on their own. And I will mute myself now. Yeah, for us the change in teaching over the last, I guess, several months now has been pretty drastic. Colgate has, we do have our campus open right now. We had students on a two week quarantine and now they just got off last week. So this is our first week of in-person classes, but still all of the classes are hybrid to some degree and have to be available for students to take remotely at any time should they need to be remote. So all of our classes are being prepared to teach remotely in some way. And for me, the biggest change in that is what I've been trying to impress upon my colleagues is that when you're teaching in a hybrid format of any kind, no matter how well you prepare for class, you're going to lose some of the in-person time to little things, you know, just setting up this or, you know, sharing your screen here or just little technical things, everything, you know, takes away and sometimes it's just 30 seconds, sometimes it's a minute, sometimes it's two minutes, but over the semester, those things add up and they add up and they add up and you end up losing really valuable time with students. So we've been thinking and I've been trying to think with colleagues about all the different ways that you can offload a lot of what you're doing in class to be done before class and outside of class. So how can we move the synchronous to the asynchronous? And this is also, and I think the Vanderbilt Center for teaching Stacey Marita Johnson is a great follow if you don't know her, a great follow on Twitter and has some great resources on, put her on on her blog, but she talks also about this being, you know, an issue of equity, right, that we need to think about the fact that not every student who's remote has the same access to the synchronous sessions as others and it could be the quality of their feed, it could be, you know, the situation they're in, it could just be how many of their, their family members or housemates are on the, on trying to use the same internet connection at the same time. So a lot of factors could make what happens for the synchronous sessions be less useful for certain people depending on their locations. So as we move things asynchronous, I think that really helps students get some more equity and allows them and then it also frees up time in the class. So how can we do what are the things that we do in class that really we could have students doing before class and then coming to class more prepared to do the things that are the most important class. And so for me when I was teaching last, last semester and we had this drastic change, I had already been using hypothesis for the class. It was a four semester class which is kind of this kind of bridge course to the literature courses from the language courses and we had been focused quite heavily, maybe more heavily than others would on the literary piece. So hypothesis had been very helpful because the students need a lot of, a lot more structure, a lot more scaffolding, as George was saying, to get into these texts that are a little more difficult at their level. And when we moved into being remote for the semester, a lot of the things that I was hoping to do and a lot of the assignments I had planned for them kind of became unfeasible because they required a lot of in-person activity. And so I decided to put a lot more emphasis on those readings and the use of hypothesis. So making those discussions and the things that happen in hypothesis offline more robust and kind of thinking of that discussion is more important for the course because of course, because again, I wasn't sure what the situation was going to be for students to be able to participate in our Zoom discussions. All those ended up being very good and very lively. I think that because we had put so much effort into doing this stuff on hypothesis and having that discussion happen and really having students interact with each other in more meaningful ways on hypothesis. And then it just became, you know, those discussions became much more rich. And really, I think for me, the saving grace of those few months, those first few months of quarantine was to have, you know, a time where I got to sit down with students and have a good discussion about literature. I think mentally that really saved me in a lot of ways. So I was really grateful to have that, you know, students were all prepared and I knew how well they were prepared because of what they'd done before class. I wish, I was just thinking I wish I'd had something like hypothesis when I was doing my upper level undergrad courses in Spanish literature. It would have helped me tremendously. So when a student is presented with these difficult texts in another language, it can be quite overwhelming, right? There's so much to take in. There's the, the literary, literary piece. There's the vocabulary piece. There's the contextual piece. All of this, right? So I, I'm curious how you might introduce using hypothesis or using annotation with these texts for your students. Do you give them specific things to look for? Are they asked to highlight different themes? What are you asking them to do with these literary texts? And Georgia, you can go ahead. Sorry, I know on Zoom it's so funny because you have to call out the specific person. Okay. So what do we, what do we do with the literary texts? We had, I had, as I said before, I used a critical article and then an essay. So the assignments were different, but in each assignment, in the very first assignment, I had included a link to hypothesis tips for annotating and as well as another link to, I think it was another university, Washington state, I'm not sure, but that had a good description of how to annotate what you're trying to achieve with annotating. And then I gave them my own instructions, which for, for the first article was, I asked them to make comments, a couple of comments before from between pages one and 10, and they didn't have to read all of that material. And then that they had to make an equal amount of comments on the second half of the article. And they should, they should bring up five vocabulary words that they had looked up. And then also trace the meaning of misogyny and what it meant for these different authors over time. And so their comments were really directed at analyzing the article itself. And I was able to see what they thought. I was able to see what kind of words they looked up, which were surprising to me because a lot of times it was vocabulary that I would have thought they knew, but it just, it just kind of reinforces how difficult it is for them to transition from shorter texts to longer texts. And then, you know, academic texts. So that, I think in itself was a good experience. And then seeing that they all completed the task, the first assignment, 100% of the students completed the task because they were held accountable for it. And I don't think they would have done that in the class if I had just said, read this part of the article and then come in and we're going to discuss it. They would not have read it. So like Corey said before, by the time we got to class, we were able to have this discussion and talk about it. And then you have other obstacles because actually verbalizing those thoughts that deal with complex subjects are also difficult. So I think I'll stop there, but that was the way I approached it. And I changed it for the second essay that they had to annotate. So I think it may depend on the assignment what it is you want to do. Corey? Yeah, I think when I think about like the why use hypothesis for these literary texts, I think of a discussion I had with a colleague. So he and I are both specialists in Don Quixote and the Golden Age literature. And he had assigned for his class to read. He has like this very limited or in addition with very limited annotations, very limited gloss, right? And I'm used to this this edition, which is the student edition, which has footnotes and it has, you know, every, you know, hard word and it's a fantastic addition. It's what I read as an undergrad. It's what I would assign. And I said, well, why don't you use this this version that is it makes it so much easier for students to see things. And he said, well, I don't want it to be easy for them. I want it to be they need to do that work right to get through. And to me, then just is going to inspire students to just go get a translation and read something in translation, right? But I understand the point of is on suit. We do want it to be on students to figure out when they're learning a language. It's not just about understanding the words but learning how to understand more words, right? I think for me, one of the great moments of learning a language was when I realized and I was also in Chile like you, Aaron realizing that I didn't need a dictionary if I could just describe what I was trying to say to somebody else and they could tell me what the word is, right? If I could just say, what do you call that? All of a sudden, I don't need a dictionary. I can just ask any of these people. They all know the words. So I'll just ask them. So getting students that point is really useful. And hypothesis for me is kind of the bridge, you know, kind of like an intermediate step between I'm giving you nothing and I'm giving you everything, right? Or really, you're giving it to each other as a group, right? So I'll tell you the tools to use. So like George was saying, I want you to look up the words you don't know, but you don't have to look all of them up because your classmates are also looking them up, right? So now you're depending on some of your classmates and you're depending on yourself. So you're kind of working together to get that. Similarly, I like to have them put in some context for things. So in the course I was teaching last semester it was a lot of, we were reading a lot of short stories by different authors. And I intentionally didn't give them information on who the authors were so that they could do some of that research on their own. And we talked about, you know, what are the sources that you would go to get this type of research, you know, how reliable it can be and different things like that. And then the other thing I really like to have them do in the annotations is how do they make, can you make connections with the other things we're reading, especially we're reading a lot of different texts. And how can you find those different connections. And then getting them to reply to each other, I think is always tricky. And maybe if we talk later about grading, I think there's some tips that I got from colleagues that help with that, but getting them to interact with each other in meaningful ways, I think it's also important because that, again, gets that out-of-class discussion happening so that when we get to class, we've already kind of started to talk about things, we know where we're going to go with the discussion. I love that you brought up that book because reading anxiety in language classes is a real thing for our students. And I can tell you that as an undergrad, if you had given me that version of Don Quixote that you just held up, I probably might have sunk into the floor and maybe not showed up the next day in class because I was not a natural second-language learner. It definitely took some immersion for me to learn the language. But that said, I want to talk about reading anxiety and you've addressed this somewhat in terms of creating context and giving students a platform to express their questions and their connections to the reading. This is more of a general question, but how do you address reading anxiety in your courses with some of these difficult texts, such as the one you just held up, Cori? I'm feeling undergrad anxiety right now, just looking at it. So how do you address reading anxiety with your students? And if you use annotation, that's great, but I'd love to hear some of your ideas. I think one of the things that I like to do in classes is the students working groups and we'll break the reading down into different sections and each group will have a part with a certain number of questions and they'll discuss those questions and go over it and then they'll present that part of the reading to the class and because there are certainly students that, because of their level of Spanish, probably don't experience language anxiety, but there are many that do, even though they speak fluently, they are not used to reading Spanish like that. It gives them a lot of confidence, I think. A lot of students come into class and when the first assignment in a literature course is a reading, invariably our students come to me and say, I don't think I'm ready for this class. I've never really read anything like this before and I can't do it. Well, your test scores show that you should be here and you can do it. You read what you can and then we'll talk about it and we'll go over it in class and you can talk to me in office hours if you still don't understand. I agree that it's a real thing. I do think if students will hang in there, over the course of the semester they get better and they realize that not everybody understands everything and that there's an infinite amount of information in any one text that we're never going to delve into all of it, but they do get better over time and more comfortable over time, but I think especially early on it's important to break the text down so that they can learn to read and hypothesis is one way to do that. Also in face-to-face classes it's easier and then on Zoom it takes like Cory was saying a lot more time to go back and forth and to break out rooms and use chat and that kind of thing, but there are a lot of ways to alleviate the anxiety I think. And I'll stop there. Yeah, I think that is right on giving them some kind of pre-reading questions to follow along, some kind of guiding questions is usually helpful. I also like to kind of give if we're going to do a reading for the next class maybe spend the last five minutes of the previous class just say here's a little intro to this text that you might want to know here's some things. And then I found the hypothesis useful to either precede with questions or things that I know are going to be hard to highlight and be like pay attention to this or maybe give some context I think they need to know that they wouldn't get anyway. For me it's really helpful in terms of geographic locations that they might not know are significant and that anybody who was from that place who read that would know when you say this place you mean something and if you're not from there and you weren't part of the history you might just gloss over that as just another place name. So pointing out those types of things I think are really helpful helping them feel more connected to the text and starting to get more deeper into it. But like Georgia said I think giving them some time to either do that basically think, pair, share model I think helps them kind of get over some of that anxiety and I find at least for my students less excited about the reading and much more about the speaking about it after so so many students that I have that I know have great thoughts and I think that's what the annotations help bring out that they can put their thoughts down in writing and then I'm telling them this is a great thought can you explain more to the class and so it's putting them starting out as a positive instead of I don't know how good it is do I want to really share that no I know that it's good so now you can speak with the class I think that's what's been helpful for me. The effective filter is live and well in language classes so this is kind of a follow-up question have either of you used images and annotations or had your students use images in annotations? I haven't but I was going to let Cory go first anyway because I feel like we got to go back and forth with who's going first so if you want to go for this one yeah sure I also haven't done much I've done some stuff I've wanted students to do it and I've tried to like model it but I think they don't seem as excited to do it as I do I've put in some videos but they haven't been as excited to do it so I don't think they have very much I guess a couple times I've seen them do it with more with links I'll see a lot of links people will link to the websites for where they found information especially if there's some context there but like I was saying about the geographic places that's where I'll use at least I'll use the images and say this is what this place looks like and why you might not know it's important okay well no I haven't and thank you for introducing that idea because I can immediately see a lot of applications for that especially with kind of a cultural studies approach when you want to you're looking at the concept of I don't know I'm thinking about colonial Latin America and the castus paintings and you want students to look at that and break it down and relate it to the concept of colonialism or imperialism or something something like that I mean I can see a lot of applications for that which would really help them also build their confidence because they can do it on their own and then share it at a later date so I think that's a great idea I was just thinking in terms of location because you brought up these ideas of locations and places and just like you know my zoom images that I change on the daily we could give students visuals so this is a question I get from professors daily and I'd love to hear about it in a language context as well is how do you grade student annotations and what are you looking for when you're grading student annotations and Cory I'll let you start this time yeah I am indebted to the way I changed my grading especially after the we went remote last semester and I'm indebted to my colleague Alison Kulizar here at Colgate who I think has also been on liquid margins broadcast so you can look up hers but I was my first thought was do X number of annotations but I gave them kind of like a breakdown of these are the different things you can do you can give context you can answer some of these questions so it wasn't you have to do X of each of these but it was for me easy to kind of go through and ask but then it just becomes pretty you know annotating to annotate right and when I get to five I got to five but what she was doing and what I started doing was when student instead of giving them you did X number you got X points she didn't tell them how many to do she kind of gave some guidelines on the types of things she'd like to see but then for the first couple annotations gave pretty detailed feedback to the students you know here's how well the annotations were for you here's what you could have done better and did that for the first couple of annotations which takes some time but you tell them I'm only going to do this for the first couple of annotations after that I'm just going to give you the grade and that's that was really useful so when we when I wanted to make the annotation part more robust last semester so we switched I told the students okay from now on the annotations are going to be more there's what I'm looking for and then the first couple I did it took the time to give them really breakdown detailed of what they were doing well what I'd like to see more and then from there on they picked up pretty quickly and the discussions really changed and improved from there and then in terms of the technical side of things we've been integrated we have our hypothesis integrated with our LMS which is Moodle for us and that has made the grading part seamless and easy so the folks who put that together I think has been it was really helpful to have that because you're not trying to like you know count things for different places and figure it all out it's all integrated which has been really really nice I would say we ours is integrated with Canvas and it has for for me it was seamless and it worked really well because it takes you right into the speed grader and you can you can see each student's comments and you can comment on their comments so that was really useful I thought and I have I had about I think I had 22 students in this literature class and we are an institution with the 4-4 teaching load so I usually have three different preparations so long detailed comments are sometimes more than I am willing to do so I use brief comments my rubric is set up so that you it's the quality of your comment and obviously that's subjective based on what I think and they do learn that because I do make comments to them over time and then the quality of the language are they paying attention to the mechanics of the writing this was an upper level class actually I probably wouldn't care about that if I were if it was Spanish 4 and just trying to get them into the text so I would dock the minimal points based on their language usage and the thoroughness of their comments but other than that it was it was kind of a borderline completion grade for me because I was just trying to get them to do it but having the rubric there and letting him know what the expectations were I felt like pushed the level of the language a little higher and of their comments and I had students that wouldn't often speaking class but provided excellent and thorough comments on the on the readings that they were doing so having expectations I think is helpful from the beginning just Georgia makes a good point there because I was teaching a lower level and so what I wanted from them was I wanted to be more again to reduce that effective filter of writing so I told them I'm not grading your accuracy in the language on these annotations I did give feedback and I did that kind of as a general thing for the class so it wasn't individualized but I would say you know you all don't know how to use Gustav or you know here's your problems with these this tense right and we could use that as kind of you know grammar when we needed to at a certain time but I wanted them to feel like this is not a place where you're getting you know really reviewed for that you're getting reviewed for the content so as long as I can understand what you're saying and your classmate can understand what you're saying and that's communicative you know you can communicate what you're trying to say that's fine I think that's a different question if you're like Georgia at a higher level you want them to be able to communicate clearly and accurately but at the lower level I thought it was good to have it less you know try to reduce that effective filter no I I think we're on the same page there and just to clarify I was very linear but I was like you know sentences begin with the capital letter you have to use punctuation and you have to use accent marks like if if they're turning in a substantial paragraph of an annotation that's a substantial paragraph and there are no accent marks I might take off a point for that segment of the rubric it's not it's not super strict but it's just to remind them that you know hey this is important here this is part of the assignment is the writing part the ideas of students annotating but then they might not say something in class you know they're still producing we've got the production level of language where they're writing and maybe the speaking comes later as a late speaker of language I can relate to that I apologize for the noise in the background I do have two children doing remote schooling right now so I wanted to head on over to some questions from our attendees and Michael has a question about facilitating students annotating after the fact so have you given instructions to students on what to choose to annotate like I guess what is annotatable before giving the assignment and I'm not sure if you can see his question in the chat he might have explained it better than I just did but so Cori or Georgia you are either of you are free to answer that I have to say I don't have the name right with me but it's one of the resources on the hypothesis website that there are some and maybe it's what's linked here it's probably in tips for students but I copied somebody else's information about what are the types of things that you might want what is an annotation and what counts and I added to it because as Georgia noted for us in languages looking up a word and giving the definition it's also important as well right I will say I also tell them when I ask them to look up words I teach them how a Spanish to Spanish dictionary so I don't want a translation look up the word and give me the definition because again that's another skill that you know reading a definition and understanding the word helps develop that skill figuring out what a word means by looking at other words in the language so it's much more beneficial than I don't want you to just tell me this means X and the same thing we do in class when you ask me how to say a word I'm not going to tell you what does this word mean I'm not going to tell you the word in English we're going to figure it out together in Spanish so doing that in the text is helpful but yeah I think there's a lot of resources maybe that's what's linked in the chat that kind of talk to students specifically about what can you annotate and how can you annotate and what counts as an annotation I also had a link from hypothesis about tips for annotating and tips from another university and then I told them the specific sections of the article and the essay that they annotated that I wanted them to annotate and one of them it was having to do with misogyny and tracing the history of misogyny so that was some guidance I think you're probably maybe talking something more about a deeper lesson devoted to what's that I didn't do that because I assumed that they had a certain level of knowledge about annotation which is probably not a good thing to do so I can see where that would be a point but I did there is some guidance available and you can just kind of link your assignment to those tips that are offered I feel like the history of misogyny is something that could be studied for quite a while that could take you down a lot of places so a couple questions in the chat so Nate asked I wonder if annotation activities might change at different levels of study and what you would think about that and Georgia we'll let you answer first this time okay I between my colleagues and I we've kind of come to this agreement that certain levels are able to read or should read be able to read certain text types and practice writing short texts or longer texts so we try to keep the levels appropriate so we have given thought to that the say fourth semester the bridge courses we're usually doing very short texts and the same for writing and then the upper levels where I am it's much longer texts so just text keeping the focus on the text type I think is is important for students yeah I can also talk about a course that was kind of experimental we have in our core curriculum we have students can take a course where they actually choose it's based on a geographic region and I've taught one on Latin America and I got permission to last year to teach it as a bilingual course so students had to we were reading text in English and Spanish although didn't count as a Spanish course it was in you know counted as their core course and so they had to speak Spanish which we had enough Spanish speakers to do which is something you can't do with a lot of languages on a campus but it was for that you know it was an entirely different situation than if we were doing it as a class because although we were pretty proficient in Spanish it was definitely very different ranges of Spanish in the class it was a smaller class but still even within that small class a lot of different ranges from one student who at the end of the semester told me he didn't know that we were going to be doing Spanish in the class even though it was pretty clear on everything and we talked about it a lot at the beginning of the semester to a lot of heritage speakers who also have a very different relationship with the Brit especially with written texts and then to you know Spanish majors who were already starting in their upper level Spanish classes and so we're much more proficient at both reading and writing in the language so in that regard we were doing annotations in a very different way because we were trying to use the annotations to help level the playing field for people so those who needed more help on the more basic things could get that kind of help from the annotations as they were reading especially when we were reading text in Spanish and we decided in that class if the text is in English you can annotate in English if the text is Spanish we annotate in Spanish right and the same thing for our class discussions we did class discussions in Spanish on Spanish days in English on English days so it was kind of back and forth but a lot of the students the lower levels the students with less abilities were the ones who depended a lot more on those annotations so the level is definitely key to what they're going to be able to produce for the annotations both of you alluded to this idea of different proficiency levels which aligns to the actual proficiency levels which I'm sure you're aware of and I like how you are using annotation to address the specific proficiency levels of individual students so almost individualizing the curriculum and the expectations based on what each student is able to do at that point I think that was really cool so Carlos had asked about using tags in annotations and then I think we're going to wrap up after that question unless there are more questions in the chat because we're over time at this point so do either of you use tags in annotations and if so how have you used them I can be brief I have not so I'll let you talk if he's had that experience no I haven't either but just seeing the question I was thinking about what Georgia is doing with the misogyny maybe having themes that run through the course because when you prepare a course it has a theme that runs through it and maybe starting to tag use that seems like a really good way to use let's use the tags throughout the semester to tag and then as Carlos mentions let's let the students come up with what those tags are let's see if they can pick up the threads as we go through it would be nice to see what they if they can figure out what they come to the same conclusion as I do is what the threads are that tie the class together it's funny I think tags are often that thing that are you know the next level in annotations that people start to think about Nate I'm going to turn it back over to you I want to say thank you to Cori and Georgia I learned a lot about using annotation in the language classrooms today thank you thank you thank you thank you yeah and I really want to again extend our thanks to both of you what a great discussion also from the attendees in the chat there I really liked how Carlos connected the study of language out to the studies of languages like science and this idea of asking students to come up with their own tag vocabularies as an exercise in and of itself that's a great great exercise to add to maybe a new discipline so thank you all