 further ado, Charles, and Autumn, it is over to you. All right, thank you everybody for coming today. We're so excited to have you, and Autumn, if you want to share your screen, we can kick off this workshop. Are you seeing the presentation now? I am, awesome, thank you so much. We are here to talk about creating counter-media texts in the open with Annotate EdTech. My name is Charles Logan. I use he, him, his pronouns. I'm a PhD student in the Learning Sciences Program at Northwestern University in Evanston, Illinois, which is just north of Chicago, and I'm very excited to be joined by Autumn. Hi everybody, my name is Autumn Keynes. I go by pronouns of she, her, hers, and I am an instructional designer at the University of Michigan, Dearborn, and you can reach out to me on Twitter at handle, at Autumn with two M's, A-U-T-U-M-M. We did want to begin with land acknowledgments. I am an uninvited visitor on the traditional homeland of the people of the Council of Three Fires, the Ojibwe, Potawatomi, and Odawa, as well as the Menominee, Miami, and Ho-Chunk nations. These homelands were a site of trade, travel, gathering, and healing for more than a dozen other native tribes and are still home to over 100,000 tribal members in the state of Illinois. I also want to acknowledge that in addition to land being taken and erased from Indigenous peoples, part of the Settler Colonials Project is also erasing Indigenous knowledge and ways of knowing. And we heard a little bit about this last week in Keynotes, and so I was actually reading this paper from Dr. Megan Bang, a native scholar at Northwestern, and I'm very lucky and grateful to be learning with in the Learning Sciences Program. And I wanted to share her work in this paper in particular, because for me, it resonated a lot with what some of the conversations were happening last week around thinking about epistemic violence, thinking about ways in which closure may actually be preferable to openness. And so to me, it's important to share this work because I continue to learn from it, and so I wanted to share it with everybody today. My institution does not have a statement, so I do not have a statement that was developed in conjunction with native peoples acknowledging the land, and so I'm very sensitive to that. But I am aware that I live and work on the territory of the Three Fires Confederacy of First Nations, comprised of the Ojibwe, Ottawa, and Potawatomi, and I acknowledge that the land that we now call the United States was taken in incongruable ways from indigenous peoples. I'm interested in these histories, and I'm seeking teachers and resources. And to come to this land acknowledgement, that I wrote myself because my institution doesn't have one, I've listed some resources there at the bottom, and we will make the links available after the fact. All right, so what are we gonna do today? We are gonna get to the annotation and the discussion, which is gonna be the heart of our session today. Before we do that though, we thought it was important to provide a little bit of history and context before we could sort of dive into the margins and get annotating. So next slide. I made this meme because I think it helps to illustrate the genesis of annotate ed tech. So over the course of the past year and a half, two years, I have been as long as well as many others frustrated by some of the rhetoric that online proctoring CEOs and companies use to sell their products, which in my opinion, are racist, ableist, privacy-invading surveillance tech. And yet a lot of the CEOs, both on their websites, as well as letters to senators here in the United States, sort of praise their surveillance with language of equity and integrity. And so to me, that was troubling. And so I wanted to sort of push back against that language and push back against those narratives in a critical fashion. And so annotate ed tech was sort of born out of that frustration. Real quickly, so I had this idea of what if we use social annotation to construct some counter narratives. And so I went to Autumn and Nathan Schneider and Aaron Glass, all of whom are part of a group called Ethical EdTech. And I said, hey, what if we did this thing? And they would say, hey, that sounds cool. And so they were very supportive. And then last November, a bunch of folks from across the world actually gathered. We use Zoom as well as a rise up etherpad and etherpads are open software collaborative writing documents in conjunction with Hypothesis, the social annotation tool that we're gonna use today. And we were able to create 95 annotations and responses, conversations on the websites of three of the major online proctoring or invigilation or invigilation, as I think it's known outside of the United States. As a means of producing what Raimi Collier and Ontario Garcia describe as a counter narrative or an alternative to conventional methods and messages. So in my mind, annotate EdTech is sort of this movable feast. It's this thing that sort of a learning experience aimed at, for me, sort of bringing that degree of criticality towards narratives that are sold by EdTech companies and then often taken up by institutions and then, of course, use, I would argue, against students. And so we're gonna be thinking about creating a counter narrative together today. And Autumn, I think this is this. I thought it was the next one. Okay, oh yeah, you're right. This is Hypothesis. This is what it looks like. So I will say as a reminder, we are using Hypothesis. You will need a Hypothesis account if you would like to annotate today. You do not need to annotate to participate. You can still read the annotations and that's totally fine. And I do wanna recognize that one of the companies that we are going to be annotating some of their language is extraordinary litigious as well as problematic. So totally understand if folks would rather sort of watch the annotation and then join the conversation or not be in the chat, whatever sort of engagement looks like for you today, fantastic. But this is the interface for Hypothesis on the right-hand side. And so what you see here is that production of that counter narrative. Here are three different examples of this is taken from the proctor you created a student bill of rights. And so we don't know if any students were involved in the creation of this document or teachers or if it was just sort of company folks. And so we see here in the first annotation is this close reading asking questions about the kind of underlying assumptions and language used by the ed tech company and then working together to expand that knowledge. And to me that's a really powerful way of thinking about annotation is again creating those counter narratives in community. And so that to me is one of the really powerful ways of using social annotation. And just in the spirit of making this part the participation that we're inviting you to be a part of today, making it your own is that it doesn't have to just be for today. So I'll draw your attention. Again, these are screenshots from the last annotated tech that was run. And you'll notice that the one on the left from Valerie November 16th was done on the actual day of the event and the next one from a cog dog is November 17th. So don't feel like you also have to stop annotating after this event is over. If you wanna go back and continue to read or if you wanna pass the annotation link on to others feel free to do that. You can really kind of make this your own. It doesn't have to be just tied to this particular event. You can also stretch this out over time as well. Actually, what do you mind going back to the previous slide? I just wanna point to I think another powerful way of learning that these two annotations point out is using the affordances of the open web. And so you have here a sort of collection of hyperlinks of resources. And so to me that's another powerful element of annotated ed tech is both the careful reading of the text that's on the screen but the way in which that text is in conversation with others. And so using hyperlinks using as we'll see on the next slide images as a way to create a really kind of rich and robust set of resources. And so here we have a sticker that was actually placed on the one of the company websites by the profit of innovation doom as the person's username. And this sticker actually comes from something that Chris Gillard, a professor and tech critic has said and actually I think, I don't know if Autumn has more of these stickers but I know she and Chris were mailing them out to folks who requested. And then on the right is actually a GIF. So you can use GIFs in hypothesis. Again, thinking about that word integrity and really reading closely and the language that these companies use to sell their products and sort of poking fun in this case, using humor as a means of surfacing some problematic uses of the language, certainly in this person's opinion. So I wanna point those out. And both of these images and then the individual annotations as well as the conversation get at I think a key idea that I wanna point out in the next slide. And that is this, that annotation carries with it political and social aspects to it. And so this is a quote from Ramey Clears and a lot of work in social annotation. And then I just wanna draw everybody's attention to a distinction that he makes between defacing something which he argues centers the ideology of white supremacy versus using sort of the language around graffiti. And that graffiti as he argues and I would agree is more of these intentional marks of justice directed counter narratives. And here he shares an example of a statue of the Confederate general Robert E. Lee in Richmond, Virginia. And this place has become a kind of gathering point for Black Lives Matter protesters here in the U.S. And so yeah, absolutely maha. And I think sort of in that spirit of criticality but also of creating justice oriented graffiti that's what we're gonna be thinking about today as we move into our annotation portion and thinking about partnerships between ed tech companies that have a very kind of specific way of thinking about teaching and learning teachers and students. And so we're gonna provide a little bit more context now about the companies that we're gonna be engaging with today. So the article that we are going to annotate is a press release about a top hat partnering with Proctorio to deliver free secure online proctored tests and exams remotely. We'll be getting you a link to the article that will enable the hypothesis annotation here in just a second unless somebody's already done that. I should say too, I can't see the chat right now. The only thing I can see is the, and I hear the little bings coming in so I know that folks are chatting but yeah, I can't see the chat right now. Just FYI on that. And we just wanted to- I just put the link, sorry, I didn't interrupt. I just put the link to the hypothesis enabled press release. So yes, please do you tweet away. Yeah, and if you wanna share that out, you absolutely can. We would love to invite more people to come in and annotate with us. But before we get started on this, I wanna recognize that not everybody knows what remote proctoring is. We are talking to a global audience and some people don't even call it proctoring. Like Charles just said, they call it invigilation. And so I wanted to talk a little bit about the harms of remote proctoring. And so we're talking about Proctorio when we talk about the proctoring piece of this. This is kind of a little bit of a complicated piece because we're basically talking about a partnership that's being created between two companies. One of the companies is Proctorio and they do remote proctoring. And there are problems with remote proctoring on a number of different levels. We have lots of examples of this technology in particular being racist and ableist. It is a 100% algorithmic proctoring. There is no humans involved in this case. And it's not to say that human proctoring in a remote setting is okay or good. That has a whole other set of problems as well. But we see people with darker faces who cannot be picked up by the algorithm, which puts them in a situation where things are much different and they get disenfranchised in really problematic ways. The proctoring companies will tell you, oh, they can just call the helpline and they'll let them in or they'll help them troubleshoot it. But we have just some really dehumanizing factors. People asking, people being asked to shine their lights on their face with fell ash lights and just can be really problematic. There's many, many examples of this technology not working. The companies, of course, will tell you that, oh, it's 100% every time you can completely control your exams and your tests. But there's many, many examples of students finding ways around. Humans are very inventive. So it is not 100%. There are many examples of it not working. It's very expensive. There are lots of cost concerns. Data security concerns, there are many, not put this, well, I can't say with this particular company, but with many of these remote proctoring companies, there are several instances of data breaches, that kind of thing. So what's happening with these data and how well are they being secured? Privacy concerns. So just the fact that students are asked to put themselves on camera, they have no choice. They have to do a room scan. So they're, and we're not talking about like a shared testing environment, right? We're talking about people's bedrooms, people's living rooms, shared spaces, that kind of thing. And then just what this does to the normalization of surveillance culture, that we're talking about a school environment where students are learning about the world around them and about what kind of people they are going to be in society and when this is made normal in a school environment. This easily becomes something that can perpetuate out into other parts of society. And so like I said, the article is about, it's an announcement, it's a press release of something that I call a fourth party integration. That is a thing that is out there a fourth party vendor. And what this is, as an institution, you make a partnership with Top Hat, let's say, but then all of a sudden Top Hat announces that there's a new feature that they have. And if you go to Top Hat's website, you don't look at this press release, but if you just go to their website and click on the page about their remote proctoring, you would have no idea that the remote proctoring is from Proctorio. There's no mention of Proctorio on that site. There's screenshots of the tool and if you're familiar with Proctorio, you might be able to see and recognize that the Proctorio is the tool. But yeah, you would have no idea. You, as an institution, would not have signed an agreement with Proctorio. You wouldn't necessarily have any idea that this is going on. This also is a huge loophole. So there's a lot of institutions that are trying to set some limits on the number of vendors that they're interfacing with because this opens up all kinds of problems for data security and that kind of thing. So this is a huge loophole to that. Now, all of a sudden, your students, you and your students' data are going to be interfacing with being collected by, stored by Proctorio and you really had no idea. You signed up for Top Hat. And this begs the question, what is Top Hat? Top Hat, they call themselves an active learning platform and they do a bunch of different things. So they have textbooks. You can buy your textbooks through them. They have an annotation tool so we're gonna be using Hypothesis today but like Top Hat has their own annotation tool. They do remote proctoring by reselling Proctorio even though that's not entirely clear at the front end. They also have a clicker, kind of like a response system type of tool and they provide some learning analytics as well. So just to give you a little bit of an idea of the tool that we're going to be, or I'm sorry, the article that we're going to be annotating, that's kind of the different players that are involved and the article itself is a press release about a partnership between the two. Okay, let's get annotating. So as I mentioned earlier, you do need an Hypothesis account in order to annotate. You will not need one if you just wanna observe and see the annotations. I think Autumn is gonna demonstrate what that looks like here in a second. I did put that link in the chat as well as Autumn, I'll let you know that I put the against online proctoring open library that we have been and other folks have been helping to curate. Autumn also has a really wonderful fellow on Twitter which she's definitely do, a kind of ongoing lit review in which she sort of calls out and shares different articles. There is a depressingly large number of them. And so as we move into annotation, what we're gonna do is we'll just turn off our, I mean, everyone's camera I think is off right now and audio, but we'll just have some quiet annotation time. I'll be in the chat if people have questions. If at any point you do wanna just sort of ask a question, please do that in the chat. You can turn your camera on and your mic on. So we'll be able to have some annotation time. As we annotate you might be considering some of the questions that we've listed here. I'm really, as I said, overall the aim of thinking about what is the narrative that these companies are telling and selling about their products and where might we produce a counter narrative? And so that's sort of the goal for our annotation time and then we're gonna have a little bit of time at the end, maybe 10 minutes or so, to sort of debrief, share any noticeings and think about broadly where annotate ed tech might fit into your teaching and learning practices and contexts. So I think that's that part Autumn. If you wanna, oh yeah, that's a good question. We use the annotate ed tech and it is important for accessibility reasons to capitalize the E in ed and the T in tech. So if you are going to use the hashtag, please make sure you do that. That's a great question Ma, thanks. Yeah, I think we're probably ready for the demoing. Are you able to see the article now? Yeah. Okay. And so, yeah, when you first come to the text that we're going to be annotating, you will see the article. You may wanna take a moment to kind of read through it and see what's going on with it and think about your annotations. And then basically there should be a little sidebar over here. You'll notice that there's a little arrow and an I and a little page. And I usually just start with the arrow to open up the sidebar. You'll notice that there's page notes here. So Charles has started us off with an initial page note with those questions that you might wanna think about while you're annotating. And then the individual annotations will be over here on the side. To make an annotation, it's pretty simple. You can just highlight and then you wanna go to annotate, not highlight. Your highlights are private to you. Your annotations will be public to anybody who's on this article with the hypothesis link or with the hypothesis plugin. And then when I click on annotate and say, and when I'm ready, go to the public. And notice that if anybody wants to respond to that question, you can come over here to reply and make a reply there. And it looks like we've already got some new annotations coming in. I can tell because there's a red circle with a download kind of arrow here. And when I click on that, I'll get my new annotations will pop in. And I see Mehabeli and Carolyn are in here already jumping into some stuff. So now I'll be quiet. Charles, if you have anything else you'd like to add and just let people annotate. Yeah, the only other thing I would add is that if you do, I see some folks doing it, wanna add the annotate edtech tag or any even the OER by domain is 21 tag two. And the reason why that is potentially helpful is that once you click that tag, you'll be in conversation, not only obviously with what we're doing today, but you'll also be able to call up annotations from any other annotation that has been tagged. So that's the usefulness of the tag there. Yeah, and so it's about 10, 30 AM my time here in the central time zone in the US. So why don't we spend maybe about 15 minutes on them or so? And we'll see. I mean, if there's a lot of energy in the margins then we can take it longer. But yeah, our goal is really just to have that quiet work time now. If you have questions, please do unmute yourself and ask or use the chat and otherwise happy annotating, happy counter-narrative writing. And we'll debrief here in about 15 minutes or so. It will annotate for about another three minutes and then come back and sort of debrief on the experience. Lots of really wonderful conversations on things. I'm really good. Okay, I've got 10, 48 my time. And so I wanna make sure that we have enough time to just sort of debrief. And I'm curious as folks looked at the article and the annotations, what struck you? Was there anything in particular that you were surprised that made you angry, frustrated? What kind of, yeah, struck you about the narrative that emerged about education and teaching and learning from this and any of the comments that you read or left? And you can respond to the chat if you wanna unmute yourself. Mejha's back, welcome back Mejha. We were just saying we're just gonna open it up for conversation about what we just did, about what we just saw. And you can feel free to jump in. It looks like Carolyn has her hand raised. Hi, thank you so much for doing this. And I was just gonna pipe in and respond to Logan's question. I think something that was significant to me was the peppering of potentially positive terms or terms that are attractive to engage educators, the peppering of words like active learning, integrity and like pedagogical approach. It's just kind of this peppering of these words that are kind of in some ways being co-opted in the actual overall message to me being expressed in the press release. So I think that was something that, so often I found myself pausing and being like, how are they defining that? I don't think it's the same way that I might necessarily define that term. Yeah, that's a great point. Were there other words or phrases that anyone else noticed as sort of, yeah, language that I'm, yeah, integrity of exams, yeah. Students. Yeah, that's when Mahaa makes me think about some of Autumn's work that she's done around weaponization of care and that it's not caring for students, but about students. Yeah. Yeah, and I'm expanding that article actually into a more full-length article. So I've been, I'm actually in the middle of drafting it. So it's very, very at the forefront of my mind. And I think it's, yes, I attended your workshop on the, I believe you were citing Jane Toronto and I usually go through Nell Nottings, but this idea of there's a difference between caring for and caring about, and caring about ideas and things is much different than caring for people. And so we see this division of caring about integrity, comparing about caring about rigor, caring about these ideas, but what about caring for students and the vulnerable students who could potentially be harmed in these spaces, right? Other noticings about either the language from the press release or any of the conversation that was happening in the annotations. I mean, one of the things that I'm really sensitive to is this partnership that this has just passed off as like, oh, there's a new feature. We have, we now can offer you this thing and thinking about what about the people who don't want that? What about the top hat customers who signed, who didn't sign up for that? And now all of a sudden there's this free thing that is available to them. I will say, I do believe that they only offered it free for a certain amount of time. And we saw this specifically around the time of the pandemic, right? All of these, the beginning of the pandemic, I should say, because we're still in the pandemic, but we saw a lot of companies come out with this, oh, we wanna care for your language. Oh, we went, Alan, we're gonna give you something for free, right? So I do think they rolled that back a bit and that now you have to, students actually have to pay specifically for like the tests that they're going to be offered. So it's not an improvement, but yeah, it just strikes me about how these partnerships can impact us, especially if you're like an administrator at an institution, you didn't sign up for this. You didn't vet this product, you didn't have a committee that looked at this product and figured out if it was the right thing for your institution and the right thing for your students. And now all of a sudden, guess what? You're now a proctorial customer, just like that. And that just seems really problematic to me. I'm also thinking about language around seamless, I think, I don't know if efficiency is in there, but I think so much of it, selling, right, on the one hand, I understand you want your technology to work, but also this notion of, I think it sort of bleeds into teaching and learning as seamless as smooth and what happens when that kind of vision of education is one that is widely adopted, rather than education as learning as challenging, as rough in places and rather than trying to smooth away all of the difficulties where those challenges really be embraced rather than everything happening seamlessly. That being said, you do want your technology to work like today, right? So that's interesting, that tension, I suppose. I love this little back and forth between Mehta and Sincolson and here with the no choice to refuse, that makes me think about the Against Surveillance event that happened last year, and I believe it was in Mehta's session with Benjamin D'Axiater and Savva Sahleising and Chris Gillier, that was one of the, when they talked about care in some of these spaces that when you are presented with this caring language, if you're presented with some of these welcoming statements, ask yourself, well, what does it mean for me to refuse if I were to say no to this gift that you're offering me? Am I even able to refuse it? And that is of what I believe is a key, that's a key indicator for anybody who's trying to figure out, is this real care or is this a weaponization of care? If you can't refuse it, if you can't say no, is there something right about that? That is true as well. I'm not sure if I'm staying here. Yeah, name right there, but yeah, the institutions definitely have a lot of influence here. And it's bigger than just any one technology tool. We're not picking on any one technology tool here or even a set of technologies because this is a systemic problem that we have in the idea that we think about the assessments that we give. It's in our accrediting agencies, it's in our institutions, it's a lot bigger than just trying to talk about problematic technologies. Yeah, it's been interesting to see it just reading a comment about, as the rise of these technologies, and again, I wanna echo what Autumn said, it's not one particular type of technology, but it seems like in conjunction with that conversation of assessment, as these technologies have risen sort of exposed sort of maybe less effective ways of assessment that at least, and maybe it's my echo chamber on Twitter, the rise of ungrading and really more and more folks who are embracing forms of ungrading as a means of sort of pushing back against a more sort of skills-based assessment, or rather not sort of embracing a wider range of types of assessment. Yeah, right? And I will add that one of the things that technology always seems to do, right, is that it exacerbates, it amplifies, it makes things more efficient, it streamlines things. And so even though I wanna say, like I don't wanna target any one particular technology or any set of technologies, I've done in-person proctoring before, and I've seen the harms that can come from that, the student anxiety, I've worked the accommodations room with people who have a differing set of needs that the accommodations don't always account for, and I've seen just really horrible things happen there, and so I do know that this technology is just gonna amplify those horrible things that come just with this type of testing. Thank you so much for coming. So I'm just curious, in our last two minutes, I know, I think take a sort of step back and think about social annotation as a broader phenomenon in particular to sort of annotate ed tech, and if folks have ideas or ways in which they envision using something like this in your own teaching practice, in one minute or less. And if you don't, that's fine too. Yeah, absolutely. Yeah, it's that normalization of surveillance that Autumn talked about. I think we may be out of time. We'll just give you a conversation. I was just about to say that was absolutely, absolutely fascinating. And yeah, I had lots of things running through my mind there at the end about neoliberalism and education and lots of different, lots of different things, which I'm currently writing my thesis on. So yeah, that was great to be part of and moderate for you. So we could all put our virtual hands together and give a big round of applause to Charles and to Autumn for that, because that was absolutely fantastic. So thank you everybody for taking part today. I'm just gonna...