 Hi, everybody. Thank you so much for joining us today. I am so delighted to be able to welcome you all to this talk. We have been just thrilled for months that we're able to have Dr. Alex Hannah join us here at SJSU today and online for those of us who are joining remotely. And Dr. Hannah is going to talk to us today about shifting the frame, the labors of ImageNet and AI, and it's our real pleasure to be able to bring this talk to you through a circle, the cross campus interdisciplinary responsible learning experience. Which is a Mozilla sponsored responsible computing challenge project aimed at bringing responsible computing education to all SJSU students, staff and faculty, regardless of background because we all have a role to play in our broader tech ecosystem to make it more equitable, inclusive, diverse, fair and representative. And so I'm really glad that each of you took the time to be here today, and we're also very thankful to our sponsors and partners of course Mozilla and the RCC and the other funders of the RCC, but also with the School of Information, the Computer Science Department, the Center for Faculty Development and Ecampus, and of course, CPGE, who is allowing us to use our beautiful space in MLK today. I am going to pass the mic over to the folks in person and to Dr. Hannah. Thank you again for joining us. We're so excited. Okay. Hello. Welcome, everybody. Thank you so much for coming to this talk. My name is Dr. Alex Hannah. I'm the director of research at the Distributed AI Research Institute. We are a non-profit research institute dedicated to focusing on the harms of artificial intelligence and also imagining technological futures, which are community oriented and based in people's needs and technology. I want to thank Professor Adar for inviting me here. I want to thank you for coming here on this cloudy day and coming inside and hearing a little bit. So what I want to talk about now is this project I've been working on for the past, I want to say, at this point, four to five years, and the name of the talk today is called Shifting the Frame, the Labors of ImageNet and AI Data. And a lot of this is my thinking on what I hope to be a manuscript down the line. I'm working on one manuscript right now. The AI con with Emily M. Bender, which is a book that's going to be out in spring 2025, which is about AI hype and the ways that AI is threatening to replace workers and art and all these things we hold dear, but is really just kind of fluff and is really hyped up. I want to take a step back in this talk and really think about all the work that goes into the creation of AI and how much of that work really goes unacknowledged. So that's really the crux of this talk. Okay, so I want to start up the road in Palo Alto at Stanford and actually this actually starts a little across the country, so it's across the country and at Stanford. So in the late 22,000s, early 2010s, Feifei Li, who is the director of Stanford's Human Artificial Intelligence Center, Professor of Computer Science, was involved in creating this data set called ImageNet. Okay, this is a non-trivial manifestation. This data set was, like this headline said by Mark Bergen at QZ.com, the data that transformed AI research and possibly the world. And I don't think that's really an exaggeration, the way that we often hear about the way that AI is transferring, you know, transform the world, et cetera, is very much an exaggeration. But really, image that's put squarely in the way we do AI research now, ImageNet was the predecessor and really set the ground for for this. And I have a few, if you don't believe me, I've got more, I've got numbers to prove it. So the ImageNet data set is a huge image data set. It's a bunch of images, 14 million images that was collected in the late 2000s and the early 2010s. And ImageNet, when it was released, was the largest data set that was released at the time. The prior largest, the second largest data set that was released at the time, it was called Pascal VOC. It only had 20,000 images. So this is orders of magnitude larger than Pascal VOC. Pascal VOC had 20 categories in which these things were classified, images were classified. So ImageNet had 20,000, actually precisely 21,842, okay, and is regarded as a key benchmark as a thing against which algorithms for object recognition, I'll explain that in a second, and object localization would be tested against, okay? So to promote adoption of this data set as a benchmark in 2012, starting in 2012, the ImageNet team, which is Fei-Fei Li, Olga Rosakowski, Jia Dang, and many, many other grad students and postdocs at the time released this at a computer science conference called CVPR, Computer Vision, I think Practice and Research. One of the biggest things for computer vision research. And they were looking for an algorithm which could effectively, actually I think it is in 2010. So they were looking for an algorithm which could do two tasks. One of them was image or object recognition or image classification. So namely, if I present a computer with an image, can it tell me what's in that image? That's the image classification task. And then also object localization, you know, if I show you a picture of a cat in an image, can it draw a little green box around the cat? Okay, these are pretty important. You can imagine how this gets used in industry. It could be used for robotics. It could be used to actually identify where something is in an object and what it is, okay? And it wasn't really until we saw this one algorithm come out, this one algorithm with this interesting name called Aleksand. No relation to me. It was related. It was released by three researchers at the University of Toronto, Alex Krasvinsky, Leo Sootsgever, and Jeffrey Hinton. And it brought forth this kind of notion, this old notion of computer science that's actually been around since the birth of AI in the 1950s, this idea of the neural network. Everything good? Oh, it's covering content. Sorry, yes. I know Zoom does this. Thank you. Oh, thank you. Amazing. Thanks. So this idea of convolutional networks, has anybody heard the term neural networks? Let's say in the past, raise your hand, raise it up high. Yeah, like everybody has heard of a neural network right now, right? And it's because this kind of idea of what we have, and this is an old idea first proposed by Rosenblatt and Campbell and I think 19, I want to say the 1940s. And this team of Krasvinsky, Sootsgever, and Hinton went ahead and resuscitated this because there was now sufficient data actually use these methods and sufficient amounts of compute to actually use these in practice. And this really became an inflection point in which artificial intelligence research one first use these very intensive compute measures. So this is a quite dated graph here from open AI released in 2018. And which we see this inflection point of Alex and the number of petaflots days needed in training. Okay, so the original perceptron, this one kind of this one layer network developed in 1960. Alex then we see these things, all the resnets used in computer class in image classification and neural neural machine translation up to AlphaGo zero, the algorithm which Google developed to be a human competitor at the game of go. I just looked it up this morning. This is probably one of the most cited papers ever. It's not actually very, very far to say, just as comparison the original GPT paper, few shot language language models or few shot learners has 20,000 citations. This has 125,000 citations. So even five times six times the magnitude of the popularity of the GPT the original GPT paper. Okay. And it's this thing is still having during that has still hasn't during value to look at papers of code. This is a few months old, but papers after C far 10 this data set is used in over 10, 10 and a half thousand papers across 102 benchmark tasks. It's still being used. I took this picture in mid 2023. I'm sure it's consistent. It's not even budging that people are still using this data set. So much so that in a paper that I wrote with Bernie Koch, Remy Denton and Jacob Foster, we actually were interested in understanding the concentration of how much certain data sets get used in particular benchmark data sets. Those data sets which are used to actually evaluate different algorithms and we found two interesting things. First off, the places in which benchmarks are created are actually concentrated in a few like very few universities very few institutions. The top being Stanford, the second being Microsoft and Princeton, Max Planck in Germany, Google, the Chinese University of Hong Kong, AT&T, TTIC, I think is a technical institute of Chicago, NYU, Georgia Tech, Berkeley and Facebook. Okay. So most of them vary at the top Stanford and Princeton, the largest amongst them of universities. We also found that the usage of benchmark despite the kind of explosion of interest in AI and machine learning benchmarks have actually become more focused as well. So institution, both on the institution level and in data sets. So it might be true that we're getting more institutions involved and more data sets involved, but we have actually more concentration across time and our data stopped at 2020. It wouldn't surprise me if it's continuing in this upward trend. What ImageNet did in another paper, Rosvan Amaronesi, Remy Denton, Andy Smart, and Hilary Nicole and myself talk about what seemed to have happened, what was new about this ImageNet way of doing work. And we found three things. I'm going to present to you one of them right now and then the other two throughout the rest of the talk because they play into different parts of it. But one thing is that what ImageNet seemed to do for AI research was to show that more data is better. They take this title from this paper written by a few scientists. Oh, and I want to go back and just say, you know, the authors of this paper, they went ahead to wild success. Ilya Sutskeva went to be former chief scientist and co-founder of OpenAI. Jeff Hinton went to Google to be a fellow. Their startup was bought for millions of dollars by Google. Okay, going back. So this had very, very big personal ramifications for themselves. So ImageNet really set the tone for how we do this type of research, huge piles of data. Not data that would say be curated and specialized. That would focus on a few things to work for particular communities or particular use cases, but throw more data at it. So much so that it really set up this kind of wave of ImageNets of X. This is a slide that Dr. Lee uses across different presentations in which he shows this kind of variety of images. So of data sets, space net, music net, medical image net and so on. But it's also led to this particular sort of concentration of data and compute in particular locations and this agreement between universities and corporations that really focused on needing to partner with them for compute. So in an article for the ACM Transactions, Meredith Whitaker, former head of AI now, former advisor to the FTC on AI, currently president of the Signal Foundation, wrote this article about how this idea of being captured, this idea of having research that does not fall into this bucket of using so much data and using so much compute, is not really possible. You really have to get involved in these institutions to play the game. So this is the grounding on which I really want to start talking about ImageNet. So this is our kind of preamble here. And I want to think about this in three ways. Again, I started at the top of the talk talking about work itself, thinking about the work that goes into the development of these data sets. And what I want to think about and keep in the back of your mind is the way that certain kinds of work are surface, certain kinds of work are obscured, certain kinds of work are forced to become invisible for the magic of AI and the magic of the data used to power AI to succeed. So the first thing I want to talk about is ontologies. I call those ontologies upon ontologies. So data sets operate as a type of infrastructure. Okay. And the paper that I wrote with Remy Denton, as Rosvan and the others I mentioned, as well as Morgan-Claude-Showerman, we talk about how data sets work as a type of infrastructure. And here we borrow this concept of infrastructure from a literature in science and technology studies, particularly a subset of science and technology studies that's called infrastructure studies. Infrastructure studies holds that the kinds of things that we see in the world are built upon other things and those things are invisible. When we say infrastructure, we tend to think about things like pipes or roads, but what else is infrastructure? The wiring behind the walls, the kind of work of how we build standards, the way that we have practices, the way we structure time. Time can be a type of infrastructure. And when you think about data sets, they operate as a type of infrastructure through four ways. Right here, I'm going to talk about one of them. First off, data sets first determine what a model learns, right? So if I have a cat classifier and it has two categories, cat and non-cat, then this model is going to work itself through the world and say, well, all I understand is cat and non-cat, and it's going to try to classify something at that. Now think about an image classification system built on ImageNet, which is aiming to classify 21,842 categories. So that's going to learn particular things. And what it's going to learn is going to be very much a feature of those data. So in a great article and looking through different classes in ImageNet, photographer and cultural theorist, Nicholas Maylev looked through it and was like, well, what I'm seeing in ImageNet is this kind of practice. And know that ImageNet was built in different ways. First off, the way they built ImageNet was by scraping images from the web in the late 2000s and early 2010s. They built it by scraping things from search engines like Altavista and Lycos. Those are search engines that existed before Google did and Google wasn't keying at the time. And so they were scraping this, especially from user-shared content on things like Flickr and PhotoBucket. And so we actually kind of see this classification of what gets learned. In this case, as Maylev says, as hammerhead sharks here are seen as objects of scientific inquiry, trout as dead trophies and lobsters as food. And if you dig into the data set, you really see how this holds. The other thing that ImageNet did, and this is going back to the Big Day in the Society article that Rosvan and Remi and I wrote, was the thought about how you could have this idea of this kind of notion of a singular interpretation of an object. So the second thing we would argue in that article is that how ImageNet had this computational construction of meaning and understanding. This idea that you could sort of objectively see something and have the same interpretation. Donna Harway is very famous in science and technology studies calling this the view from nowhere. The idea in which you're looking at something and everybody sees the same thing. And there's lots of proof that that doesn't happen, that these things fail, especially when you look, try to give it something or a category outside of the West. And so this is really signified by this slide from Lee's lab, which is called the Vision Lab, in which you have this kind of God like eye with this lens. That's an iris in the solar made from sky with this motto to build computers that see. Moreover, the kind of ontology that is the dictionary upon which ImageNet is built is called WordNet. And the categories which exist in ImageNet are drawn directly from WordNet. The way Lee describes this, Dr. Lee describes this is that they are, this was constructed because of a chance meaning she had when she was junior faculty at Princeton. She went there, she met with Christine Feldbaum. Christine Feldbaum was director of the WordNet project. WordNet, I just want to bring this up is like a dictionary. They call it a large lexical database of English. So nouns, verbs, adjectives and adverbs are grouped into sets of cognitive synonyms, what they call it, syn sets, each expressing a distinct concept. Think about this like a dictionary. Okay, I turned it a dictionary and I look up chair because I know it starts with a C. But say I want to look at what, instead what WordNet does is I want to know what chair is. Well, I know that chair actually inherits from different properties. A chair is a type of furniture. A piece of furniture is something that is man-made. That man-made thing then is an object and it then goes all the way up to this root node called an entity. And you can actually see this, I took this word Pembroke Welsh Corgi, which is an example used across ImageNet. And you can see down here, I don't know if you can see it with this zoom thing in the bottom. I don't know how to move this. But what you can see, oh, here we go. What you can see is that the Pembroke, the Welsh Corgi then inherits two dog, then to canine, to carnivore, to placental mammal, to mammal, to vertebrate, to chordate. I don't know what a chordate is. Animal, organism, living being whole object all the way to entity. Okay, so this kind of weird synth networked dictionary. Okay. So in this, a lot of this is kind of thinking when I'm getting back to this idea of work, is that who put together these kinds of things? Who said this inherited from this? Who said this and this and this? Actually, where do these words come from? Oops, here. And I was actually pretty interested. And I looked at their FAQ and said, where do these words come from? They said, our lexographers write them. I'm like, okay, that's a pretty glib answer. Where do you get the definitions of word in the long answer? So they say in a book published on this, from the forward to word net, an electronical, lexical database, people sometimes ask, where do you get the words? So, and they go through a pretty long list here. So we began from the 1985 words in Coursera and Francis Standard Corpus of present day edited English, formerly known as the Brown Corpus, which was developed in the 1940s, which principally because they provided frequencies of words and different parts of speech. Then they basically went, they asked this guy, Charles Osgood. They went to various thesauruses like Laurence's Udang's basic book of synonyms of antonyms, Erdang's revision of Rudel's The Synonym and Finder, Robert Chapman's the fourth edition of the Roger's Internet International Thesaurus. And one thing led to another. They basically tried to find words and their relationships, whatever they could. This is a really curious way of thinking about this. And it's one of the least documented, in my opinion, parts of word net. Where did you get this? I've met with Christine Feldbaum with Dr. Feldbaum at Princeton and asked, where do you get these words? And kind of had the same answer. Not to dog the approach or anything, but to also know that so much of this work has gone unacknowledged and has immense consequences. So I wanted to dig into this. And I went ahead and I looked at the Brown Corpus. The Brown Corpus is available. It's online. I didn't misspell this. This is how manuals spelled online and looked at what kinds of publications were in it. So the list of samples they had, they wanted to get to, I think, a million words in the Brown Corpus. This is for kind of a general study of language and of the English language, by the way. So they collected these newspapers, some of whom don't exist anymore. Some of them have merged like the Atlanta Constitution, which is currently the Atlanta Journal Constitution, the Chicago Tribune, etc. They also got a lot of Christian texts in here. So the physician and Christian Christ without myth, Christian unity in England, as well as a lot of things that came from kind of general living at the time, including pretty interesting texts like America's secret poison tragedy, poison gas tragedy. I've been here before North County School cares for the whole child less discussed retirement and how to have a successful honeymoon. You can go and look at this stuff right now. By the way, I mean, if you're programmatically inclined, I don't know. Like how many people here know Python just raise your hand again. Yeah, you can do this today. You can go, you can install the natural language toolkit and LTK. You can import and LTK. You can get the Brown Corpus and you can select by these IDs here, which text you want. And so I did this here and in VS code and went ahead and joined sentences together and found this really interesting. I mean, this kind of things in which this, this modern AI technology is based on, right? My interests here. I thought it was interesting was who rules the marriage bed. And I need to read this because it's, it's hilarious in tradition and in poetry, the marriage bed is a place of unity and harmony. The partners each bring to it unselfish love and each takes away an equal share of pleasure and joy. At its most ecstatic moments, husband and wife are elevated far above worldly cares. Everything else is closed away. This is the ideal, but marriage experts and here's the conceits say that such mutual contribution and mutual joy are seldom achieved. Instead, one partner or the other dominates the sexual relationship in the past has been the husband who has been dominant in the white passive. But today there are signs of these roles being reversed in a growing number of American households, marriage counselors report. The wife is taking a commanding role in sexual relationships. It is she who decides the time to place the surroundings and the frequency of the sexual act. It is she who says I are nay to the intimate questions of sexual technique and mechanics, not the husband. So think about this and think about the ways in which we're thinking about the ways in which AI research and the kind of text and ontologies that structure are drawn from this very old data set. All the way back to thinking about words and how they're related to that to another and in the sexual conceits, even being placed into upon who should be the top in the marriage relationship. So that ends kind of okay. So a lot of the weird things that come from this are really embedded in this and they do shake out. It's not that we just take these word frequencies as in husband or wife relation. These weird words. So I keep on moving this thing around. Although I don't know. Can I move this thing? Oh, here. Nope. That didn't work at all. Nice. Okay. So this actually has pretty consequential ways for how this is. It isn't just husband or wife or marriage or bed. These weird words get into the data set itself. In a study that Kate Crawford AI researcher Kate Crawford and Trevor Paglin did in 2019. They looked at the person subclass of ImageNet and they found all kinds of words that we would regard to as arcane offensive. And not really something you would attribute to an image. So they found these words amongst them, including and I need to read this out. Bondswoman or Bondsmaid, closet queen, drug addict or junkie, failure, loser, non-starter, Jezebel, mistress or cut woman, second raider or wimp, chicken and cry baby. You see this kind of function happening in the data set too. You can go to WordNet now and look at this. Closet queen, which inherits from homosexual, homophile, homo gay person, which inherits from organism, living thing, whole object, physical entity and entity. And has this kind of parallel sense of a causal agent. This is pretty consequential and these different kinds of aspects of ImageNet were present in the data set until 2021. When as kind of a result of this excavating AI study, they removed many of these. But without really acknowledging how they got there to begin with. And so I want to move on to the second part of the talk, which is heroes emerge only in times of great need. And this talks about another part of the kind of labor that is more acknowledged and yet also under acknowledged in the construction of ImageNet and AI data. So you don't only just, what they did is that they didn't just scrape the data from the web. They also had to verify that these, what was in the images in the search were actually the things that were in the images. So they have 20 million images, probably more at that time, or sorry, 14 million, they're probably more at that time. And they need to sort through 14 million images. And I did the back of the napkin math on this because I wanted to try to look at all these images myself. And it was something like if you just spent one day, you spent three seconds looking at one image. It would take you, I think, and if you worked 24-hour days, it would take 200 days just to look through them if you just did it and you did nothing else, right? And so they also did that back of the napkin math and they found that to actually have PhD students to look at them, it would take them 19 years. Given that PhD students, you know, have to eat, they have to sleep, they have to take other classes, etc. At the same time, this tool came out and was developed called Amazon Mechanical Turk. Amazon Mechanical Turk presented this new tool in which you could almost get labor on demand, especially this crowd work labor. And with it, and I don't know if this is, this was the first crowd working platform, but it was definitely became one of the most popular. Remember Amazon early 2000s, still mostly selling books, getting into the cloud computing business, which is a major part of their business model now. And as part of it, this kind of I need for annotated data. So at the time, Dr. Lee said this was probably the largest single project on the Amazon Mechanical Turk platform. It involved 49,000 workers from across 167 countries. That's almost nearly all of them. You think about how many countries are registered, are recognized by the UN. And really played an important part in the economy of ImageNet. ImageNet would not have existed, but for the massive number of workers working on this. And the way they're kind of addressed in this data set is kind of funny. They talk about the kind of idea about this work, that it's kind of an amount of drudgery. They poke fun a little bit about this in a presentation in which they ask, are we actually exploiting chain prisoners in this? And they're like, no, actually most people do this work as a side gig. They do this because they're looking to gain a little extra income. And kind of in a bold statement on the right, they say, well, did ImageNet lead to the economic recovery after the subprime mortgage crisis in 2008? And they kind of show this ImageNet graph kind of as a reflection of investor optimism after that crash. But to turn this construction on its head, these places are people's lives. I mean, these people typically work from home. ImageNet workers are workers in an informal economy. They typically address those contractors. And Mimi Anuha, the artist, turns this construction on its head. So she shows these images of places where crowd workers are actually doing their work. These couches, these rumpled desks. And in an installation she called The Futures Here, she inscribes these kind of comically heroic types of inscriptions upon these places where crowd workers are doing work. So in the first panel with the couch, it says heroes emerge only in times of great need. And then in the other in which people are kind of doing this at a, I think this is like a cafe or a cyber cafe that's popular in places like Egypt and Kenya. She writes a world unlike any other. And so kind of in this construction, you know, there's a way that this work is very alienating, but it's also a place where people often go because they don't have much other choice. Because it allows some flexibility due to caregiving responsibilities or chronic pain or disability or any number of things. One way in which folks have tried to pull power back into the hands of M-Turk workers is a platform called Turkopticon. And Turkopticon was developed in the early 2010s by Lilly Irani and Sik Silberman as a way for people to actually review requesters, review people actually demanding and paying for the data, and actually to turn the eye towards them and saying, well, these people pay like crap, or they don't pay in time, or they don't communicate with you at all. In 2019, Turkopticon became a worker-led organization aimed at improving conditions for M-Turk workers. And so if you want to look at some of the kinds of, the archive is a little incomplete here, but if you want to look at some of the reviews that ImageNet as a requester got in its early days, this is actually after the development of ImageNet because it's 2012, but they were still using the same requester ID. They're pretty rough. So this first worker says, play a game of identifying birds and get paid and look at car images and identify cars. Both of them awfully underpaid. I think this requester is mocking us. I just did two hits and hits stand for human intelligence tasks, but no way to continue wasting so much time for him. So the pay is very bad, et cetera. The second one says, one of the all-time worst requesters on AMT has changed the requester ID to run from all the negative reviews. Do not be fooled at this same ImageNet AMT. See the other profile for real reviews and then this other profile, which I should say is dead right now. And then three, wow, they expect you to do six pages of tedious work identifying specific years and models of cars for five cents. I started working on one and returned it after I had already wasted 10 minutes on their qualification test that is included with your first hit. My boss, director, executive director of dare to make your brew use this tool to try to, because she worked in Lee's lab to address the cost different cars and Google Maps images. And actually found that their reviews are getting terrible and terrible. With quotes like apparently the virtual indistinguishable cars are super distinguishable and then, right? So you're trying to label cars, whether it's a Ford F-150 or a Ford F-150 XLT. And then I paid very close attention to the pictures and let the requester know that I feel these redecks were unfair. We'll update when I get communication back. I should have updated this with the slide because there's another issue here. But Turkopticon, I would be remiss, has been waging a campaign against mass rejections speaking of the rejects that they get back. Requestors have this one way ability to reject a bunch of work that requesters that they have done unilaterally and not get paid for it. So they are actually taking money and labor that's happening here. And I should also say we're going into day 11 of a unilateral decision by Amazon to probably suspend hundreds of workers, 30 that we know of right now. And we have been doing a mutual aid campaign with Turkopticon to actually get money to workers who need it because their money is being held by Amazon right now. And so if you want to contribute to the campaign, let me know, you can Venmo me or whatever. So I want to raise that. And so in this, we have an acknowledged economy of the M-Turk workers, but also not the power asymmetries in this and the way that they work in the economy of this data set, this data set that is structuring how we do AI research. The last part of this talk is called being a data subject and dating data full times. And what I really want to address here, and this is probably the most, how do I want to put it? This is probably the most, not necessarily the most radical portion of how I characterize work, but it is definitely a thinking on work that I think turns the head on how we think about privacy and how we think about the rights to our own biometric data. So after so much of this work in bias and fairness that has really gripped, I think, the AI ethics and the fairness, accountability and transparency space and computer science, one kind of corrective that IBM decided to do is they decided to release a data set saying that we're going to study the fairness and facial recognition systems, and we're going to do this in a way that is that is fair, diverse, etc. So they constructed a data set called diversity in faces, where in which they had a variety of people, different faces, different skin tones, genders, etc. Unfortunately, the data itself was taken from a Flickr data set without individual consent. As I mentioned, many of the images and image that are from Flickr and probably most of the other large facial data sets, unless they say otherwise. And so was this fairness data set, right? And so Olivia Solana NBC News, you know, prompted this pretty, pretty important article, Facial Recognitions 30 Little Secret, Millions of online photos straight without consent and parallel things happening in different data sets, including mega face, in which Casimir Hill, who has a great book on facial recognition and clear view AI, which is called, it's something like your face belongs to us, it's got this creepy title, it's great, but it's, you know, in a reporter for the New York Times wrote how your kids are performing surveillance technology and talking about non-consensual images taken from Flickr and other places on the internet, right? And so this is, you know, gotten so bad, it's, this is, this lawsuit I think has since been dismissed, but Stephen Vance and for a few other people filed a class action lawsuit against IBM, Google, and I believe a few other companies, but IBM was the main one as the developer of this, because they had this face representation in those data set. And under Illinois, one Illinois law, which is called BIPA, the Biometric Identification and Privacy Act, you can, you can sue. And they said, you know, this, the concern by Stephen Vance was subjecting people, subjecting them to increased surveillance, stalking, identity theft, and other invasions in privacy and fraud. We can even turn the critical eye here to Jay Crawford and Trevor Paglin by really identifying what they're using their data for and the kind of, even in this critical interrogation of ImageNet, we have to also think about what it means to take these images and display and redisplay them. In an article called excavating, excavating AI, the elephant in the gallery, computer scientists and a photographer, Michael Lyons kind of calls out this usage, not of the ImageNet data, but of this particular data set called Jaffe, which is the Japanese affect and facial, I don't know exactly what the acronym stands for, J-A-F-F-E, but the way in which those different faces in them of these Japanese actresses, mostly in the way that they are expressing kind of fake shock or different emotions for this data set, and that being then extended to be used in their own artistic enterprises. Another intervention here that I think is critical is this piece by Everpimkin. And I wonder if I can actually show this because I'm not on my iPad, so I could probably actually click this. Well, I'm going to do this because I think it's very evocative. I don't really get to show this in talks, so I am glad that I get to. So in this piece, Everpimkin shows the different ways in which these data sets are used and what it means to actually look at people and these data sets. So Pimkin creates this kind of mesh of these three second short clips of videos, which are classified into discrete actions by human annotators. And each video is slowed down, interpolated and upscaled immensely into imagined detail. The stream of videos is haunting. It has ghosts like the faces are rendered ghastly by the resampling process, but really commonplace by how we look at AI now. I mean, these kind of blurred out faces, if you generate something in mid-journey or stability AI, it produces this sort of same ghastliness and it obscures a lot as what it shows. So Pimkin planned on sampling only a small percentage of these data, but ended up watching all one million three second videos. And I don't want to imagine how long that took, three times three million, or three times one million divided by how many hours in a day, et cetera. They remark on their hours of watching the violence of these works. In the archive, quote, there are moments of extreme emotion and personal vulnerability, tears screaming in pain, moments of questionable consent, including pornography, racist and fascist imagery, animal cruelty and torture. And worse, I saw horrible images. I saw dead bodies. I saw human lives end. And so Pimkin taps into the heart of the matter here. Not only are these machines built upon categorizations of unacknowledged labor and classification, the unacknowledged and underpaid and oppressive nature of crowd work, in which one platform has a unilateral ability to shut down the platform and cut off people's livelihoods. There's also the kind of matter of the contents of these images of viewing this archive and the way that viewing the archive itself reinscribes some of the violence. And here I follow a little bit of Sadia Hartman, especially in her work, Venus and Two Acts, about the idea of reading into an archive and understanding and watching there. So many of the frames that people we have to contest in the archive are limited. We don't really have a way of getting images out of the archive. You might have some of that in the EU or US or some kind of practice of defending one's likeness, but we have to also contend with viewing this archive and being in this archive is a type of work and something that people have rights to. And so that kind of gets at the sum of this. Let me go back to this view. These three, I want to structure and proffer as all part and parcel as a type of cultural labor of AI data maintenance. Tools like mid-journey or stability AI or chat GPT don't grow magically from the head of Sam Altman or a man. I forget his last name or Ilya Sootskever. They are labored and toiled upon by many thousands of hands, faces that don't have names to them, mTurkers in orders of 50,000 ontologists, lexographers who have been lost to the archive. These things have to be surfaced, I think, in the economy of AI. And if they're not, then I think it does a disservice to what it means to intervene and build these things in a way that is community led and for people and not surely from a position of making tons of money for a set of five companies. Okay, on that happy note. Thank you so much. And I look forward to the Q&A. Thank you so much, Dr. Hanna. If there's not any hands raised there in person, I have one from the chat for you. That sounds great, yeah. All right, so this first asks, AI tools at the moment seem fairly accessible to everyone. In that accessibility, it seems to address an inequality issue. Do you think it could be generating more issues in the background? Yeah, I mean, thanks for the question. I really think, I think it depends because it is, one has to think about what it means to sort of democratize AI. This is a term that's used a lot. But what that tends to mean is that you're offering for free a set of tools that are still concentrated in the hands of a pretty small set of actors. We also have to think about what kinds of inequality are fomented in the process, right? So you might say I can write a movie script in, you know, 10 minutes, but the movie strip will likely be trash. And will also be at the expense of many, many, many writers in, let's say, Hollywood. These writers are not well paid. I mean, we think that people living in Hollywood are flush, or we think that actors in the Screen Actors Guild are doing well. The 90, I think it's something like 70% of actors in the Screen Actors Guild make less than $29,000 a year. Writing rates are quite lower. The rewriting rate is incredibly low. When it comes to writers. So as we saw from things like the writer strike, then, you know, you might be trying to open this up in some way, but it's going to shut down and has been built on the backs of someone else. So I would say that there's a way in which there may be an apparent to quality or level playing field, but it's always on someone else's back. Thank you very much. Yeah. Yeah, go ahead. Wait, wait, wait, wait. Hold on. Yeah. Do you want to hear? Is there a microphone so they can get a mic? Yeah. Yeah. You can use that or you can use this. Well, we had a question before, I think. I know that's my name box. Okay. Mine is actually not, not as involved. So I'm a professor in the humanities and I've been chatting with Dara on G chat. And this is a super fabulous talk. Thank you for coming. I work in deep humanities and also digital humanities. And one of the things that I really interested from you and also from our audience is the fact that you cross so many disciplinary boundaries so effectively here in this research, linguistics, digital media arts, photography, literature, creative writing and all those other things. I'm wondering if you can talk and help our students who are primarily from computer science here think about how you worked with humanities and linguistics and all this other stuff. Yeah, totally. So thank you. Thank you for the question. So my background is, you know, I have a pretty interdisciplinary background. My undergrad degrees were in a computer science math and sociology and my PhD is in sociology. And so a lot of what I've done, I think is, I think taking a lot of classes and different disciplines helps. I've taken lots of creative writing classes and classes, but I would also say fundamentally is thinking about what it means to accept as true what we would call epistemology. And that's how we know what we know. And I think a lot of it is really thinking about what does it mean to like, what does it mean to have knowledges that are very valid across different ways. So not only taking those classes, but also knowing that to understand that means to sort of unsettle what it means. So if you're in computer science, typically computer science research looks like doing a set of experiments on a certain method. And I've read many computer science papers. But also what does it mean to find meaning in a novel or in a set of art and what does it mean to for that to discover deeper truths. So I think that's, so, yeah, I mean, read. Getting on my LeVar Burton high horse, but yeah, read outside, read outside your discipline and read different things. Yeah. Hi, Alex. So my question is currently I'm also working on some AI related stuff like using machine learning to do the optical inspection for cosmetics. So one of my questions is because currently we always use, I think it's machine learning because we need to do the labeling and then change the models. But one thing that we are experiencing is how can we like lower down the type one or type two areas, especially just now when I look at your slides, you show that how you identify the facial recognition and also cause in that type of process, how do you change the model and then minimize it? Like just wondering based on your experience. Yeah, I think it, I mean, thank you for the question. I think it really depends on, you know, depends on the task, of course, and understand what, you know, if you want to increase, you know, your precision and recall, but I'd also understand like, I don't know exactly what it is, but I'd say if you're using, you know, some sort of thing and, you know, it has, it works across if you're doing things with faces that it's working across a lot of different faces, a lot of different genders and skin tones and whatnot. Right. I mean, that's one thing that's kind of a minimum, I think. And there's a lot of, I think tools, especially in facial recognition that I mean, I would say that I probably object to facial recognition more on kind of that it's used in law enforcement and whatnot. If it's used kind of in cosmetic development, it might be a little more of an acceptable use case, but yeah, I think it depends a lot on what that entails, but at least the kind of evaluation I think is very important. Thank you. Thanks. Thank you for the great insights, Dr. Hanna. So the problems are very, very brought out, but I was just curious if you can think of any better business model for the AI ecosystem to address these issues. Yeah. Yeah. No, that's a great, that's a great question. So I want to, I want to give a shout out here to one of the people in, one of the people that is a colleague of mine at DARE, his name is Asma Lashteka, and one of the things that he's been doing is he's working on machine translation technology for languages in the Horn of Africa, namely Amharic and Tigrinya. And one of the challenges that he encounters is that many of the companies say they have machine translation and automatic speech recognition technologies that work for people in those regions. And what he actually finds is, they don't work very well, they actually work quite poorly, and that sort of thing takes money away from smaller machine translation organizations because they say, well, Facebook already solved this, or Google already solved this, why do you need to do it? Well, they need to do it because they are actually sourcing the data much more ethically, and they're actually paying people and giving them a pretty clear sight line into where their data is going. And it's also done with a community formation in mind. And so he and a few other people have been working on trying to develop sort of like a union of African language startups that focus on developing precisely for their communities rather than one or two big models owned by centralized technology companies. So I'd love to see much more of that in the AI ecosystem more than just kind of one or two big tools. Yeah. Thank you. I think there's some questions in the chat. There are. So Hector would like to know, AI under surveillance capitalism is concerning. Are there any strategies that public libraries can do to inform their communities about it? Oh, I love this question being in a public library. Yeah, that's a really, really good thing. I'm not a librarian. I love librarians, but I am not one. I would say that there's definitely an element of public education that ought to go into that and to thinking pretty critically about cameras and about the data that's around us. There's a great thing. There's a great piece of work here by our data bodies. And they've got something called the digital defense playbook. And it's kind of a workshop, you know, that is meant to be oriented towards community discussions of data and data privacy. They've got a version in English and Spanish and I'm sure they've got a free PDF version, but you can get it at the store too. But it's really, oh, I see it's a PDF, but they've got really nice kind of workshop tools on this. So I think that's probably a really nice kind of way to facilitate that. I would also sort of question and think about, and I'll drop this in the chat, I would also kind of think about what kind of strategies, you know, that libraries adopt. And the ways that libraries are often, whoops, libraries are often can be sites of surveillance. There's a really nice book by Dan Green called the promise of access. I'm just dropping lots of citations here, but I am going to actually look this up. There's a really nice book here by Dan Green, who's an information scientist called the promise of access, in which he talks about libraries quite a bit and the way that libraries are often forced to shift and become and look a lot like startups when libraries are often kind of the safest places for, you know, our houses, communities or people who have no place to go. So it's a fantastic book. There's a bit on libraries here and Dan Green is also a really nice guy. So that helps us. Please keep dropping the citations. I'm writing them all down and we'll send them out after this once the video of today's talk gets processed with a little like intro reading list based on the things that you've talked about today. Absolutely. Thank you. I think Vic, did Vic have a question in the chat? Hi, Alex. Yes. So thank you. And again, like wonderful talk. And I just had a question and I'm wondering if you would be able to provide some insight. So we have seen that as academics and as researchers when we are using the generative AI for image generation and so on, right? We have seen that they lack fairness. They are biased. And then we saw the Gemini model, which came out, I think, last week, and we saw that there were a lot of people that tried to make it more diverse and ended up being wrong for different reasons. So what would be the direction in your opinion, of course, like how academics and researchers, we should approach using generative AI as back in for this kind of like image generation, text generation purposes. Yeah. Great question. I think we're just trying to figure out the, this, we can, we can, yeah, so I'll simply, you don't have to share anymore too. We can just stop the share. Yeah. Okay. Great. Oh, yeah, I'm back here. Okay. Yeah. Really great question. I mean, I think the kind of idea here is, you know, the correction here is, you know, Google trying to sort of swing the other way. And it's, I found, I want to have a, I have a, you know, I found, I want to have a, I have a meta commentary on this and then I want to get into it because I think it was really funny how Google responded and said that the, you know, the image generation of, you know, whatever racially diverse Nazis or whatever was, you know, like unacceptable. But, you know, Google also never said it was really unacceptable to have fairness problems in their, in their thing. And I think that really says, says something about Google more, more likely. I would sort of say like, I would step back and sort of say, well, what's the purpose of making representations of people and kind of thinking about what that entails in the economy of thinking about what these tools are used for. Why are we sort of representing people when instead of trying to hire people or actually pay people. So for instance, one of the cases is that has come up is Levi's, for instance, had a case in which they wanted to use computer and AI generated models to represent diversity in their models. And, and there was a lot of outcry, especially from people in the modeling industry who basically said, well, why don't you hire, you know, darker skin models? Why don't you hire plus size models? Why don't you hire disabled models? And so I do worry that the kind of this gets more at the heart of what the purpose of these models are and what kind of labor it supplants rather than the sort of, you know, the concern that a, you know, Gemini has sort of adopted a a politic that often right wing people don't agree with, which I think has a little bit a lot of where the reporting has gone. Thank you. Yeah, question here. Did you hear me? Yes. So I see these companies trying to integrate generative AI into like their core products, right? And it seems kind of like immature and I don't really understand like why I would want generative AI to write emails for me, because like, I don't really want to write emails in the first place, but I feel like I should write them. What do you see is like, like the right use case for generative AI? Where should we deploy it? What should we be doing with it? Yeah, no, I mean, great question. I mean, I am not very bullish on generative AI, obviously. I mean, I think there's a lot of cases and what is, and where it's done a pretty shoddy job of replacing things like writing your own emails or, you know, ask, you know, doing X, Y, Z. There are probably some better use cases here. I mean, I think I do think language translation can be helpful use case, but one in which it is assisting people rather than supplanting them completely. That can be helpful and can open up access. I think that some of the assistive uses are pretty can be useful. So if you're trying to provide, let's say, cases in which, you know, you're supplying a description to someone that someone that's hearing impaired or other division impaired and kind of maybe some other other cases for someone who's hearing hearing impaired. But at the same time, I am also not of those communities, and I don't know how much of a desire there is for that. And so I think there's elements of that that could be useful, but also often when those use cases come up, they're also done without the consultation of people in those communities. So I'm hesitant to say that there's like any case that is unilaterally good or bad. But I would say things like language technology with community control are pretty useful when they're done very intentionally and with kind of a focus on the data and the sourcing and the consent involved. Yeah. Yeah. Hi. There's one point that we can have models to generate synthetic data that can get past these biases on actual like manually chosen data. You think that's like a future possibility? There's already models that generate synthetic data and they have all the same problems. And so if you have underrepresentation in your basic models, you're not generating your synthetic data is still going to have those problems. Seb Boubeck at Microsoft has presented something like that. And but it's not even clear what those things necessarily will look like if your base data doesn't look good. And my colleague, Nyaling Morosi has also talked about all of the tools that we have to do that have to do with doing focusing on underspecification. Still don't get around the point that you have this long tail of data that doesn't exist or is very under sampled. So, you know, what that what I see in synthetic data, the danger is that you're going to perpetuate stereotype, right? You don't have an adequate representation of black people in your data set and you have majority white engineers coming and saying, well, we need more synthetic black people in the data set. You're going to result in something that looks very stereotypical or or kind of overweights particular examples, right? So, yeah, that doesn't actually represent your diversity in any kind of meaningful way. Yes, I'm great at taking screencasts. Okay, any other questions? If there's none in the room right now, I'll ask on the flip side of under representation, how do we deal or can we deal appropriately with AI with problems of over documentation and over representation? So, for example, indigenous folks are way over documented in lots of really negative ways. Do we never over document anyone in a good way, right? So, how do we or can we deal productively with that problem? Yeah, I mean, that's a great question, right? I mean, when you have kind of an over representation in the data set often, often not done in a way that is representing people faithfully or how they want to be reviewed. And Joanna Radden has a wonderful article called digital natives, which is kind of about this data set on the, that indigenous people living on the Pima reservation in Arizona and how they have been over studied for this kind of above average incidence of diabetes, but very little of and that data set now is actually on the big UCI data database where it has a bunch of toy examples, right? And it's on that website because people thought it was just an interesting multi-variate classification problem, right? And but at the end of the day, people living on that reservation have not gotten any of the benefits of that, right? So over represented in the data, but those data have been abstracted, decontextualized, and now they're just kind of on a toy machine learning website without any context, right? And so I think a lot of it has to do with governance, has to do with control, like what are these things actually being used for? One of the models here in talking about indigenous communities has been Teheku Media. They focused on building automatic speech recognition and machine translation tools that have focused on really, really, really making tools for their community and not releasing the data unless it's to kind of trusted partners. And so they've got a nice piece and I'll put this up in the room and I'll share it in the chat, but I think it's on OpenAI's Whisper. So OpenAI, I think it's called like OpenAI Whisper. I'm just going to search colonialism. Yeah, so they've got this nice blog, but they don't write a lot of papers because they're building a lot of tools, but it's called OpenAI's Whisper is another case in colonialism and talking about that kind of re-inscription. I keep on pressing, whoops, wrong link, I keep on copying wrong because it is Windows and I am not used to it. Thank you, Christy. And so I think that speaks a lot to those concerns and kind of that over representation. Oh yeah, why is my camera keep on going up? Yeah, but thank you. Great question. There's a 9% here. Oh, hi. So I just want to follow up with the questions that the previous student also asked. So because currently we are, like for deep learning is more about neutral, I mean neural network. So the model is changing by itself. So let's say for GPT. So every students can put their input to say, like, oh, the information provided by check GPT is one of, I mean, true or false. Then in that case, how, how is the, the judgment or decision, decision tree that is made to say, like, after you say, you're wrong and then how, how, how's the response, like, what's the looping about for the decision? I don't think I understand the question. Can you give me an example? Okay. So let's say sometimes, like what the response from GPT, you can say, oh, I think your answer is wrong. Then like, it's kind of like the correction during the training process, right? And in that case, sometimes it will say this, oh, my fault, that that is incorrect. And I apologize for things like that. Right. So I'm just trying to understand how the, the deep learning process it is, like when the trainer is saying, like, this is wrong. I mean, I think the way that they do that is they, I think a lot of that is done through, through reinforcement learning with human feedback. So the idea that you can provide this kind of correct response and then that deactivates a certain set of neurons in the, in the huge architecture, right? Yeah. I mean, that's my understanding of the process. And so you give it some things to test against. And then, you know, so I think that's, that's my understanding of that, but I'm not also not an expert in the RLHF. Yeah. But thank you for the question. Any other questions? The food smells really good. There's, yeah, there's no other questions in the chat unless anybody has any other questions. I don't think so. Okay. Well, then I'll just say thank you again on behalf of the Circle community, our partners and friends. We're so, this was wonderful. We're so delighted you were able to make the time to be with us today. And we hope everyone will enjoy lunch and have wonderful conversations there in person. For everyone who joined us online, we hope to see you for our next talk and to be able to continue to share opportunities to engage with wonderful speakers like Dr. Hannah. Thank you, Dr. Hannah. Thank you. Thank you.