 So, there's already a theme going on, which is this whole West Coast, East Coast thing, and I think it's probably suitable to call out and recognize our sort of West Coast sibling center, the Stanford Center for Internet and Society. Are there Stanford CIS'ers here? Ryan, Kalo, I see. You might see Barbara Vancevich come a little bit later. It's just an amazing center that's doing incredible work, including a number of cases being brought to defend the rights of fair use out in the courtroom on the front lines, and so just really great work being done there and so great to see the two centers coming together. One representation of that will be a course that I'll be teaching in January, which is a joint Harvard-Stanford course that meets here at Stanford, like that wasn't a tough call to make. The Harvard students will fly over for it, and they'll be here for three weeks, so if you see like pasty-faced people walking around looking confused, they're probably the Harvard students in the course. And we're going to take up four difficult problems of cyberspace, and we're taking nominations for difficult problems. So if you've got a problem, it's kind of like this course is meant to be Lucy at the desk. For five cents, we will try to see if we can make some progress and give some advice, which will be at least worth five cents, but I'm going to talk about micropayments shortly. And I guess maybe that breaks us over to the topic I wanted to talk about today. And there is also a kind of West Coast, East Coast thing going on here, because when I think of the West Coast, I think of optimism and adventure and the crashing waves of the Pacific Ocean that I've seen at least once since I got here on television. And when I think of the East Coast, there is this sort of reputation for a little bit of pessimism and such. And so I want to start by giving my most optimistic view of a set of phenomena for which I don't even know if I've found the right label for it yet. It used to be ubiquitous human computing, but it didn't fit on the slide, so I made up minds for sale. And then, oops, excuse me, what got me thinking about it, among other things, was this. This is a tween bot created by a grad student at NYU named Casey Kinzer. And what it basically is, is a cardboard robot with an all-important smiley face pasted on it. And it's got a radio shack motor in it that can only propel it forward at a constant non-threatening rate. It can't steer, and it has a flag on it with an all-important first-person message from this anthropomorphized friend that says, I'm trying to get to X, can you help? And she then took these robots and placed them on the streets of New York City. Kind of daring thing to do. You would think maybe people would call Homeland Security or something. But no, as a matter of fact, people actually helped the robots get to their destinations. So this is one chart of a tween bot being deposited in the northeast corner of Washington Square Park, and over 40 people intervened along its route to help it get to its desired destination in the southwest corner. I found that extremely intriguing, in part because it's not a bad metaphor for how internet routing works. I don't know how portable this is. It's like, what lesson do we learn from it but that you can occasionally get people in Washington Square Park who might already be feeling a little funny to help the robots get to where they're going. But it's this kind of process, the idea of trusting the world out there to do the right thing and to actually bank on that to get something done that formed the optimistic part of my book on the future of the internet and on many other books as well. There's kind of a lot of excitement about, I'm not sure I would call it mass collaboration. I mean that has a kind of Vichy France sound to it, but whatever. But there's lots of excitement about people out there being able to come together or have their work agglomerated to do good things. And again, I share that as well in the optimistic part of my book. This was the original cover of the book of the future of the internet from Yale University Press. I was not blown away by this cover in part because the very month that this was going to come out, I actually had another book for which I was a co-editor coming out that also featured a large hand saying stop on the cover. And while they are a left hand and a right hand, it did not seem right that I should have two bucks with large hand saying stop. So I asked Yale if they couldn't come up with a different cover. And three weeks later, they gave me this. And I, again, I was like, well, I respect the 8-bit font that's sort of evocative because after all, the book is about computers. I'm not entirely sure about the large traffic light. I mean, the word stop is in the title, but I otherwise don't quite get it. I was fearful that the next round would give me a coaxial cable with a spark coming out the end, which would just end the project. So I figured I should make the medium the message, and I put it out to worth1000.com, as in a picture is worth 1000 words, where it turns out there are a ton of people in the world that have really good photoshopping skills and nothing else to do. So they go to, you know who you are. I'm sure there are several people in the audience that fit this description. They go to worth1000.com and they participate in contests. Are you worthy to see if they can produce good photoshops? And some of them even pay. So I think I staked $150 on a contest that would be for the book cover. And I got a number of entries in from different people around the world suggesting what the book cover could look like, some of them good, some of them less than good. And yeah, well, so some good possibilities. And then I saw this one and I really liked it. This is from a guy named Ivo van der Int in Holland, whom I've never met and probably never will. It just seemed only to carry the pessimistic thread of the book. And so I asked him if for another $50 paypal would he have any problem with maybe doing a different version. And he said that would be fine. And so that became that. And that is the actual cover of the book that shows that there's some choice in the matter, the optimism and the pessimism. There's sort of the, I don't know if this is the West Coast or that's more the West Coast, the kind of calamitous falling into the ocean sort of thing. But we digress. I don't mean to analyze my book cover for too long, except to say that I offer a credential as being excited about the prospects of putting work out to people in the world to do. And then magically the next morning, having 30 entries of covers to the book, all of which are kind of better than what the Yale University press people were doing. So I won't be publishing with them again, I guess. So then being an academic, I thought I should start to taxonomize these services and means by which you can put work out to people to do, especially because I started having a few qualms about them. And I know I think among us tonight, there are people who are behind some of the companies I'm going to talk about. I'm eager to engage in dialogue and hear what you have to say. But first, for the rough taxonomy here, I figured basically there are some ways of putting out work where the skill set you're trying to harness is so rare, so abstruse, that they're going to be up here. There's very few people that can do it. And maybe you're going to have to pay a lot to get that task accomplished. And then there are other skills that are much more common. And you can draw in a much broader base of participation. And I wanted to basically give a tour of this pyramid as it's starting to shape up, because it may, our instincts about what's good and what's worrisome may differ from one section of the pyramid to another. So let's start at the very tippy top. I would think something like the XPRIZE, just down the road at Moffat Field, the XPRIZE Foundation, they want to stake an award on some particular thing that people might choose to accomplish. And maybe it's to put a reusable space vehicle up and bring it back down again. And with that, you could win, I don't know, a million-dollar prize if you do that. It turns out that they then say that over $10 million of investment happened among companies trying to win the $1 million prize, which is like, cool. You were able to inspire a bunch of innovation. Although it does feel a little bit like Swoop-o. Does anybody use Swoop-o? If so, can I talk you out of it? I think the way Jennifer Granick uses Swoop-o? So Swoop-o, I can't believe I'm now advertising Swoop-o. It's the new kind of auction for which what happens is the bids go up in increments of just a penny or two. And here's six seconds left on a Buffalo 1 terabyte drive station. It's about to go for under $10. The only thing is it costs to bid. So you click the bid button, and that'll cost you $0.60. But then you're the top bidder, but it also resets the clock for another 30 seconds. And the clock can get reset indefinitely as people bid up in pennies. And all of them pay $0.60 as they go, which means you might end up bidding more than $1 for $1 because you thought you were on that last marginal bid, but you weren't, and then you have to keep going again. Anyway, I didn't mean to go into an entire deconstruction of the XPRIZE Foundation, which I think is doing wonderful, great work and is completely unlike Swapo. So I'm going to get rid of that slide. All right, let's move a little bit further down the pyramid here. And for that, I would look at something like Innocentive. The pharma giant Eli Lilly comes up with a marketplace where I'll let them explain it how it basically works. You have companies over here represented by armless people in front of buildings. And through a very old fashioned laptop, they will offer awards to scientists who are arranged around the world in a kind of construction paper silhouette, small world after all format. And they are the solvers. And they solve challenges in exchange for the bounties placed on the problems. And the kind of problems you assign through Innocentive, they basically fall short of something Nobel Prize worthy. But they're more than like you just got to know Photoshop. So here's a good example. Browning in juice, $20,000 to the first person that can come up with a good way to avoid that problem that you haven't thought about, but now you realize you've noticed that Tropicana juice in the clear bottles that wasn't meant to be refrigerated, it can turn brown after a while. It's still drinkable, but who would want to? So if you can solve that problem, there's $20,000 waiting for you. 270 project rooms have been open. That means 270 people have started the process of communicating. Perhaps anonymously with the seeker to try to offer them that solution. And the first one across the finish line, whose answer is accepted by the deadline, ends up collecting the $20,000. So it's amazing to see the kaleidoscope of projects available in the science and applied technology categories for a bounty. OK, so under that, I might put something like LiveOps. LiveOps is, as you can see, pictured here by the computer monitors tacked into a grassy field wearing headsets. It's kind of confused about whether these are people or machines. That's the point. It's a virtual call center, where you can add more call center operatives as easily as with cloud computing. You could have more servers or even virtual servers thrown onto the rack to help you if you get slash dotted. And the story that LiveOps tells is kind of the most evocative story of how they work is that when Hurricane Katrina roared through New Orleans in 2005, the American Red Cross announced a hotline, a 1-800 number, for people to call who were concerned about relatives or who are the relatives about whom concern is warranted. And it was quickly swamped. So they called LiveOps. And LiveOps was able to establish within several hours thousands of people who could answer the phone, hi, you're calling the Red Cross, how may I help? Just like that. And how does it work? Well, there's not even a boiler room somewhere. It's in the privacy of your own home. You can apply down a mechanized runway to see if you could qualify as a LiveOps contractor. You have to prove that you have a good internet connection and a headset, and you basically know how to use it. And you answer a kind of SAT or GRE-like questions where you do reading comprehension. This is like what the SAT is really all about. So you read a script, and then they'll ask you questions about the script. So what are the, and I appreciate that they put it into scare quotes, benefits of the weight loss super machine package. And if you can answer the right way, it scores you points. And eventually, among the so-called, I think, 3,000 people who begin this process each week, a handful, maybe 30 fall out the other side, ready to be interviewed by a human and perhaps made a member of the LiveOps family. And when you become a member of that family, you basically start as like a video game. You're at level one, and you're taking pizza orders. And you're just the pizza shop. You get the script, you know, thank you for calling me. What would you like? You type it in, boom, you hit send, it goes to the pizza shop. And as you succeed at that, other levels might open up for you. So you can then maybe be the drive-through person at the Squawk box behind the McDonald's and somebody asks for something. And because you are not dangerously near a fry later, you can upsell much more gently and easily as you type in what they want, hit send, and it goes right back to the restaurant where the car is idling, where the employees then prepare the food. So that's why they have ads like this. Calling all, this is a wonderful new word, mompreneurs because you can be in your house with your Diet Coke and your computer. And I think just off to the side, there's a crib with the baby. And if the baby should start mulling, you can jack out of your call when you're done. And then like, that's it. You don't have to work hours. It's flexible to the level of the transaction. And when you're done, you go right back in again. So it's almost like virtual, fungible, scalable on both sides, not just from the point of view of the company that has the agents working through LiveOps, but also from the point of view of a given agent. You can do it for five minutes a week or 30 hours a week if you want. And it's entirely up to you. And you can just go along. And they even say it's not just limited to this kind of American framework. There are companies that, whether through their own forces or by rotting on top of other services where they then basically end up subcontracting. Here's an opportunity for Kenyan refugees who are in a refugee camp to do what we might call micro work and to be able to help. And in fact, this, through a company called Cloud Prower, offers now an iPhone app that if you would like to be helpful, you can run the app and double check the work of the refugees to see if they're doing a good job. And you don't get to fire them or anything. It's just you're saying whether whatever the answer was was right. And then that's a nice way of being able to donate your own time to help people here. Now, what kind of micro work might they be doing? It might be translation tasks. Or it could be something of the sort that's assigned as we move lower in the pyramid through a service called Amazon Mechanical Turk. Now we're here in the heart. The actual epicenter of Silicon Valley, I think, is right down there. So my guess is, how many people have heard of Mechanical Turk? Right. Can we just be smug for a minute? Because the rest of the world still hasn't quite heard of Mechanical Turk. I love how it's artificial intelligence. It's the kind of task that you wish you could farm out to an AI to do, but for which deep blue just isn't deep enough yet. So why don't we get people to do it in a way that we can treat them like the rack space so it's as good as artificial intelligence. It's artificial artificial intelligence. Of course, it's called Mechanical Turk after the device that is supposed to play the robot. This is a robot character that plays chess, and you can see the gears inside. And of course, really, it turns out there's a poor cramped human inside in one of the first literal sweatshops playing chess for the Mechanical Turk, but you're not supposed to know that. This is the artificial intelligence. That's the evocative thing. So what might you be asked to do? Here's a classic human intelligence task where you are asked at a penny per task to provide related keywords for these images. Now the thing is, you look at this, and you can't help yourself. You're like, box, box, box. It's like, yes, it's a box. And you're like, cool, and then it takes you to the next one. You're like, trucks, trucks. It's like this dopamine kind of, it's like just the rat is pressing the bar and the pellets are rolling out. And you're just like, I am drowning in pellets. Like life is great. And there are grad students who sit and do this all day long, I think because it's probably the only source of positive feedback in their lives. So you sit and label the images, you get paid a penny, and then somebody's images are labeled. It's not even clear for whom you're doing it, but who cares, right? That part of the task is just to disaggregate the person tasking it and the person or people doing it. Here's another mechanical turk task for three cents. What is the difference between vanilla and French vanilla? Like yellow? I don't know, what's, Baskin Robbins has cost a bit more, but worth it. Like in the early version of price discrimination, but you're asked to keep your answer between 50 and 60 words long. You're alerted that it will be reviewed for quality by an editor, probably another turker who's asked to edit the following responses to these questions, and it has to be 100% unique, which I mean the academics among us are, there's no such thing, we stand on the shoulders of giants, but it's only three cents, just write the damn answer and move on. Now for all we know, this is somebody asking for an answer to the question in real time. There's somebody like at the counter at Baskin Robbins like please, what is the difference? And then the answer comes through after being reviewed by an editor. In a wonderfully recursive and meta task, this is a real favorite, they asked turkers, paid them a penny to write on a piece of paper why they turk, take a picture of themselves and send it in. So here's I turk for making money in my leisure time, I turk for the lulls, and I turk for drug money, just kidding. And what I like about this is it's kind of reminding us, like there are real people behind the way in which we treat it as just a mechanized, artificial, artificial intelligence kind of thing who are out there doing this stuff. Are there people who use routinely mechanical turk who would consider themselves turkers among us? No, not yet. Anybody considering doing it now? All right, we recruit a few, I'm going to try to collect a commission on that. So you might think, what could possibly be lower on the pyramid than being paid a penny to scrawl something on a piece of paper and hold it up and take a picture? And the answer is well how about if you get paid nothing? We instead make it a game. So the brilliant Louise Vaughn and Carnegie Mellon University came up with this thing called the ESP game where player one and player two are each, never going to beat each other, but they are shown a photo and they are asked to guess what the invisible other player might also be guessing as a label for the photo. So, you know, they start guessing and eventually they agree on something and it's like success, you each score a point. Now you may wonder what can you do with these points that you score? And the answer is you accumulate them and it's almost like the Ski Ball tickets. Do you remember Ski Ball at like Coney Island or something? And you'd have like this huge basket of tickets that you'd stagger to the counter where they'd be like, eh? And they'd be like, you can have a miniature porcelain dalmatian. Like, wow, I guess I should have put a few more hours into Ski Ball. So it's just like that except you get nothing. The tickets are not bankable for anything. Right, so why not play? So it turns out that people love playing the ESP game and accumulating points. In fact, Louise says he's managed to create 4.1 million labels with 23,000 players and there are many people that play over 20 hours a week. He says in fact that his advisor for his thesis made him put in a gatekeeper so that if you were using the game from a .edu address, you would be capped at 20 hours. It would just say like go back to work on your thesis and that would be it. Which in a way is kind of a thumb on the scale for American academics because they didn't do it for .ac.uk or other institutions around the world. So he then just runs some numbers and he's like, you know, 5,000 people playing simultaneously could label all the labels images on Google in 30 days and you know, games you've never heard of and services you've rarely heard of. No offense Abigail, sorry. Average over 5,000 players at a time. And so he's like, this is easy to do and sure enough, right? Google's like, we're gonna buy you, right? Now they say that all the time. I know, but they said it to him and they bought the ESP game and part of the success of Google images and the related images and the other stuff that's a form of artificial artificial intelligence is coming through people playing the ESP game. Just recently, a wonderful paper came out of human computing for electronic design automation. What's electronic design automation? That's a task like where you're trying to figure out how to cram the maximum numbers of transistors or equivalents on a particular section of a circuit board and how to wire them in a way that shortens the distances most efficiently and you can have computers try to design chips in another kind of weird recursive self-aware terminator way. But they at some point hit a limit onto what they can do. So these folks have figured out a way to take the puzzle of how to do a task like cramming the transistors tighter and they've made it functionally equivalent to a game where the goal is to light up all of these circles green and how do you light up the circles by pressing these buttons that surround the circles? And if you press the buttons in just the right way you can get all the things green. The makers of the game don't know the answer but it's out there and so people just play the game in order to light the circles and when they do they score points and in the meantime design automation is taking place. So what could be below this on the pyramid? Well I think maybe below this is just when you are existing in the world maybe your labor is helping somebody else. You don't have to detour from what you're doing and play a game. So for many search engines some who knows how big but some part of their rankings are determined by linking behavior from one site to another with the rough metric being that sites that tend to get a lot of inbound links might be sites that are deemed by the world to be credible whether they're coming from hard links in a website or comments in a blog or something like that. And so then the Google robot can come along and note those links and basically use your work to help everybody else and if that might be seems I don't know how Marxist people are feeling today. If you're feeling like that's some form of exploitation which my guess is no one is feeling Marxist today. We are in the epicenter of Silicon Valley. But if you were feeling Marxist what I would say is yeah but look at something like Google flu trends which again is just being able to aggregate anonymously people running searches on flu and flu related words in order to actually be able to track the progress of the flu from one state to another or even around the world. Very amazing stuff for which people don't even have to be playing a game or otherwise altering their behavior in order to create social value in the world. Okay, so that's the basic tour of the pyramid. What are my worries about it as I'm starting to think about this? How is the East Coast creeping into things? I've basically carved them into two rough areas. One are worries that I might try to hold on the participant's point of view. And it's the kind of stuff that maps a little bit to the kinds of concerns that give rise to labor law. The kinds of reasons why we have a minimum wage, why we have if you work somebody enough hours you're supposed to give them benefits. We have rules against child labor. Now again maybe you think all of these rules are bad rules in which case you should be perfectly delighted by what's going on out on the internet that I've been describing because these are all contractor relationships that don't invoke any of the protections by and large that we have come to understand in labor law. But if you think those protections are worth something, if you would be against repealing them in real life then you might stop to ask, well wait, don't we maybe want to have them under certain circumstances? Who knows what those might be in their online counterparts? And then systemic to me is even more interesting. Sets of concerns that are kind of about society itself. So let me walk through them a little bit with some examples. I mentioned child labor. This is the PBS Kids site. As soon as you land on the site, it's like come play games. And okay, that looks fun. So let's just change it a little bit. Imagine that that is one of the games that's available. Do you have a problem with that? Can you imagine your kid playing this amazingly addictive game? It's like Color Me Hungry is worth 10 seconds. But this thing, this has some depth and nuance and your seven year old is sitting there drooling, clicking on stuff for 20 hours a week doing what may be electronic design automation but is actually front-ended from PBS Kids. How many people are, I'll try to do it in as non-identificatory a way as possible. If you have some measure of qualm about this scenario at the count of three, I'd like you to hum. So some measure of qualm, hum at the count of three. One, two, three. Okay, that was an extremely tentative hum. It's like I'm concerned but I'm more concerned than other people will find that I'm concerned. How many people are not concerned at all? It's like this is just fine. One, two, three. All right, a slightly more confident hum and a lot of people who didn't hum either way still trying to think about, I don't know, it's hard. I don't think we have a good way of thinking about this. This isn't just like somebody being worked on a farm or in a bakery extra hours. So it's hard to come up with a good instinct about it. Here's another innocent of task. Some anonymous institution is seeking pyrazolopyridinol diazonines. And you're like, what are those? Which answer is if you have to ask, go back to the browning and the juice. There's nothing for you here. I don't know. I don't know. Sure enough, there are five project rooms. This isn't like high volume stuff. But then you're like, what do you do with this stuff? Is it a poison? Is it a precursor to a bomb? Is it, I don't know what it is. Who knows what it is? All we know is that somebody wants some and there are people that are out to make it. And that starts to get you wondering about what it means when you labor in such ignorance of what you are doing that might make the labor less satisfying. In which case you might just say, well then, don't do it. And the other part of it is, what if you're laboring for a cause or an employer that you actually actively would not want to help? And for that, let me develop a hypothetical starting with non speculative stuff. So let me anchor this hypothetical with this phenomenon from 2006. The state of Texas set up a webcam, a number of webcams along the border. And the webcams are just sitting there. And then they invited people to stare at the webcams with the idea being if you saw anything suspicious, you should click, I see something suspicious. That would get sent to other webcam users. And if enough people turn their keys, the missile launches and a police officer goes out and checks out whatever it is the web users appear to be excited about. Now this is weird, but it's like, you know what you're being asked to do? Millions of people did this. Millions of hits at least were generated. No doubt many of which were people just checking out what the fuss was about. But others were known to be clicking the I see something weird here and spend a lot of time on the site. Here's smart drive, which is not a USB thumb drive. Instead it's literally about driving because what'll happen is you can install in your driver fleet of vehicles for your business and accelerometer. So if anything weird happens in the car, it gets recorded and immediately reviewed in India by somebody who then tries to identify backing, breaking, unsafe lane change, et cetera, et cetera. It all gets noted and sent to fleet operations and boom, you are pegged by a human judging your actions in near real time. Now good or bad, I don't know if it saves lives on the highway, maybe not so bad. But an example again of mass surveillance enabled by this, this is supposed to start this month in the UK of course, Internet Eyes, where the UK has more CCTV per capita than anyone else and of course the CCTV isn't just run by the government, there's lots of private CCTV cams but not enough people to watch them at the rates that people who own the cams would like to pay. So you can become a customer if you've got these cameras and ask that people randomly around the internet watch your cams for you and press a button if they see anything wrong. And here is an opportunity to become a viewer, here is a typical viewer enjoying herself on the couch staring at a cam for hours on end where she can earn reward money. I love this, this is wonderful British understatement, have a chance at reducing crime. When I was at Oxford, the Thames Valley police on the side of the car where like on an American police car would be like to protect and serve or we'll kick your ass or something like that. The British one is reducing crime disorder and fear. I'm not making this up, it really says that. I'm like, yeah I feel a little less fearful. I'm still terrified but it's a little less, like thank you Bobby, I appreciate that. But then it's a little more overstated and potentially, potentially become a hero and save lives. You sign up for free, you watch and report crime. So kind of interesting, I have signed up and I'm waiting for my introductory packet. I'll let you know what I find when it gets going. Back here in the States, again from 2006 at the University of Colorado on a key significant day in April, they have the great marijuana smoke out. So the bunch of people assemble on a field and they all smoke marijuana and there's not, there's so many of them you can't arrest us all, what are you gonna do? And they don't, instead they take pictures of everybody and then they put the pictures up on the web with a $50 ahead bounty for anybody who can identify the person in the picture. Now, I don't know if you think that's fighting fair but at least it has, I suppose an upper limit as exemplified by a more recent incarnation of exactly this phenomenon. This is a Persian web page that went up not long after the recent election protests in Iran where you are asked to provide any information you can to identify these protesters. Now, if you think on them, I think it's pretty clear what will happen. You'd be under no illusions as to why the state wants to know who these people are. So there might be an upper bound that the people most likely to identify them are also the people least likely to want to turn them in especially there doesn't even appear to be a $50 prize for doing so. All right, so now here's where things get speculative. So far this is all real. The speculative piece is, what if we took a service like Mechanical Turk and farmed this task out in the following way? This is my back of the envelope calculation. There are 72 million people in Iran. Our first task will be to take the identity card photos for each of them in the National Data Bank and just divide them among men, women and children. To do that, we put it into Mechanical Turk and it's a simple task, you get paid a penny, look at the following five photos and just label them man, woman, child, man, woman, child and you get a penny for each one. You have no idea why you're doing it, you're just asked to do it. And then we have our divided database. Now we take a photo of a protester we want to identify, we create a new task, we put that on the left and we take one of the photos from our matching corresponding databases and we put a number of those on the right and we simply ask people in a penny a pop, does anybody on the right match the person on the left? And by my estimate, if we do that, basically $17,000 or so will identify any arbitrary protester in a country of 72 million people. And if you're feeling like you're kind of a McKinsey type, you can no doubt make this much cheaper. You can come up with also, you're like, oh wait a minute, I got even a better heuristic, why don't we, right, you're already like, oh wait, we should concentrate on the cities, people in Tehran, right? They could do this and the people making the identifications in India, in the United States, idly doing this on their lunch break instead of mind sweeper would have absolutely no idea the implications of what they're doing. And that seems worrisome to me. Okay, so that's about moral valence concerns and not knowing who's commissioning your work. Another set of concerns has to do with the fact that often so much what we're excited about enabled by the social web, but that also has had echoes in real life for the longest time, is that there's a whole realm where we use other people's judgments to help us know what is legitimate and what is not in the social and civic spaces. And through Mechanical Turk, you see a growing number of tasks, for example here, reviewnatmed.com on your blog at $2 per hit, a pretty expensive hit. Here's what your blog has to be, basically real, and then you have to write it, it has to be unique, it has to be permanent, you must link to us, and you can't disclose that this is a paid review. So that's an example, now nowhere does this say it has to be a good review. But note that in Mechanical Turk, it's entirely up to the commissioner of the task whether to pay you after you've done it. And are you really gonna like litigate up the chain a penny task that you didn't get paid for? Like that's, if nobody's paying you to litigate it, that's not worth it. So there's a lot of power on the part of the commissioner and you know that as you're doing the task. You can actually see if you don't own a blog, you can at least comment on others. This is just a quick search for comments in the tasks and post comments and replies to a website, comment, rank, and subscribe on YouTube's videos, sign up and confirm your email, artificial, artificial intelligence, all the barriers that might exist on the web to prevent a robot from commenting on something including a link back to a spammy website are exactly the things that you could ask people to do at very low prices in order to get around those barriers that ask if somebody's a human. And I'll talk more about that in a couple minutes. This is the most naked one I've found. Write a positive five out of five review for a product on a website. Use your best possible grammar. Write as if you own the product and are using it in the creative writing part, it's tell a story of why you bought it and how you are using it and then on your way out, mark any other negative reviews as not helpful once you post yours. This turned out to be for a Belkin router on Amazon and word leaked out to slash dot. And so if you go to the page for the relevant router, you will see alternating five star and one star reviews, one star is coming from the slash dotters, five star is coming from the Turkers, none of whom have bought the router, right? It's a little weird, right? It's like, wow, 570 reviews but nobody owns the damn router. Or if one person owns it and reviews it, you're like, I can't trust that review, right? Somebody paid them unless it's three star. Who pays to write a mediocre review? So very puzzling phenomena and I don't take as a lesson from this, oh, you wouldn't possibly use this to generate good reviews because you'll ultimately get caught. I take from it, you should be a little more subtle than this in giving people their tasks. And I see something like dig. I'm sure there are a number of diggers among us, perhaps the founder is here for all I know, and there are opportunities to rank stuff by voting on it as you surf the web and then you can visit dig.com and see what's been all the talk of the web. So okay, that's great, that's social reputation at work. Here's the corresponding website, subvertandprofit.com where if you are a digger, you can sign up and what will happen is if you are an advertiser, you pay $2, I think I just lowered their prices, but the basic one was you pay $2, $1 goes to subvertandprofit.com, $1 goes to the digger to dig, hold the nose and dig something that's not that cool and then go back to your normal digging. And so just imagine you use dig and you use it more or less for fun, you just do it. I don't know, there are diggers among us, how many people dig? All right, only a few. So imagine that you are a digger and you're offered at that moment a buck, a cold hard dollar to just click on that link and it will be deposited into your account and there's a promise of more where that came from, how many people would do it? All right, the rest of you don't wanna admit it or you're ready to negotiate, like two bucks, okay, maybe. If you go to subvertandprofit.com, you'll see they've signed up a number of services, it's not just dig, the big missing one is Wikipedia. My guess is the Wikipedians are crazy enough that you can't buy them. They're just like, no, the honor of Wikipedia must be maintained, you cannot pay me, but you can ask me nicely. Like that doesn't make any sense. So that's another example. So imagine those nascent ways of subverting social media and reputation ranking applied to the political sphere where members of Congress and others try to gauge where they're at with their public. How do they gauge phone calls coming into the office, what happens at a town meeting, constituents who come into their local constituent service offices and polling? Each one of those, if you put on your McKinsey hat, is eminently subvertable through these intermediaries that we're talking about, whether it's LiveOps or Mechanical Turk, like services on either layer, you could see somebody being conscripted to call a member of Congress, it must be yours. It has to be from your district, you identify yourself and you get extra money if you keep them on the phone at the office for more than five minutes. And here are your talking points, but it must be your own words. And you realize then that somebody willing to write a big check could, just like the Hurricane Katrina story, within three hours, generate an arbitrary amount of outrage or support for any particular proposition through any of the channels that are normally used to gauge authentic public reaction. And what we know members of Congress have done with email, which is to say dismiss it entirely because it's so readily spammed and just echoed with so little effort by the constituent, now you can invest that money and have fake motivated constituents put in that effort, get them to protest a meeting or not however you like. I find that extremely worrisome as far as the potential for political abuse. I mentioned I would get back to the artificial artificial intelligence piece of things. Louise Van Ann is one of the inventors of the captcha, the squishy green word that a computer can generate, but paradoxically cannot understand. And here you see, before you leave that comment, you have to prove that you are a human by typing in the word that you see. Now Louise Van Ann got this idea of having people do work as they type in their captcha answers. So this is an old book getting scanned and through bad OCR, there's a number of words that aren't right and the spell check knows they are not. So why not give people two words in the captcha, one of which is the test word, they don't know which one is which though, the other one of which is a word from an old book. And then you type both words in order to get to where you're going. And as you're doing it, for all you know you're scanning mind comp. And that's, I don't know, scanning old books, that seems like a good cause, but I start to worry about the tussle that we see in just this zone as he again runs the numbers and is like, well, for one thing we can set up captcha sweatshops, it's trivial, people solve captchas all day long and just get paid to sit there and solve captchas, or just pay them nothing, instead give them free pornography. And to see each element of the pornography, each successive picture, you type the squishy word that you see, where do you get the word from some place where a robot is waiting to leave a spammy comment as soon as they enter the captcha. You take that image, you put it here, the person types the word, gets the free porn, the answer goes over here, the comment gets left and everybody's happy. It's an extraordinarily evil system made possible because we have these platforms available today. A final concern is what I call crowding out. And here is what I think otherwise seems like an unambiguously wonderful quality from YouTube now, there's this enable revenue sharing button. And that gives any individual putting a video up on YouTube now, the ability to get what the big content players who have forged deals with YouTube, what they get, which is ad words go up, people click on the ad words, Google gets some money and you get some cut of it. That seems better than Google getting money and you get none of it. How could you be against this? But then you start thinking about the ecosystem and you start to wonder, people who would normally be putting videos up on more community oriented sharing sites like blip.tv are suddenly not wanting to because you'd be a chump. Why wouldn't you put it only in the place where there's a chance of making some revenue from it? And I can't help but think that if Microsoft had had unusual provision before Wikipedia but realized that something like Wikipedia could happen and decided to tweak in CARTA so that people were solicited to offer edits and suggestions and if their edit got used they'd get a nickel or 50 cents, you'd have an explosion of offerings of change to in CARTA and then you try to start Wikipedia and people are like, why would I do for free what I could get paid to do over at in CARTA? And I see this potential crowding out without anybody even meaning to do it but there's this phenomena that happens when you inject money into the equation that I think is worth looking at. Okay, so what should we do about it? This is like the future of crowdsourcing and how to stop it. So what do we do to keep the good parts and not have the bad parts along with it? And I guess for those concerns that have to do with participants and labor standards, all right, let's just look at our current labor laws and see what we would want to copy, adapt and paste. And I don't know if we would even look at differences if you get paid a cent then like the AOL volunteers who used to be community sysops on the old proprietary AOL. They got free time in exchange for their services. There's now a class action suit that's been going on for 10 years where after they've been doing this, getting their free time, doing their work, they're like, wait a minute, there's a Fair Labor Standards Act, you need to pay us AOL and that suit remains unresolved. That would be a zero cent kind of thing, not just one cent and maybe even negative one cent. Julian DeBell's story of gold mining in the world of Ultima where Mexican sweatshops are set up where people just mine gold, virtual gold all day long that then gets sold on eBay and they're human, they're allowed to do it. Robots are not, which is why you have the people in the sweatshop doing it. And I don't know how we should feel about the fact that technically they're paying for the privilege of mining the gold but are they still owed labor standards? So that's a really good question. Do we care about addiction? I mean, maybe that's the problem. If you know somebody who's doing something for 60 hours a week, at some point you're just like, this person needs an intervention, a federal program. I don't know, something to help wean off of it. But it just feels weird to intervene with somebody who does Wikipedia 60 hours a week and is actually good at fixing typos. It's a little different when they're just clicking on yellow buttons and making green things light up that you think their brain might be in danger of melting. At the very least, I would worry about what we call reputation portability. With a firm like LiveOps right now, if you wanna leave LiveOps, great, you're gone, that's it. You don't have a portfolio you can take with you that says you'd leveled up to 42 and actually achieved Scientology Answering Service task. It's like, no, you'd start from scratch at the next thing and there might be, you imagine, some competition promoting policies that say if you have earned certain things, you get to be able to have that certified so that you could go to a competitor and then let's start to see what happens as people emerge from the sea of participants who are really good at what they do. And then there's also maybe just the sense of click-workers of the world unite, that you should get together and unionize or something. So this is from Turker Nation, the Mechanical Turk Message Board. This guy was not happy with some of his work. No more slave wages, blah, blah, blah. I love this reply. The best way to govern a work you do is to learn to say no, nobody makes you do penny or less hits. No union on earth can give you self-discipline. It's like, well, that kind of ends that discussion. But then you also see, right, just recently we saw the FTC do this thing, these guidelines saying it might well be considered an unfair, deceptive trade practice if you accept money to make an endorsement, like a tweet even, and you don't say it was paid. And I think many of us, our first reaction was like, oh my God, the government's gonna start regulating tweets, like how lame is that? But then you start to realize, given I think just how big this iceberg could be, and the target of regulation doesn't have to be a tweeter, it could be the platform, the Mechanical Turk that suddenly has to maybe vet the tasks or act so that that netmedtalk.com blog isn't allowed to say that you are not allowed to say that they commissioned you to review the site. I actually think this is probably a good and well-timed given that this whole thing is in its infancy of regulation and one we're supporting. Finally, on the question of what about just you go about your life and others can just benefit from and appropriate your work, like the Googlebot. Here's an example of a very left-wing blog that decides to link to Stormfront.org unapprovingly, Stormfront being a neo-Nazi site. And the first comments are immediately, why did you put in a link to that putrid site? I can't believe you linked to Stormfront, in part because you're helping Stormfront's karma by linking to it. And Google and others got together and came up with the idea of the nofollow tag that you could inject into a link that otherwise makes a link behave as it always would to browsers, but it tells the search engines, I don't mean to enhance the credibility of the site by the act of linking to it. And I think from the point of view of Google, they're just like, great, that's wonderful. We'd love your help in helping us know that you didn't mean to signal what typically is signaled by a link. I think actually a number of academics would really be into this. Somebody writes a really terrible paper and a bunch of other papers pile on. This is a really terrible paper. And then the citation count of the paper goes through the roof and the person gets tenure. So if you can have like a nofollow link in your paper, like I'm linking to it because I hate it, that could be salutary in the academic world as well. And you can even see in the physical world as more and more there are cameras and microphones everywhere. There ought to be nofollow tags that you could basically either physically wear or have a parallel RFID, sorry, so that people can recognize your desire about what kinds of appropriations you would like or not like and then choose to respect it. I'm not saying there'd be a law that they can't use the photo, but that at least it gives that moral valence back so that when you see something on the web or you act on the web, you have a chance of expressing some human element and asking another human who's gonna do something with it to respect it, just like Google respects nofollow or almost all major search engines respect robots.text exclusions where if you're a webmaster you say please don't index my site, that's the kind of thing that makes the web more human when people agree not to do it. So I now reinterpret this act that I thought was so cool when I first encountered it, Casey Kinzer's project. And I look at it and I'm like, yeah, it is cool. And it's drawing upon the goodwill of people. I just hope it's not like a new model for FedEx, like package, courier, delivery, where it's like, you know, again, your goodwill is being misappropriated in some unexpected way. This is kindness as an input that could get exploited. I also worry about kindness as an output. This is a really weird mechanical Turk task for 50 cents. You are to do something kind and then take a photo of it and send it in. And the commissioner of the task basically sees this as a kind machine where people are pushed into a funnel, processed through gears and hearts, fall out the other side. It takes in people and produces kindness. So kindness can now be, we have cloud kindness. And I'm just like, huh, the next time somebody does something nice to me, am I like, cool, there's more niceness in the world if somebody paid 50 cents somewhere. Thank you, unknown benefactor. Or is it like the last authentic expression of one person to another holding a door or helping them out in some way, you're always gonna be wondering if it's like just another transaction that's part of a net. And that makes me feel weird. Thank you very much. So shall we open it up to conversation? Does this have people reacting, including any principles from behind these various firms and companies? Anybody have thoughts? This gentleman here, there's a microphone coming over. Feel free to identify yourself. Why not just treat this as a form of work and regulate accordingly? Yes, one practical issue with that is it could make a race to the bottom or the top, depending on your point of view. LiveOps does not allow you to participate if you are a California resident. No doubt, because they're worried that California has particularly... So regulate nationally. Are there reactions to this thought? I mean, I can answer too, but I'm eager to get other people's views. What do you think about what David has to say about this? So I think it's a very narrow gap between regulating that and regulating my contributions to source code of particular open source projects. Yeah, how about that, Marguerite? Yeah, the seeker. We don't worry about Batman not knowing what he's doing when he jumps into these things. You do worry kind of about the man on the street. Not only towards the top of the pyramid as far as special skills. Absolutely, I like to think that. And yet he accepts no payment. But he knows what he's doing, right? Or most of the time. And so the problem here is we definitely agree that because of the way the system is gamed, people don't necessarily know what they're doing. And so the gentleman's comment, if people can show that they know what they're doing, then a lot less regulation would be necessary. It's when there's a lot of unknowns that you would stand. That's what government does. It steps in and takes care of unknowns. So there are at least two forms of exploitation. One is exploitation of ignorance, where you don't disclose and define print what's going on. Somebody's playing a game and they had no idea that it's being used in a certain way. And they would prefer that it not be. And it's like violating their autonomy to extract that work from them and not have them know the result. Which is why something like the no follow tag might not be a robust text, a bad idea. But there's also, I thought, you were talking about a kind of exploitation that was more innate exploitation, where maybe your view would differ between somebody who does mechanical turk as a graduate student instead of minesweeper because it's fun, and somebody who does mechanical turk to make ends meet. And that's why they do it. And they'll sign any form of opt out that they're offered by the commissioner because they need the pennies. But that's exactly the person you wanna protect, right? Maybe protect them by insisting that they get paid more than a penny or nothing at all, one or the other, kind of like minimum wage. Uh-huh. Other reactions, thoughts, questions? Eza. Of the other side, which is the I am creating value by living. Do you know Sense Networks? Sense Networks. Yes. They have, a guy named Greg started with a group of people out of MIT. They do something very interesting. They've worked with the telcos and have around half of the location data from everyone that lives in the US. And they mine it in, yeah, this thing. In real time. So they can tell, say, Traffic. Traffic whether you're going to work on time or not, whether people are going in earlier or later, which turned out to be a great predictor of when the stock market crash was going to happen. They can tell whether you're going to lose your job or whether you're gonna change cell phone networks. And how do they gather this data? Your phone checks in with cell towers. And do you opt into that or it's something sold by the phone network? Something sold by the phone network. Uh-huh. They don't sell, they just mine the data. So that's some form of data that you're creating. In Japan they're starting to do something interesting where to power the machines that read your tickets, they put micro piezoelectric underneath the floor so you step on it, you generate electricity. So that's a very real form of physics work that you're doing and they're creating value out of it. So there's a whole bunch of, I don't really have a question, they don't have a whole bunch of really sort of passive ways of gaining information through you living, where does that fall? And where do you fall on the excitement versus nervousness East Coast, West Coast divide on these new things? Oh, I'm an entrepreneur, so I like... Hell's yeah. Yeah, exactly. Buy me a share. Because I'm gonna be getting the money from what you do. But, right. But when you take off that hat and put on the peon hat, you're worried. Yeah, very much so. Okay, excellent. Yeah. A successful silicon valley entrepreneur needs to be able to switch hats at the drop of a hat. I'm also kind of interested in the extent that you can use this for actual academic research to solve problems that have never been solved before. Yes, I think a ton of mechanical turk tasks are sociologists so pleased that they are no longer limited to undergraduate psychology majors at their schools. And they ask the turkers to fill out these lengthy surveys. They're like, this is so cool. And of course the turkers, I think there's like turk honor. And the honor is like, you want me to fill out the survey, what you will get is a filled out survey. I do not owe you honesty. And so there's like a developing field on how to ask the same question eight different ways. So at least you can tell if they're being consistent. Anyway, I completely hijacked your question. No, it's all right. I mean, not just sociology, but artificial intelligence. Yes. Have you seen an open mind out of MIT? It's basically, it's a task to try to give computers common sense knowledge. So they're just leveraging the fact that all humans have access to all of this knowledge and computers don't. So dealing with the set of an expert system that is a very particular domain, trying to teach computers sort of the whole breadth of the common knowledge that we all share. So this is the kind of problem that we might only be able to solve through these techniques. So it's sort of opening a whole new world of research. Computers don't currently know the basic things about the world we consider common sense. You can help build a database of such knowledge in simple English sentences. So you are asked to just type a word in the search box, see the highest rated statements and then add your own statements about the world. Yeah. And then we can... And if there's enough statements about the world. Well, this then gets mined into more semantic networks and it's incredibly useful for improving speech recognition and predictive text entry and all sorts of applications. If I click on companionship, will it be safe for work? Probably not. A pet is used for companionship. A friend is used for companionship. You would socialize because you want companionship. Yeah, it's kind of an, I can see William Shatner reading this. Interesting. Anyway, my overall point is there's also research that can be done that was never possible before. Yes. For these techniques. And again, you're excited about this. Not worried about it. Entirely just excited. Got it. In part, because I think they solicit the sentences from people. People know what they're doing when they do it. Oh yeah, there's no sort of moral vagueness here. People are trying to help artificial intelligence. Yeah. Yeah, it's kind of a random set of statements over there on the right, isn't it? But you want to vote on them. You find, I want to do it. Cool. Other thoughts, reactions, questions? Yes. Brad Templeton. So you might have covered this, but I often point out what is by far the largest scale transfer, badly economized transfer of time for value that takes place in society and it's television advertising. So television stations get about a $10 CPM, which means about a penny every time they show someone an advertisement. So if you watch four hours of television, you'll watch about an hour of advertising with this unless you have a TV or other such device to get around it. And so they're getting about $1.20 for the hour of time you watched the advertising and they gave you four hours of television, which obviously cost them less than $1.20 to produce in value in exchange for that. So that's considerably less than a minimum wage and that is by far the grandest scale of this. Now it's a very different sort of economy equation, but I don't know if you've considered this as well. I think the efficiency of advertising, not in terms of how well it generates sales for the number of ads you place is what advertising executives call the efficiency of advertising. But the efficiency in terms of how much of the target's time it takes up in order to generate the value is changing quite a bit because of the web. Google also gets a penny just by having your eyes plant over something for a tenth of a second. All the ads you have to watch for stuff you would never buy is a waste to everyone concerned. Right, so are you worried or not worried? Well, I'm not worried about that because I think that the more efficient forms of advertising like Google and other companies do are probably likely to trounce the really inefficient ones. Again, efficient in terms of my time. And efficiency is good for everybody again. Like why not get the dog food ad if you have a dog and a cat food ad if you have a cat and if you have a dog, you never have to see another cat again. Other than the giant surveillance infrastructure required to do that, yeah. Yeah, I wondered when the EFF part of it was going to rear its head. I just wanted to point out somebody has been working on user labor markup language. I haven't fully grasped this, but I think in part it's talking about exactly what you're talking about as far as this phenomenon of when you're watching ads, you are doing a form of labor. When you read the rationale, it's a little Marxist, so I don't quite understand it. But I mean, I don't know, you can read it. If anybody understands it, please tell me because I think it bears on this. Five cents, five cents to explain Marxism to me. That would be ironic. Lauren Gellman, I don't know if we can pass the mic up. This might be simplifying what you just presented, but are you more concerned about the people who are doing this and they're being exploited, or are you more concerned about the activities that are being done with this time? I'm more concerned about the systemic piece, honestly. I think the participant stuff is interesting and it's more easy to grasp as a potential problem. At least it tests our intuitions the way that to me, the question of internet taxation, taxation of internet commerce tests our intuitions. For those of us who don't like those damn taxes, it's like isn't it convenient that a Supreme Court decision in 1992 makes it so that Amazon cannot be compelled by the state of New York to collect sales tax when people from New York buy stuff and so they just have a five to 10% subsidy against the same sweater vendor in New York who wants to sell it to you on the street. If you don't like taxes, then you're happy about that. If you think that taxes exist for a reason and like you believe in horizontal equity, then maybe they should have to collect and remit sales tax. So just similarly, I think we at least ought to go through the exercise of figuring out, inventorying our labor standards and the animating ideals behind it and see how much they carry over. I mean, David thinks that a lot of them carry over that you have these potentials for exploitation. On the other hand, a lot of the coercive environment of the workplace. What chunk of labor law is just about union organizing and elections and what notices you can put up on what boards at the company and when the company's not allowed to prevent the employees from getting together. It's like on the web, that's not nearly as much of an issue because you're doing it all from your own home and you can go to Turker Nation and talk to your fellow Turkers. Well, also, I mean, if you're just concerned about the labor piece and it doesn't really matter what they're doing, you have to sort of regulate it all. But if you're more focused on some of these outputs are using community effort to do things that weren't done before and where maybe we were happy they weren't done before and now we might need regulation because simple economies of scale are keeping every police officer and I ran from going to every single house to identify every person. Those are, I think, more complicated questions. I think that's a really strong point that the plus of all of this are the increased efficiencies you get for any number of tasks and the way that by chopping up the task you can actually have the people with the best comparative advantage or most desire to do it for a penny, to do it without having to move to Iran to identify the people. But some things we don't want made more efficient. We actually want the government to have to spend a lot of money if they're gonna identify a certain category of protester and then we say, well, do we blame the human cloud computing infrastructure that it's making a subset of tasks easier? I mean, I think our EFF friends would say the abuse of a technology is no argument against its legitimate use. Long live the Sony case. But I don't know what the EFF would say. Luckily we have a quorum of the board, I think. What would the EFF say about the Iranian example being used when you see it so readily empowering a government to possibly deprive rights? Cindy? Yeah, over to Cindy Cohn. Geez, there's more than just me and you got the chair of my board so I'm in trouble if I say this wrong. But I think that the... I'll be the way to say it. In public. See why I love working there. I think that the, yeah, I mean, I think to say that the problem is the technology as opposed to the action that is being asked, the technology being used to do is to kind of misidentify the problem. I'd also be very interested in knowing what the results are, especially of, I mean, the Iranian one is a little different and your hypothetical took it far beyond what the government actually did. But the fence one, I'd be very interested in figuring out whether it was actually worthwhile for law enforcement to do that. Chicago's got another thing that they just introduced as well and I'm very curious to see whether the promise of this actually meets the thing because then I've got an easy way to get at it which doesn't work, which is a really powerful argument in my world because I think there's a lot of excitement around this crowd sourcing as the answer to every problem you ever came up with. And I'm, I guess maybe holding my fire a little bit until I see whether crowd sourcing actually- Wait till it's successful before you try to take it down. Well, it's successful in some things and we're concerned about some of them and some of them we love dearly. But I'm not sure crowd sourcing law enforcement's gonna work such that I need to stop suing the government for actually wiretapping us because of the theoretical crowd sourcing surveillance problems yet. Well, really, our couple of months over, they're gonna do both, but- But I'm not sure it's gonna work and so I mean a lot of these law enforcement deployments of technologies have just been failures all on their own without me having to do anything. There is a weird symmetry to it because it's one thing for law enforcement to do it. It's another just to see it being done for the crooks. So this is whozerrat.com, a subscription site subtitled Real Life, Real People. It's the largest online database of informants and agents. So here are the rats of the week and this is how you can find out if one of these people seems to wanna buy drugs from you, you can consult whozerrat and figure out if you don't want to transact with them. Oops, we have to join for $50 to learn more about him. But that's an example of trying to crowd source the other half of things. You could see this site, right? And again, it's like if you're Republican, you're like in horror, if you're a Democrat, I don't know what you're thinking. But if you're about, because I don't know how tough on crime you are, that was the stereotype I was doing. But you could see in Iran, this could be extremely useful, right? Who's a Fedein and be able to identify the rats in the populace, like in East Germany, they're like whozerstasi.com would be extremely useful. But I think that a lot of that happens already. I guess I'm not sure what the digital angle is on this. I mean, I think if you lived in East Germany, there was actually a fairly solid network about who was a stasi that was as reliable, I mean, the reliability problems exist, whether you're offline or online. And I'm not sure that the digital stuff changes that much. I mean, your mechanical Turk idea is kind of interesting, but I'm not sure that that kind of stuff changes. I mean, it strikes me that whozerrat is just a slightly darker version of don't date him girl. But. There's an iPhone app now for that too. And I think all of those things people are learning as we move deeper into the digital world to take some of this stuff with a really big grain of salt. So I don't know if that answers your question. No, it's kind of the Adam Theer. Let's not be chicken little prematurely. So you're behind whozerrat, but not behind the... So maybe like two more thoughts and then I smell the food. So Jonathan, you're asking I think systemically from a kind of economic and legal perspective about the nature of these relationships between perhaps end users and producers. And I would wonder if you could point to, for example, cultural questions or historical questions that the hacker ethos has existed. The argument that joy, hard work, creativity and sort of that gives rise to potentially a whole series of innovations is significant. Perhaps as in the mechanical Turk example where these are in a sense perhaps folks who are hackers people who hacking is also defined potentially as sort of people do it because potentially they get back in a big way later on but also for prestige. Whether Turks are actually starting to get paid in some ways in a very small way for their kind of hacking. I'm developing this world university and school idea like Wikipedia and MIT open course where how could that begin to be a way to potentially inform folks in a different sense. Well, I think that I'm not sure I would call the typical Turk or a hacker. It's almost the opposite to me of a hacker because if you're a hacker, you answer to yourself. And I mean that both for good hacking and bad hacking like part of the definition of a hacker is that you freely choose to engage in some highly idiosyncratic tinkering activity where you feel like you know the shape of the map that you're acting upon. And mechanical Turk seems almost the opposite to me at least as far as how much you know about what's going on around you. Not to take away that mechanical Turk is probably an extremely generative platform itself that a hacker could use. You could do like a slow moe just like the IETF has done packet transport through avian carriers. They actually have a protocol on how to do it with pigeons carrier pigeons. It was an April 1st RFC and somebody actually implemented the standard. They actually got pigeons together and did it. And they did it with quality of service. They waited some of the pigeons down. Where was I going with all that? I had some extremely strong reason. Why is something having to do with hackers? Well, I guess for the point of view of the pigeon, they are not hackers. But from the point of view of the system, it's extremely intriguing. And you could see all sorts of ways to do just like the pigeons, a manual denial of service attack where you kindly ask Turkers to all visit a site and it reload. And now you have human zombies, not zombie zombies. That's good hacking at the layer of what to do with Turk. I don't think it makes the Turkers hackers. It makes them Turkers. So if I'm building a school, I would at least want to see people using platforms. I'd actually be thrilled to have my students in elementary school or middle school using something like Wikipedia all day or at least much of the day and engaging in like the liberal arts skills you need to try to tell Hot Pants 15 on the discussion page why he's wrong. And instead, George Washington was born in this day. I don't think it's the same if we sent our kids to mechanical Turk all day long to, I don't know, help them earn a little money for the school. So I guess that's my reaction. One last thought before we break for food over here. Yep. Aaron? Hi, John. We meet again. We meet again. No, it was great. I really enjoyed the kind of things you've added on to some of this discussion. And I had a few responses, but I wanted to ask you to elaborate a little bit more on one of the later points you made in the talk about this sort of implicit standard of authentic human something or other. I mean, I don't think you got a chance to really dig into it, but you sort of referred to that I think as kind of a baseline for what could count as meaningful activity on the internet in some way, shape, or form. And I wasn't sure how to respond. In part, so in interest of full disclosure, right? I'm not here just as part of an affiliate of the Berkman Center. I'm part of one of these organizations and social scientists that does a lot of the sort of Amazon Mechanical Turk crowd sourcing labor. Technically, I guess I'm also on strike today because you see Berkeley students and faculty are on strike. So I'm a resistant worker as well. Oh, hey, thanks. Good luck with that. Yeah, right, I look like I'm on strike, don't I? I'm sure the state of California has more money to pay. Yeah, we just have to find it. So I guess part of that is, I mean, if I can't even figure out, to a certain extent, what kind of my authentic position is vis-a-vis my relationship to the talk, what, where does that conversation start in terms of how to establish baselines related to authentic action online or speech or programming? I don't know the answer to that question, which is why it's so interesting to me. It's also why it's, I think, so valuable to be honing hypotheticals, possible futures that may or may not come about, try to catch the most interesting and like, hey, cool, wow pieces of what's currently going on, and then to try to test our intuitions about it and not just react knee-jerk-wise. I think there are ways that if we were a debating society, we could choose to construe some of the services we see to paint them in the worst possible light and trigger our most nervous instincts and ways to paint them in our most excited and like, wow, this is creating opportunities, like Kenyan refugees, you can't argue with that. And there's a real power to that. So I would not want us to fall victim to status quoism and just try to say, well, whatever has been is what ought to continue. We do that a lot, I think, with privacy. We just want to grasp whatever our existing level of privacy is, and I don't know why that should be so. And that's why I'm eager to just have the dialogue continue and to hear what people think. And I guess on that note, we should thank the extended Berkman family and friends for coming out for this and for coming together. Thank you. And...