 Hi, my name is Lana Swartz, and I would like to welcome, well first, I'm going to welcome you all, and then I'm going to start off with some brief housekeeping messages. First of all, please be aware that our luncheons are webcast live and recorded for posterity on the website, so keep that in mind as you participate in what we'll note out be a wonderful Q&A. Also, if you'd like to Twitter, you can use the hashtag Berkman, hashtag Berkman. Finally, Mary has requested that we leave conversations to the end, so take good notes and be prepared to stick your hand up at the end. Now it is my pleasure to introduce Mary Gray. Mary asked me to keep her introduction short, so she has plenty of time, so I will just read exactly what she asked me to say. Mary is a senior researcher at Microsoft, and a Berkman fellow this year. She maintains an appointment as an associate professor of the media school at Indiana University with adjunct appointments in American Studies, Anthropology, and Gender Studies. Her most recent book out in the country, Youth Media and Queer Visibility in Rural America, is awesome. That's my, I added that. And it looks at how lesbian, she didn't, how lesbian, gay, and bi-transgender, and transgender youth, young people, negotiate their identities in rural parts of the United States and the role that digital media play in their political work, local belonging, and connections to the broader imagined communities. Mary's current book project, co-authored with computer scientist Siddharth Suri, which you will hear about today, examines digital workforces and the future of employment through case studies of present-day crowd work on four different crowdsourcing platforms, comparing workers' experiences in the United States and India. Thank you, Mary. Thank you. Hi, everybody. Okay, I am so excited to present this material, and I have so much of it that I wanted to share, so I'm going to, and thank you for indulging me in holding your questions, but I'm going to share as much as I can. I'm working through an idea of ambient workforce, and I don't know if it works, so I would really love some thoughts on what sticks for you and what doesn't. And I want to acknowledge one of my co-authors is here, Meng Yin. So she may be the person I gesture to for one of the slides, because sometimes I flub exactly what's going on in that slide. So this is work I'm presenting. It's joint work with Sid Surrey, who's in the New York City lab of Microsoft Research. And today I just want to introduce you to thinking about crowdsourcing as work. I mean, we tend to think about it, or often we'll have conversations, lots of related conversations about peer-to-peer production and collaboration in settings like Wikipedia. And I really want to ground it in the experience of doing paid work and what that can mean. And secondly, I want to argue that crowd work is perhaps a recognizable iteration of the kind of contingent labor that's been at the core of tech innovation for two centuries now. So I want to give it a little bit of history. And then lastly, I want to set up, hopefully get to a set up of a conversation about the cooperative potential of an ambient workforce, and I'll unpack that term later in the talk. But what I'm aiming for here is to say that it's not just a good thing, a nice idea to be able to organize crowd labor or crowd work in a cooperative way, as opposed to the somewhat atomized ways that we think about crowd work now, that it actually could be a very important business model to platform economies. So let me see if I can set up a provocation for a discussion that might connect to this. I want to say that there are plenty of historical cases we might turn to. I'm going to turn to a couple. But if you just think about how often we talk about, we're almost there with automation that we're always moving the goalpost. So just a common sense understanding of how often are very excited, optimistic goals around automation fall short. And every time they fall short, we reset what we'll do to get to the next milepost. And in doing that, we create rather than eliminate labor markets. The hitch is that often the jobs that are created are framed as temporary and often they're not even visible to us. So in setting up that at any point, these temporary jobs that are just helping us automate will disappear. We miss the opportunity to be able to see those jobs and the expertise and the skills that go in them because we're not really considering them jobs at all. And then the other paradox that I want to throw to you is thinking about the necessity, not just the luxury, but the necessity of relying on a workforce that's always on ever expanding and responding to a mix of bursty and idle workflows. We might just consider this a 24-7 work shift. But what it would take to be able to sustain an on-demand economy built on crowd work is precisely recognizing not just the presence of an ambient workforce, but the value of it to the very business model. So I'm trying to poke here a bit at the conversation around the rise of the robots to say that perhaps we should be thinking about innovation as the opportunity for the rise of human computation, not just the robots. So let me start with what I find to be a pretty reliable definition of crowd sourcing as a job by legal scholar Alec Felsner. He put this definition forward in 2011. And this makes a lot of sense, distributing calls to an online pool of workers that we often call the crowd. I think it's important here that we often imagine this crowd as autonomous, anonymous, and widely distributed answering this open call. I'm going to refer to this work as crowd work throughout the rest of the talk. So if we think about these features of a crowd, anonymous individuals, autonomous individuals making themselves available to work that comes down the line, that's not necessarily seen as core to the central production of a good or a product or service. We might be able to see some of the precursors of crowd work in the moments of piecework, the dominated textiles production in the 18th and 19th century, and later the shift to outsourcing where the return on investment for particular companies meant that there were activities the company was doing that seemed peripheral and unnecessary to the core needs of a business that shifted away from a model of vertical integration that was critical to building the assembly line that came out of piecework. So as the shift to saying we don't need to own everything and run everything, we'll take some key pieces of our business and move them outside of the firm's boundaries. We really see the rise of a long and actually very invisible global supply chain. And from there it's just a hop away from the mid-90s and the building up of a reliance, particularly in the tech sector, on what are called perm attempts. It's actually a reference and hopefully you all can see my fantastic image generator of Clippy letting you know the case that established the existence of the perm attempt in tech companies was a case against Microsoft that was settled in 2005. It effectively established the ability of vendor management systems to be able to outsource tech jobs to agency employed contract workers so that companies no longer had to directly employ full-time workers for critical operations like coding or other kinds of engineering. It could be turned over to vendor management systems. Arguably these different precursors set up the shift to a wider expanse, the possibility of a wider expanse of crowd work to a range of sectors beyond technology. Healthcare, we already see it in education and there's a really nice list I can show at the end of the talk of the industries that are really ripe for crowdsourcing. But if we take away the magic of crowdsourcing and just call it a shift to contingent labor, I think we get a better sense of what industries are poised for that kind of shift, away from full-time employment to reliance on a contingent workforce for day-to-day operations. So if that's the case and we think about crowd work as this next iteration of contingent labor in innovation around tech sector, then we might look at what exactly the API leaves behind when it manages workflows for a platform. We might say it pretty much strips away the things that are familiar to us when we imagine the kinds of employment that we associate with, dare I say it, good middle-class jobs as a way of setting up the refrain. And perhaps several of us in the room can relate to these jobs, these forms of employment. But in the wake of stripping out these different facets, the assumption always was that these are just not necessary to getting work done. And what I want to share with you with the rest of the talk is the pretty profound amount of evidence we have that workers put a lot of these features back into their day-to-day task crowd work. So the products and services that are probably familiar to many of you in the room that are the output of crowd work, things like platform-driven services, translation, a lot of content management. So every time something's flagged or we're trying to figure out is that adult or not, this is work that primarily goes out to crowd workers, to this contingent workforce that's paid either through temp agencies or is working on a platform. And my question for us is this really the future of work? For me, if you go back to that paradox of the last mile of innovation, it should tell us that this is not necessarily the future, that this future is very far off, that we'll continue to try and automate different different functions and different operations. As we do, we're going to continue to produce labor opportunities that are effectively these contingent jobs that are very hard to see and in some ways are expected by the API to remain invisible, that that's part of the magic of what automation sells is this sense that there's no human in the code. So if you accept my premise that this paradox of innovation means a different future for employment and for crowd work, then perhaps another approach we might take to understand this future would be to talk with workers who are doing this work today. And that's where our project really started. Sid and I met almost three years ago now, sat down, and when I asked him, so who are the people who do this work? He looked at me, he said, I've always wondered that. And when we started asking other other engineers and researchers, so who are the folks who do the kinds of human computation for natural language processing or other kinds of tasks that they were putting out on platforms, the range of responses were, I don't really know to, I really don't want to know. And nothing gets an anthropologist more curious than that kind of reaction. So our basic questions were who participates in crowd work? What do their lives look like? And importantly, can we use their experiences to teach us about other possible futures of crowd work? Reasonably, rather than thinking beyond that moment when we've reached automation, perhaps it might make sense to look at the practices that are going on right now. So we have four different cases that just kept growing almost like Kudzu. And so the four platforms we ended up focusing on are probably familiar to many in the room. The first is Microsoft's own internal platform. I want to underscore pretty much every tech company has a need for an internal crowdsourcing platform for operations that they don't necessarily want to put out beyond their firm. So Microsoft has this platform, but I can assure you other tech companies that you might be familiar with and have on your phone right now also have these kinds of platforms. This represents a lot of job opportunities, we might say. The UHRS stands for Universal Human Relevant System, another fantastic name branding opportunity for Microsoft. The other three platforms are Amazon Mechanical Turk, probably the first place we started because it's the most publicly known. Lead Genius, which was called Mobile Works at the time when we started this research as a social entrepreneurial enterprise, B Corp, that generates leads for sales forces. So it's a very particular kind of task work. And Amara.org, which is this really fascinating nonprofit that started out as a volunteer base of people who wanted to do translation work mostly through TED. And as they did translation work and captioning work of videos, Amara became known for this really high quality work. And different companies started coming to Amara saying, can we just pay you to do some translation and transcription and captioning now because we really need to get this done. And they found themselves in the middle of balancing the volunteerism that really built their community and these job opportunities that were presented to the folks who work on this platform. So to study these different platforms, we both looked at the workflows across three of the four platforms and some of the work that we could scrape from one of the platforms. I'm going to share some of that data. We're actually still processing a lot of it and trying to figure out how to balance the need for these companies confidentiality, but also share some data in the aggregate. But we also did a lot of field work and interviewing. So I'm giving you this mostly because I'm trying to come to grips with just how much data we have the process. And I would argue for grad students in the room that if we're going to look at large-scale systems and try to understand the social implications and the social experiences of them, I think we need new methodologies that let us integrate this range of data. It's been a challenge and I don't know that we've necessarily landed it, but we've got a model for what it might look like to look in an ethnographically sensible and sensitive way at large-scale data. One of the first things we did was map the crowd. And this came from a hit that we, a task that we placed on Amazon Mechanical Turk. And we asked workers to self-identify to put a pin in the map where they're located. And interestingly, the reporting from their self-reported locations was more accurate than anything we saw in the IP addresses, particularly outside of the United States. So it became a really nice reminder that asking people for information can sometimes be more accurate than mining it from metadata. But when we map this, we noticed, if you'll notice some of the gray areas, some of the patchiness, it maps onto connectivity. So the ability for crowd workers, and this is such an obvious statement, right, the ability for people to access this work deeply depends on their access to the internet. And so if we want to think about this as the future of work, and I think about this in terms of the library box project and other efforts to offer resources in places that have low-resourced communities, we have to think first and foremost about what it means to not have connectivity in a world that might be dependent on crowd work. The next thing that came out of this initial mapping project, we asked workers how they found out about the task. And it immediately showed us some really interesting patterns in how people share information. So what you're looking at, the spikes represent moments at which this information went on to popular crowdsourcing discussion boards. And we can literally see more information being circulated through organically organized information discussion boards outside of the platforms than anything that was happening through search. Giving us a really strong signal that information sharing and collaboration was happening outside of the platform. But there was really no way to see the amount of collaboration that was happening without looking at what kind of conversations and exchanges of information were happening on discussion boards. The next place we looked, one of the survey questions we asked people was how they found out about the platform that they came to work on. And you'll notice or hopefully you'll be able to see from your seats that much like the sociology of work literature would suggest, there's a significant, almost a majority of people finding out about the platforms that they came to work on through a friend, a coworker, or a family member. So rather than searching the internet, although web searches were an incredibly important source of information, it in no way displaced the kinds of familiar networks where people have always shared information about how to get a job, who to trust. This became really poignantly important when I was doing fieldwork in India. And because of the history of business process outsourcing companies scamming people who participated in those companies in the early 90s and late 90s, workers were really hesitant to just try a different platform, to try any platform. So information sharing and hearing it from a friend or someone you trusted became the first point of entry for being able to access this work. This slide showed us, and this is the work that Ming led. This slide shows us that there is actually a lot of collaboration happening within the network. You're looking at over 10,000 Amazon Mechanical Workers reports of their connections to other people. We asked them how many people would you identify as close friends or colleagues, co-workers. And over half of them have connections and a good percentage have connections with more than one person. This, and actually Sid and I had many laughable moments where I said, I know this from the ethnographic material. When I was going out and doing interviews and watching people do their work, it was really clear that they relied on individuals, often sitting in their homes with them to figure out how to do a task, to figure out how to evaluate whether a requester was a reliable source of work or not. So not surprisingly, workers are not autonomous, anonymous, individualized, atomized participants in crowd work. They're also often collaborating in sub-communities and networks that they create themselves. There's a real interesting distinction between the kinds of collaborative networks that form in India and the ones that form in the United States. So there are patterns to what kinds of collaboration happen. And for anybody who's ever run an experiment in this room, I hope this is a little chilling because it means you probably don't have the kind of unbiased distribution that you thought you did when you ran that experiment. So importantly, for the ethnographic material we have, it's really clear that workers help each other out. This is a quote that I have from one of the women that we interviewed in Tamil Nadu in the South in India. And that she's directly taught several of her friends in her community how to use this website, how to navigate the signing up that's required, which as somebody who's tried to sign up and failed the first two times, it's a little dizzying if you don't have somebody who knows how to walk you through the system. Many of the workers we talked with talked about particular workers who posted quite often. In India, it was a gentleman named Salman Khan. And another in the United States, some of you might recognize the name Spam Girl, folks who run forums become key information gatekeepers and sharers in these communities. But there's also some really particular information sharing that's happening that's also about not just how to do the task, but how to be successful at maintaining this kind of work, how to stay awake, how to make friends and identify folks who are going to give you the best requesters names, the best employer's names. So there's a real range of sharing that for many of economists, this looks completely irrational, but I can assure you there's good evidence that it's widespread, mostly because the people who are doing this work imagine that the value in sharing this information now is that it will come back in dividends when they need help down the road. So the relationships that people are forming hopefully remind us of our day-to-day experiences with our coworkers. So importantly, the lesson I want to make sure that you take away and you share with all your friends is that we often imagine that crowd work works like the equation above, but it actually works much more like this, where workers are in their own network sharing information, learning in many ways, forming cohorts that move across time, sometimes three or four or five years on the same platform, acquiring knowledge and sharing that knowledge out. And that this has implications for how we might actually build better crowdsourcing systems, but also how we might be able to use this as a basis for more cooperative approaches to crowd work. So just to recap some of the preliminary findings that we have and hopefully that come through in some of these slides, workers are collaborating extensively and unfortunately we have to date not really lifted the hood of the API to be able to see how much collaboration is happening and the value that's generated by it. I would argue it's actually necessary, certainly as necessary as the API and it's tough because we don't have an experiment that would break apart the collaboration that we're not recognizing in the first place, but if we could just value the amount of collaboration that's already in this system and see how to better support it, we might see some opportunities. I would argue that there's not just a kind of a loose need for this invisible ambient workforce, but rather crowd work depends upon it. There's a real Pareto distribution, so there's a core group of people and I have some really great data and could show you the slide of people who are working about 20 to 25 hours a week on this platform. Some are working more, but as a median there's a pretty core percentage, 20% of people who make this a full-time gig. They're able to do it because they're coordinating with each other. They drive a good 80% of the productivity across all four platforms. It's pretty consistent, so we can imagine that this is a feature, not a bug, of crowdsourcing. We certainly see that power law in Wikipedia. We've always been puzzling over, how do we get rid of that long tail? I want to argue there's actually a lot of value in keeping a system that has both full-time and this part-time flux in this system. This comes out of observations that I met folks who literally depended on the ability of forming a full-time opportunity out of crowdsourcing. I'm using opportunity because that's the language they use, so I want to suspend a certain critique here that they somehow don't know what's valuable to them and say that for them they're forming these opportunities and at the same time they're also really valuing the ability to step away at any moment. I don't know about you, but I work pretty hard not to feel like I have anybody telling me which hours in the day I work, so I can work 17 of them if I want. I'd say for many of the people we've interviewed and certainly for this core that are making up the bulk of the productivity of crowdworking sites, they're depending on the capacity to come and go, even though for many of them they are often working very consistent hours. As an example, one of the folks that we followed for two years, his name is Zafar, at one point his mother was in an auto accident and he stepped away from his position at Lee Genius for three months. He did not have to do much to be able to step out of the system and when he came back there was not much that he had to do to integrate back into Lee Genius because they've actually shaped it in ways that allow for that come and go, but for him being able to do that meant the difference between being able to take care of his parent, take care of his mother and keep his job imagining it was something he would be able to have down the road. That was something he did not have an opportunity to do in any other part of the job sector that he had access to. Arguably these full-time and part-time workers can indeed be the same worker at a different life stage and how do we build an employment system that really allows for the presence of both full-time and part-time workers who are valued equally? What would that look like so that they don't become different tiers of workers? Lastly, for many of these workers, they cared about more than the price point and I'm struck by how many economics papers I've read that are really focused on what's the right price point for a task. For the folks who are doing this work consistently and really producing the most value out of crowdsourcing, that's one of many parts of the equation for them in evaluating whether they're going to pick up a gig or not. When we were, and I don't have a good experiment to testing this, but the number of people we interviewed who said I'd be willing to work for someone who would acknowledge the work I did and said thank you. I would absolutely be willing to work for that person even at a little less money knowing that the work I did was appreciated and that I understood where it was going next. That's incredibly valuable to be able to imagine what it is that someone's getting out of that moment of acknowledgement and we can imagine a system that's asking them to remain silent and somewhat invisible but that carries a great amount of value. That's something they right now get from each other but the genius in Amar are two examples of cases that work that acknowledgement into the work that's done and in some ways can perhaps be a reason that they have such great, they have such capacity for keeping their workers, they have less attrition. So as a takeaway and maybe my main claims for the presentation today, if we think about collaboration among workers as critical as the API then perhaps we can see it as what's fueling on demand economies. We really don't have good evidence that that's not the case until we actually evaluate and place a value on the amount of collaboration, even recognize the amount of collaboration happening in these systems, we don't know how vital this collaboration that happens outside of these systems is to the productivity of crowd work or to on demand economies. So it's both invisible to the platform and again in many cases it's what people are expecting from the experience of hitting an app and making a request and having something like a burrito magically appear at their door. So what are we going to offer in exchange to a worker who's willing to take on the request to remain invisible, to remain unacknowled through their work and yet see their work as vital? And how does the presence, the current amount of worker collaboration set the stage for a cooperative approach to platform economies? If workers are already organizing and finding ways of sharing information and in many ways providing a lot of the support that we often equate with an employment situation, how might we be able to ignite that into opportunities for cooperatively organizing platform economies? Let me return to my provocation that I began with. So if you buy this paradox of tech innovations last mile to automate that it always produces jobs, that it yes eliminates jobs, sure the horse is out of business we all know that, but that in the wake of every effort to automate we end up with some jobs that we just imagine are going to go away and they often take decades to disappear from that innovative process. Then what would it mean to think about these moments as job creators and how to think about them as something other than temporary or other than disposable? Let them be ambient. Let them be part of the work environment. Could we see the requirement that's nested in on-demand economies as this ambient workforce and see it as a potential benefit? If you can hear the tenetiveness in my voice is because in many ways when I think about this work the last thing I really want to come of it is that it just does away with all protections for employment. At the same time I don't think we've reckoned with how little protection there is for the vast number of people in contingent and temporary jobs now, so could we pay attention to how contingent labor is such a productive force in a way of organizing employment and refit how we think about employment to focus on those workers as opposed to a core of W2 or full-time employees who are assumed to be producing the greatest value or to have the most unique expertise because in this system it's inevitable that that expertise becomes less and less value and someone's presence and their effort and their commitment to a particular task becomes the most important piece of the equation. So hopefully this is about the innovation that could bring about human computation as a front and center model for employment rather than putting robots front and center. With that, thank you. Hi, my name is Kate. I'm a fellow at the Parkment Center. That's a great talk. Can you talk a little bit more about the ethnographic methodology that you used for this and in particular back to one of your original questions about who are the people who are doing this work? If you could share a couple of stories both from the Indian and U.S. context of like who are they and kind of what drives them to do it? Yeah, thanks for that question because I'm still so hard not to present the ethnographic material and I'm still working through a lot of the U.S. material and I would say the consistency. So let me introduce you to a few folks that we met. When I started the field work in India, probably the folks I ended up hanging out with the most were a pair of sisters or sisters in law part of a joint family. They're Muslim. They're expected to stay within the home and not work in formal employment and both of them had joined Amazon Mechanical Turk two years before I met them. They did not know anyone else on the platform on Mechanical Turk. They only worked with each other although they did have a few as over the year that I got to know them. They had family members who they introduced me to, an uncle, a cousin who also had accounts. It was much more common for me to meet people whose families had introduced them to different crowdsourcing platforms and therefore they literally were working as a family unit and Asra and Sabina would, Sabina had less confidence in her language skills and the task she was doing were classification tasks in her spoken language skills but she had perfectly strong written English language skills both of them spoke Urdu as their first language and interestingly most of Asra's support came in the form of Sabina you can do this, you know how to do this. So I would watch the two of them work through a task and most of what was being imparted was encouragement. It wasn't specifically how to do it. It was at different points Sabina watching Asra doing that task. In the United States as a comparison certainly spending a lot of time with Christie Milant who goes by Spam Girl who created a forum or took over a forum called Turker Nation that's one of the largest networks for workers on Amazon Mechanical Turk. She has been doing coding and has a really high computer literacy for most of her life so when she came to Mechanical Turk it had everything to do with needing a part-time job whilst she was taking care of her kids and had just shifted from a home daycare that wasn't running that well that was just getting a little too busy for her to wanting to do this part-time work. We met both of them and the strategy we took for the ethnographic work really was a kind of a flip from what I did from my previous work. We put tasks we put surveys out on the four platforms and it was a really long survey an 80 question survey so to have 2700 plus people actually respond to that long of a survey was pretty amazing but at the end of every survey we asked would you be willing to do an interview in person about the work that you do and whoever said yes we contacted them in the way that they requested and met them and for the people who were willing to stay in touch and allow us to be present while they were working and to meet their families we found a group of people who were willing to do that so it was a mixture of snowball classic ethnographic field work but also being able to really follow the object of crowd work onto the platform to the people who work on those platforms and then to be able from there to meet the people who were in their lives who no longer did it and to be able to talk with workers who had given up and to find out why they had given up or why they had maybe had their accounts suspended so the demographics are interesting in that they're similar they're they're in most cases the majority of people across all four platforms have another full-time job this is a secondary income but when asked why are you doing this work the majority of them said that they're doing it for a second income for money but the next highest reason was because they wanted to be able to work on skills that might get them jobs elsewhere so there was pretty pretty strong amount of consistency over some of the demographics of these two groups you mentioned that and you were just talking about that you know these people talk to each other on forms they're independent of platform I assume that's true we're not just mechanical Turk but for the other three as well yeah now is that happening because the platforms don't set up any such forms for themselves or because people prefer independently run forms to forms that are part of the platforms yeah no it's a really interesting because when we when we realized that there was the same what we call in a paper organic collaboration that there was this amount of collaboration that was happening off-platform and that we found it in both lead genius and amara that have structured engineered forms of collaboration they have chat room set up they have ways for workers to be able to collaborate it's pretty clear that there is really a mixture of both where it's possible in the case of mechanical Turk and UHRS there is no structured forum on platform for workers to to communicate and I think that's why we see such robust discussion groups that have organized informed off-platform I think in many ways the the recommendations that we have at the end of the paper about collaboration that I'm happy to share with you all at the end it's really to think both how to facilitate organic collaboration rather than assume you can engineer the collaboration that's necessary so it's really creating ways for workers to use that conduit of the platform both for casual conversation it was often clear that the that the forms became break rooms or water coolers where people could vent and also talk about how to find other platforms that they could do work on so it's it's it's across all four platforms and I think for the platforms that structure it even there the kinds of conversations they're still really generative I did want to say one more thing on that it's really interesting to see how in India because of the lack of a real cultural history of using a website web-based discussion forums there's almost a complete dependence on person-to-person or phone conversations texting or Facebook and it's because Facebook's ubiquity in India because it's bundled with smartphones that it has become the dominant way for workers to either form closed or open Facebook groups and so the consequences of that in terms of what kind of information can be effectively shared are also I think part of the equation hi thank you for your presentation thanks so I'm wondering if you could comment on the transition of existing industries into global competitive markets through crowdsourcing for example my mother is a translator and she used to get 25 cents a word and now she gets three cents a word because she's competing with people in India who will charge much lower rates and do it faster overnight etc so I'm wondering if you could speak a little bit to that issue yeah I think one of the most important things we have to grapple with particularly if we could imagine a more cooperative approach to this Amara is a really good model for this their translation pays at about the captioning pays at about 18 dollars an hour which is fairly high for trans I mean that's maintaining a pretty high level the quality is incredibly high so they're hitting a market that's interested in the quality of translation of any of you have ever paid for bad translation you know the value of good translation so part of it I think is grappling with rather than ignoring both the labor arbitrage that happens right now but also really embracing that this is a global workforce what would it look like to be thinking about our labor politics and the political economies of labor markets as a global question as opposed to a nationalized question where we try to figure out how do we shore up you know shore up our borders so that US workers get X amount that's in that's impossible so what we could do is shift to a model and this is what I'm hoping for maybe hopefully through the berkman is to think about models where we could imagine perhaps through an LLC corporation umbrella that would allow a nonprofit status for workers who could pay pay into basically create dividends that can be shared internationally that would allow workers to form cooperatives international cooperatives that pay the same rates as opposed to creating really disparate labor forces and and split labor markets they're really just taking advantage of circumstance thanks so I think this is really an interesting narrative of optimism around collaboration and can you maybe speak to the other side what are workers fears and anxieties platforms like Turk optic on and Turk nation and so many others exist right for a reason are there parallels as well for some of the other crowd work sites that you're talking about and yeah just generally maybe to complicate this narrative yeah that's a no thank you for asking that because I think and I am I'm definitely accusiving an optimist more days than not and I think and I think one of the most interesting things when I hold up the example of Amara and how much and I think in many ways because it came out of a nonprofit and is still a nonprofit that its motivations were a volunteeristic that it was able to shape a community around those incentives if you will so that the the interest was in maintaining a semblance of of a real democratic participation that was there from the get-go so for Amara what's intriguing to me is as they shift to a model that is a labor market does pay it has paying jobs is that the places where where they really where they strained to be able to make job opportunities equally available those questions come up I think they do spend so much energy trying to figure out how to make sure that everybody is getting a fair chance of being able to participate in the labor market as they choose to and to be able to register that choice is the toughest thing because the further a platform gets from listening and actively requesting feedback from workers the farther it's going to be from being able to create that that equity but I think one of the biggest challenges here is right now the legal frameworks for being able to provide equity for workers is there's a real barrier because as soon as a platform provides even a modicum of of mentoring or training it can trigger it can it can certainly trigger and cross the line into curation of a workforce which means you're effectively responsible for employment and that's not a bad thing but what would it look like to be able to create more room for platforms to be both a setting where people are finding opportunities but also confined resources without tripping that wire so there's something something else other than full employment and the way we think about it now but the downside so the genius is a good example workers are constantly wondering who's getting paid more so they have a model in which people are paid pretty much the going rate what the rate would be for that country and it creates a great amount of tension and a bit of animosity so there's a renationalizing and interestingly a pan-Indian mist to their labor force which makes no sense if you know South India but there's there's this really interesting consolidation that's happening precisely because there's a sense that it's us and it's them and and I think those tensions work against the opportunity for a globalized workforce it renationalizes it in ways that I think are are harmful I don't know if that gets at your questions on the talent thanks Ben thanks for a wonderful provocative presentation I was looking at the map and thinking how remarkable it would be if people in South India and in Los Angeles are really participating in the same labor market what's the evidence are these actually the same markets are they separable and part b of the question is when you're looking at collaboration cooperation amongst workers is it geographically rooted or dispersed across different areas it's they're the same labor market they are literally competing for the same tasks in some cases there's some and I think this is where those national borders can be aligned depending on the platform but it's pretty clear that workers are willing to cross those borders to be able to certainly Turk Opticon is a great example for those who don't know it's a plugin that allows workers to be able to provide feedback on specifically Amazon Mechanical where Mechanical Turk employers requesters who post jobs so all workers are contributing information about the reputations of those requesters and sharing information that we could argue they would want to keep for themselves that they don't necessarily individually benefit by saying this was a great requester and they paid me well you would think in economic terms well to get your self-interest to share that great information we've got clear evidence that no they're they're really creating a consolidated labor market I think at any moment it's also analytically easy to split this into a thousand different tasks but again you peel off another layer of of any crowdsourcing platform and it's clearly giants like Google Microsoft Facebook Instagram and Twitter any big company that has a lot of content that needs managing of any kind they're all subcontracting to subcontractors who put their jobs on these platforms so we could we could be picky and say well that's not really Google that's that's a supplier to Google but if I look at who's benefiting from the productivity of those workers at the end of the day there's some pretty big companies that benefit from the productivity of this long supply chain so some of it is perhaps how we've thought about supply chains is really independent and in of themselves separable from these large companies I think in many ways we just have vertical integration done differently hi I was wondering I know you mentioned a lot of people use this as a supplemental income or they already have a job but I don't know if some of the people you are able to interview were there some people who were like unhappily unemployed or have been displaced from technology and what was their feedback would they prefer to be in a real water cooler rather than working remotely we met the gamut so and maybe this goes back to some of Kate's questions too that I mean we met there's a worker I interviewed who's in Texas she for quite a long period of time was a did editorial content a PR firm that was a contract agency for a large tech company in that state and her mom needed care and so she left her full-time W2 employed position with that agency and actually was with the university too so extra irony and she when I taught when I asked her it's like you know your her mom is now doing better she could go back and find full-time employment and at this point for her the break between the amount of time and cost of commute and other other pieces of being at a full-time job were less interesting to her than the benefits of being able to stay at home making a much lower salary but in terms of figuring out the cost benefits for her it feels like a better just raw dollars cost or benefit to stay home and we met folks who were incredibly frustrated with being pushed out and not surprisingly there was a real spike in the number of people who joined Amazon Mechanical Turk in 2008 in the recession in the recession so folks who were absolutely doing this and went on because they were madly trying to find some income that was going to pay the bills in many cases this is the difference of two bills in their household getting paid or not getting paid so they're this is not sunshine and roses this is definitely people making the best of a circumstance in a broader labor market where contingent labor is quickly becoming the dominant form in which most people find employment and even if you have full-time employment for 18 months with a with an agency after those 18 months if you step out of that position and go and find another temp agency to work with you might be able to fill it with this time so we really met we met the gamut of folks who were doing this and yes now it's a choice and we met them in India and we met them in the United States and folks who choice is not the right word it's out of necessity and this is an opportunity for them to be able to stay above water thank you hey Mary there's a great presentation and so you've been sort of I think teasing towards this I think it's a good follow-up to the last question and so I wonder how much you think that the issue might be not so much that crowd work is really temporary and has lack of protections but rather that the full-time jobs that exist today sort of lack some of the characteristics of crowd work and the flexibility that would allow people to hold those and on what like how could real jobs look enough like crowd work to attract the kind of people who are choosing crowd work yeah that's I mean it's really interesting because I think it's both and in some ways our full-time work looks increasingly like this where we're bringing our own devices we have the possibility of working from home but that is such a small percentage of the labor market right now in the United States and that's informal economies you get outside the United States I really stopped using the word precarious when I got outside the United States because for India it's an 85 cash economy it's mostly informal so I think in some ways it's can we see the ways in which the ways in which the work you were just describing that is full-time work and many of us who have that kind of work might want more opportunities to step away from it when we want those kinds of features and at the same time recognizing that the vast majority of jobs are really in a position where they're not getting to make those kinds of choices in the first place like what are the kinds of what's the social contract that we want to set for full employment what could it mean beyond time like what would it mean to no longer have a job just anchored in the amount of time you spend at a specific location that's really the question that's up for grabs here and I think it's not just a hypothetical it is how most jobs are organized now to be to be able to respond to the demands of creating a different product to go in a rapidly different direction but a lot of this these are political choices we've made around what kinds of features come with a full-time job and so it's where I think we're at this moment where we could really ask the question what do we want full employment to look like and how do we create a system that does not reproduce the tiered system we've always had where we have some folks who have that really awesome awesome full-time job that maybe isn't so awesome after all and this other tier that maybe it'll get to that maybe they'll get that kind of job like I think in some ways it's it's can we bring both those questions together right that will be one too thanks so much Mary I'm wondering if you could share a few preliminary observations on the gendered dimensions of crowd works and I I feel like I might be going down a feminist utopic kind of vision right now but just getting the drift from the anecdotes that you've shared I'm having a sense of women being able to build on existing social networks in these collaborative environments that actually avoid the gendered inequities that exist in the corporations that create the platforms but I'd love to know what you think so I see that too in my data I know I want to see it so I need some help not seeing it so clearly and to keep the critique because I think let me bring the critique to the table when I look at the case of India it's so clear clearly the gendered expectations of who works in the home who does not the expectations around childcare and family care that are not just in India but are here so when I see the demands for flexibility for for women in many cases that the demands come from all of the other demands on their time so they need flexible work because they have inflexibility in the lack of support to take care of their children and elders in their lives so gender inequity produces this need for I need to be able to get a job whenever I can right to fit it in when I'm not taking my kid to school and not taking care of my elderly parent so I feel like there's the the realities of and women are getting this opportunity particularly in India I think the narratives are very clearly there of an opportunity to participate in the labor market in the workforce and to feel a sense of autonomy and to feel a sense and I want to keep to the word of that sense that relationship of I have the power to make decisions with a particular amount of money and to make decisions about how I use my brain like also particularly she was going nuts not being able to use her education this became an opportunity for her to feel a sense of accomplishment we might minimize that and say that's sad but I actually think it's pretty profound to be able to find a way in circumstances where that's not socially condoned to be able to find the space for that and in the United States to see cases where women are able to find the hours they need to be able to take care of themselves because they're not spending it in a commute to an office job to be able to make sense of and really deal with the conflicting empowerment of that so I do see possibilities I think the other one a really lovely anecdote of and this was for an interview that I did in the US and in India of young women who said it was so nice not to have to participate in the office politics of having to deal with gender discrimination but more sexual harassment so in both cases that have women in both countries say it's just really nice not have to deal with like the sleazy boss that's telling but if that gives somebody the opportunity to not feel that the oppressiveness of that I would not want to minimize that and at the same time I think it becomes a call for how do we still have the persistent sex sexual harassment in workplaces that makes it feel like a relief that you don't have to go to an office like can we answer both those at the same time that would be awesome well let us thank you thank you can you talk a little bit about the process of collaborating with a computer scientist and how computer science other than the nice network visualization that she showed us plays a role in this yeah sit and I talk quite a bit about what made this work and I joke and I actually don't think I don't want to minimize this his partner is a feminist archaeologist there are like five of them in the world and I think that gives him great capacity for being able to talk with anthropologists and understand the value of qualitative research so it really for me starts with collaborators being able to see that their tools are not the only tools in the shed to feel like they may have really great tools for answering parts of the question but that their places where somebody else's approach will be able to shine a different light a different angle on the subject material and I think that that's at the core of what made this such a fruitful and generative collaboration that we really could see I cannot and never have imagined being able to picture the distribution of people I could interview like that's just not I mean unless you're working in a really small place you really can't see that as an ethnographer to be able to see it in a distributed system was really profound for me or to be able to to be able to verify the amount of collaboration I was seeing ethnographically with a quantitative measurement that was going to be so satisfying to the folks in the room who need numbers that was that was really helpful and I think the toughest moments were were when I could see what he was showing me quantitatively and that it didn't feel like it was enough to see it qualitatively and I knew it was enough for him and I think it's enough for me maybe maybe but to be able to see the possibility of both those approaches really resonating I I think we're still working out what it looks like to formalize that but but at the very minimum I think it requires really seeing the value of the approach of the other of the other parties and I don't think we're built to do that in our programs that's really tough to get there please join me in thanking Mary