 Welcome, everybody, quite an impressive group. Ben Michael Hill is an activist, scholar, and consultant working on issues of technology in society. He is a joint PhD student at the MIT Sloan School of Management under Eric Von Hippel and also at the MIT Media Lab. He's also a fellow at the Birken Center this year and is also a fellow at the MIT Center for Civic Media. And he's here to talk to us today about his work. Welcome, Ben. All right, thanks. OK, so all right. So I'll prefix this by just saying a couple little notes. One is that this is, all goes well, part of my dissertation, and also an academic paper, which is sort of framed in terms of the sociological literature on social movements and innovation. Now I understand now, even more than I did five minutes ago, that this audience is much broader than that. So what I've tried to do in terms of framing this for this audience is focusing a little bit more on the phenomena, both the phenomena, the sociological theory. Although I would encourage anyone who is interested in that side to talk to me, I'm happy to circulate drafts of the paper, that kind of stuff. The other thing I just wanted to say is that this is unpublished work. And I'm interested in spreading this to the world. That's why I'm here talking. But if you want to cite or quote it, I'd appreciate if you ask me, mostly so I can make sure you get an up-to-date copy of whatever it is that I'm working on that reflects the newest ideas. And then the final thing is that this is qualitative work you'll see and involves a lot of presenting my data, my quotes from people. I have permission from everyone so far to show their names. But I think for the interest of, for serious reasons, I'm hiding the specific names, but revealing the names of the projects so you don't get surprised. All right. So I'm framing this work broadly. I'll take a little bit about my broader research. It is research into peer production, which is this large and growing interest in online voluntary driven projects, free and open-source software, and actually where I come from as a community. I've been a developer and a bunch of free software projects for a long time before I decided I wanted to become an academic sociologist. Also a lot of work in remixing. I've done a little bit of that work in remixing communities. And then, of course, there's been lots and lots of research in Wikipedia. And I could say wikis more generally, but actually almost all of this work has been on Wikipedia per se. I mean, the original peer production article actually used Wikipedia back when it was, before it was, you guessed right, OK, I guessed right on that. And it was a little project with 10,000 people or something, 10,000 articles. And it said, wow, maybe this will turn out to be something kind of big. And as it turned out, it did. Now this research has really focused on, has been driven by an interest in the collaborative potential of this type of organization. And basically the benefits that this sort of inexpensive, large-scale collaboration can bring. The people have looked at projects like the Apache web server or the Linux kernel or Wikipedia and said, wow, look at all that great collaborative work that can be possible. Of course, the vast majority of this work has focused on the projects that did become large, successful collaborations, because those were the ones that were visible and those were the ones that were impressive. Nowhere has to spend more clear than in the context of Wikipedia. This is just the number of academic articles with Wikipedia and the title over time, and which you can see is there's now more than 3,500 peer-reviewed academic papers which have been published about Wikipedia, just like an enormous amount of scholarly interest. There are actually two conferences, one academic, one not academic, which are dedicated to people presenting studies of Wikipedia. There's one of which just happened last week. I was unfortunately couldn't go to it. But maybe some other people here were able to. So there's been an enormous sort of academic interest in Wikipedia, per se. And I think that it's worth sort of unpacking exactly what Raymond's model is here when we look at these sort of collaborative projects like Wikipedia. And this is sort of taken from my description of what the model behind the interest in these sort of large collaboration comes from. It's this idea which was described most famously in the context of free and open source software, open source, in particular by Eric Raymond, but also sort of picked up by a lot of academics who suggested this is actually a model which businesses, for example, can benefit from. And the model works something like this. You publish your work openly, release it under a license that's a free software license, or an open source license. In Eric Raymond's model, with enough eyes, all bugs become shallow. Basically, the community comes in. They see the bugs in your software, or they see the missing information in your encyclopedia. They begin to fix it or improve it over time. And then you end up with this stuff which is of higher quality. The reason that the Apache web server is better, according to this model, is because it's open and the community has come in and really been able to help improve it. The fact that there are so many people, because it's open, there are so many people contributing to it, finding those bugs and fixing those bugs. And the result is stuff which can out-compete the proprietary alternatives. But I want to suggest that there's a less studied sort of part of here, which is that there's this missing piece of the model, which is the attracting contributors. In most of these projects, right now I'm still talking in terms of my broader sort of research. But there's this, many people who tried this have realized that actually this is a little more difficult than it sometimes looks. I don't know, has anyone here tried to start a free or open source software project? Has anyone tried to create a Wiki, for example? Public Wiki, not just one for themselves, but one that was designed for a community to get involved in, right? Does the original, this is a model of a community improves higher quality, like result in that it really makes sense to you? Was it your experience, sometimes, a little bit? So I've been doing this for a long time, and this wasn't my experience. What do people think the, I mean, I like to ask this question, but in free and open source software projects, what do people think the median number of contributors are to a free and open source software project? By looking at, and I have data from about a dozen different websites that host free software projects or open source software projects. Median number of contributors. Any guesses? One source code fix? Yes, we can do one source code fix. Sure. One, what? Seven. One, one. 25. 25? Three. Right, so here's the median is one. This is from SourceForge. If I limit it to only projects which have been downloaded 100 times, the median is two. If I look at it to projects which have been released several times, which have had a series of contributions to them, the median is still one. It's very hard to fight this. If I look for data, I'm not going to show you all these, but if you look at data from Google Code, if we look at data from all different places, it's very, very hard to fight this power law function here. The vast majority of attempts at creating free and open source software projects never become collaborative at all. Here's from a very different place. This is Scratch. It's a remixing community. Scratch is a reference to hip hop scratching. In the ID, it says community for remixing. The vast majority of projects are contributed to by exactly one person, or never remixed. You look at this, I've looked at dozens of communities of people attempting to create these sorts of projects. And this is like a law of nature. It's everywhere. And it's very hard to make it go away by trying to reframe this. Yes? The people who participate in these communities don't actually care about the remix angle and only at academics or other community extremists or something, actually care about the remixing. So if the producers aren't excited about that, and maybe their use of the service would differ from what you look for. It's certainly true that if you talk to, when you ask participants in these communities, you say, are you really upset that you don't have contributors to their sites? The result from the interviews, or at least the write-ups that I've seen, has basically suggested that people are basically OK with the fact that there aren't other people contributing. They're not incredibly upset about this fact. But in many cases, especially after the first project, they sort of expected it to be more successful and would have been happier had people show up. So I think that that's my reading of some of the work in that area. That's a question that people have asked. And one of the things you do is you say people have talked a lot about what success should mean in a free and open source software project, right? Are these projects unsuccessful? And I think that if you ask the people who are creating the projects what successful means, they don't actually need it. They'll say, I'm willing to consider this project successful, even though I didn't get anyone contributing to it. Because I released it a few times. It solved my problem, whatever. So I think that that's fair to say that this isn't necessarily a critical problem for free and open source software or for Wikis. But I think that it is a critical problem for a theory that doesn't look at this missing step in the sense that it assumes that it will work out when actually it almost never does. In science studies, you look at the frequency of citations of papers. That's right. Most papers have been cited. That's right. Zero times. That's right. I see. Let's go for it. OK. Sorry. Research has shown that only a small portion of these peer production projects attract any participants. This has been shown by a bunch of different people in free and open source software. It's a well-known thing, at least among a subset of academics who care about this. It's been shown in remixing communities. It's been shown in Wikis as well by taking big Wiki hosting sites. And you look at actually most Wikis that are attempted to be big collaborative projects never really take off. So that's my research question for this study, but not just for this study. It's actually also my research question for that. That's the research question that my dissertation is trying to deal at. Sort of explained in a very phenomenal, like phenomena focused way. Why does some peer production projects successfully attract contributors while most don't? And I want to point out a few things that I'm doing here. One is that I'm taking projects as the unit of analysis, which is something that historically a lot of the academic literature is saying this has not done. They've actually focused on the individuals. You've looked at, you've gone to the Apache Project for the Linux kernel and you've interviewed a lots of people and you say, which ones are contributing more or less and how does the ways in which they say they're motivated based on some survey sort of result. That's like an example of the type of work that are here. The second is that I'm focusing on mobilization. That is that my dependent variable, if you want to think of it here, is actually the, it is the getting lots of people to contribute, right? Now those people might be contributing. In this sense, we might think of a community like 4chan, who's producing things that many people in the world think are maybe not of value to the world. If you don't know what 4chan is, don't look it up. But there are a bunch of communities that have successfully mobilized people but who maybe aren't creating things that everyone agrees are of high value, even though they're very productive. So what I'm looking at here is mobilization. I'm not actually talking about quality. I'm talking about lots of people coming in. And so I'm sort of in some ways buying into that or at least assuming the second half of that Raymond model that if we get lots of contributions we will result in higher quality. At least I'm at least suggesting that that might be the case. And the third thing I'm looking for is contributions towards a project's goal. In the sense that you'll see in my data this is actually a pretty important thing, that there are some projects who can get lots of people to join. You can get lots of people to sign up but no one actually does anything. No one really contributes and maybe you've had that experience as well. And so for me, attracting contributors means that you can't just attract them. They also have to become contributors. And then as I've already mentioned some of the, a lot of the literature so far is focused on individual motivations. So for example, decreased transaction costs. I'm gonna come back and talk about that. And reputation. So to look at this today, as you know from my title and abstract, we will start it. I'm looking at sort of failed Wikipedia. So we can think of this as, if you're interested in the problem with looking only at the successful projects it's sometimes called sort of selecting on the dependent variable because if we care about mobilization, if we only look at the projects who successfully mobilized, we actually have trouble understanding why it is that those ones were able to succeed. Because we're only looking at the success cases. So my research is in general based on taking big populations of attempts at these sorts of projects, right? Pure production projects. And looking at why some of them get traction and attract contributors, why some of them mobilize, while other ones don't. And luckily for me, Wikipedia is not only the, Wikipedia being this sort of poster child of pure production from the very beginning and now the subject of this enormous amount of academic research. It turns out there were, it would be great, right? If we had, look at all the failed Wikipedia's. Well luckily for me and maybe for all of us, there were a bunch of failed Wikipedia's. At least sort of like a failed Wikipedia's that happened before Wikipedia was launched. There have been some that have been launched afterwards. I haven't included them in this right up, though I've done some interviews of some ones afterwards and we can talk about that later if there's time. Now, all of these projects, now what does it mean for something to be a failed Wikipedia, right? I'm making a claim that it's similar to Wikipedia. All projects were self-identified as encyclopedias or if they didn't call themselves encyclopedias and that's kind of important. They were referred to as encyclopedias in the press in the sense that people were writing them up and said there's this new encyclopedia project. We're very interested to hear that in 2000 there was actually an article in USA Today about online encyclopedia projects which didn't mention Wikipedia because of course it didn't exist yet. It was about three other projects which probably no one had heard about in this room had heard about. I also asked each of the founders of each of these projects if they basically had described their project, like you think your project will be as an encyclopedia, they all basically confirmed that. In all cases the content of the encyclopedia was crowdsourced from volunteers. They didn't pay people to do work on this. I have one project which I've not included in here which was an online encyclopedia project which was paying people in advertising revenue or a portion of advertising revenue for that. I've not included that in order to keep this sort of crisp although including it I think would keep my results basically unchanged. I think it's a little bit different. And then the sample includes the full population of projects that were publicly announced to my knowledge. If you know of other projects I would love to hear of them. I'd like this to be comprehensive. I think so far it is but if you know of more I'd love to hear it. This is my research setting. These are the eight projects that I've included here. I can walk through this a little bit. So the first project, these are sorted up, it's sorted in terms of time from when they were founded. The first project was founded in the early 90s before the web called the Interpedia. From my records it looks like there are about 400 people that participated but they only produced a small number of articles. The second project was the Distributed Encyclopedia which was a project which was kind of a cool idea. It was certainly, it was a web project but it was the idea was that you would create encyclopedia articles on your own webpage and then you would email someone and then they would sort of add it to the list of articles and if you wanted to modify someone else's article you could take it and modify it and then they would link to both and there would be some sort of people could create competing indexes and the idea was that it would sort of work on this way. Again a little bit different, a little bit earlier as well. As far as I can tell they never got anyone else to start writing articles but did get a bunch of people who were interested in it. This is one that I've had trouble getting a lot of good data from. I've talked to the founder but this is the thinnest date on this one. The second project was everything. Everything was a project. It's also called Everything 2. This is one that some people may have heard of. It's actually still around today. It's a project which ended up getting 50,000 contributors and building more than half a million articles. So been pretty successful. Although it sort of, it saw itself as, it didn't want to constrain itself to traditional sort of definitions of encyclopedia and we'll come back and talk about that. But it saw itself as sort of really being a database for everything. But early on especially, so everything, it's interesting. A few of these projects that exist today, they sort of identified their marginal utility when Wikipedia was released. So everything today is much more focused on like creative writing. And part of that is because after in a world where Wikipedia is so big, they sort of decided okay, well Wikipedia is already doing this thing that we thought we were doing originally. Maybe we should sort of retrench it. So that's changed with time. But early on, looking at some of the archives, it seems like calling in a encyclopedia project is pretty valid. So you call it a failure, sort of stands out in that list. That's right. We'll talk about this as well. So there's, not everything is equally a failure in this mode. And everything is by far the most successful project in this list that's not Wikipedia. H2G2 is a project which was the Hitchhiker's Guide to the Galaxy. It was created by, it was created by basically by Douglas Adams, the author of the Hitchhiker's Guide to the Galaxy. Not him personally, but sort of a company that he was somehow the inspiration for. It is currently being maintained by the BBC. Although it was on the list of things that the BBC is gonna cut due to budget cuts in the British government. So I think that there are some people in the community thinking about what they're gonna do with it. But it was supposed to be the Hitchhiker's Guide to the Galaxy Earth Edition. So the idea was it was an encyclopedia about Earth, but written in the style of the Hitchhiker's Guide. The Info Network was a small project created in 2000. One of the ones that was written up here. It was in many ways the project which was most like Wikipedia. When I look at the early versions of that site, it looks more like Wikipedia does today than Wikipedia did early on. A very sort of similar type of project, although again a little bit broader in the way that they thought of what should be included in the info. We have maybe one contributor to the info here, maybe, no. Okay. GNUpedia was a project which, so GNUpedia is one people have probably heard of. It was the project created by the founders of Wikipedia. But was more of a, users more traditional sort of Wikipedia Encyclopedia Authorship Model. Got 2,000 participants involved, but again only 24 articles. GNUpedia was a project which was created by some people from the free software community. And similar sort of idea, more traditional sort of Encyclopedia Authorship Model. And then of course Wikipedia is the project that everyone knows. Yeah. You've used participants and contributors. Are they interchangeable? So they're interchangeable. That's right. That's right, yeah. So this is a complete list of all the projects, just the ones you're focusing on. This is a complete list of all the ones that I know about that existed before 2001. Do you know one that I know? So we're created before Wikipedia. Since Wikipedia was founded, there have been a bunch of people who've decided that they want to do better or different than Wikipedia and have created projects subsequently. But I'm looking at all the projects that existed before because this is a sense in which it's not clear that anyone of these is necessarily going to be a success. Now one thing that I'm doing right now is I'm actually expanding this a little bit in the sense that I'm gonna probably go forward one year and I'm looking at a few projects that were founded in the year after Wikipedia. But before Wikipedia was clearly success. There's a sense in which Wikipedia was smaller than everything, for example, for much of the first year. And so I think it's a little bit, some criticism I've got when I presented this a month or so ago, or a couple months ago, was that I should really be looking at these, I should be, my cutoff should not be when Wikipedia was founded. It should be when Wikipedia was clearly the project that you wanted to contribute to if you wanted to work on an encyclopedia. I think that that's a valid criticism. If you know of, I've got a list of those projects but I'm interested in hearing those as well, but maybe afterwards. Yeah. I'll ask more about it later, but I'm just curious about your delimiting projects that are aiming to be a universal encyclopedia. That's right, general purpose encyclopedias. You know, every fan culture has its pedia. That's right, although most of those, the ones that are called pedia are mostly came after Wikipedia. That's right. But there was, there were similar projects at that. I've also only made the only ones that are in English. There was one in German that existed before, which I've not included in here. Most of them don't speak German, so it's hard for me to look at the archives. But that's right. I should be clear about what this is referring to. Okay. This is to give you an idea of the timing of the process. One argument is that, well, maybe these projects just came to, they were just ahead of their time, right? And I think that that, you could maybe make that argument, it's pretty easy to make that argument for interpedia, which really did seem to be, if not ahead of its time, ahead of Wikipedia's time, before the web, maybe that's part of the story. It's interesting, I've asked the founders of the projects and they don't think so. Similar with, you might be able to make that argument with distributed encyclopedia project as well, which is that all the rest of them continued going and were active before and after Wikipedia. So actually, most of these projects, and if you think about it, it's sort of, this is the era when the web is sort of exploding. Lots of people are making these websites. Most of these projects came around at about the same time. Now, in terms of my data, this is, I used, most of the data here is sort of semi-structured interviews in the sense that I had a bunch of open-ended questions. Yeah. Wikipedia was clearly the product we wanted to contribute to, starting around maybe mid-2002. Right. So I wonder if all of these products would have survived and happily lived on in the work of Wikipedia eating their market. That's possible. That's a... I'll never know. It's an interesting question, yeah. But that could be the real killing factor, it's not too soon, it's just too much Wikipedia. Right. Okay. So my data here is mostly consists of interviews of founders of each of these projects. I sort of tracked down the founders of each of these projects. I did one in-person interview, the rest have been on the phone. I did one over email. A number of these projects had more than one founder, so I've included those more than once. I've also put together as much archival data as I can, so a lot of time in the wayback machine, grabbing as many of these pages so I can actually see the process. I've managed to get Usenet data, so the conversations. So, and also I've gotten the people's personal email in some cases, they've managed people who've kept it, they've been able to give me their personal email so I can look through that. And then also sort of discussions about the projects, which adds up to about 3,000 pages of data. I've also got newspaper and media coverage of all the projects, I think comprehensive. And then what's interesting is that for four of the eight projects, I actually have pre-launch planning and discussions. So I can actually, and what I do here is I actually sort of cross-reference these. So in a couple cases, people have said things, told me things in interviews that I went back and looked in the data and I found, I find more and less support. And so what I do for some of the, some of people's recollections. I mean, this is of course 10 years later in most cases. So I've tried to get as much data as I can from different types so I can sort of piece it together and try to figure out what really went on. The methods is sort of a multiple case study analysis using sort of iterative stepwise coding, which basically means that I recorded each of the interviews and then had all this sort of archival data. I transcribed all of that, all those interviews for my sins. And then I basically went through and coded it. You can think of it as sort of tagging that data. I went through and I tagged it in a sort of, sort of in inductive way, in the sense that I looked for themes which are emergent in the text. And then in, with deductive codes as well. So I looked in the sort of academic literature on this topic and coded it as well. I used some piece of free software called RQDA to do that. I then categorized the codes. I grouped them and then I used the themes to create a set of hypotheses and propositions. Now, I will say one thing which is that, this is the only piece of, this is the only qualitative project that I'm working on. I'm primarily a quantitative person. I'm presenting this here in part because I think it's hopefully of interest to people, but also because I think that it's a little bit, it's easier to understand whether the data is texts and quotes and interviews and stuff like that. That said, I'll say one limitation of this type of work is that I can't test these hypotheses, right? I can create a set of propositions or a set of sort of suggestions for why I think these projects work, but I can't test them in a scientific sense, right? That's just a limitation of these methods. So that said, I can test them in other data sets and we'll talk about how I plan on testing some of these hypotheses in the other parts of my dissertation. So anyway, I had four key themes which emerged from, or propositions, which emerged from coding. I'll go through these quickly and then I'll talk about each of them in depth and provide some sort of background for this. The first was that Wikipedia attracted contributors because it was built around a familiar product. It was built around something like an encyclopedia, something that people were familiar with. The second is that they attracted contributors because it was focused on substantive content development instead of technology and that goes, I think, again, goes against some of the common ways of thinking about Wikipedia. The third was that it attracted contributors because it offered low transaction cost participation, which is something which is actually pointed to by Yochai's work and by a number of other people. And the fourth was that Wikipedia attracted contributors because it de-emphasized attribution and social ownership of the content. So I'll talk, this is the rest of my talk, so I'll talk plenty about these. And of course I can't speak causally and I can't test these, how causally he's using these data and these methods, so. The first, the first proposition, that Wikipedia's familiar product was a key to its success. Here's a quote from the founder, one of the members of the founder funding team of Everything2. I've interviewed a few people from that project. He said, I don't think we ever used the term encyclopedia and that probably would have been smart. Wikipedia had a much more focused purpose than Everything2. Everything2 was sort of by its nature, sort of Zen colon-like. Everyone who's involved with it thought it completely defied description and that I think was ultimately to his detriment. Versus Wikipedia which was like, we're going to be like an encyclopedia, like the world book encyclopedia, but huge and comprehensive. We're gonna keep this impartial tone and everything has to be referenced and that sort of thing. Right, so the idea here is that Wikipedia attracts contributors because its product or goal is familiar to the potential contributors. And the failed projects tended to deviate from this model. Now Joseph Regal who is, I don't know if he's still involved in the century, he was certainly a fellow last year, wrote a wonderful book on Wikipedia and one of the things he does in his second chapter is really sort of put Wikipedia into context. He says that Wikipedia, there's a sense in which people say Wikipedia is this radically new type of product. And he says actually Wikipedia is very much in this long series of what he calls the encyclopedic impulse or something like that. And that Wikipedia really is best seen in this series of going back to Deidre of people producing very similar types of encyclopedias. And I think that you can think about this in terms of, there's a literature in sociology on frames which basically suggests that we have like kind of templates in our mind which we can bring to bear on a situation. And there's been a whole bunch of work in different parts of sociology which have suggested that common frames can help sort of certainly making understand things easier but also can lay the groundwork for potentially for mobilization for contributing. So you can think about this in terms of, I mean I think that if you've contributed to Wikipedia you understand that there are really thousands and thousands of pages of guidelines and rules. But if you basically have ever, if you know what an encyclopedia is, you don't really need to read almost any of them. For example, one of the, a key rule in Wikipedia is around notability, right? What articles will get in? What articles won't? And they have to be notable. Now, if you basically know the type of things that are in the world book or Britannica, you basically, you understand enough about notability to wing it, at least for a while. If you understand how encyclopedias are written, you basically understand neutral point of view in the sense to just write like Britannica and you'll be pretty close. Now, the failed attempts often expanded or sort of altered this encyclopedic frame. So everything too, for example, was described on its own website as a flexible web database created by Blockstackers Incorporated, and then the company we started it. Which seeks to find the best way to store and link ideas. The result, it's absolutely crazy. And they have this, this colon thing is absolutely true. You read the article about what is everything to and they will bend over backwards to not tell you what everything to is. Because they didn't want to, they didn't want to constrain everything to. Because, you know, right. The info network, his founder said, I don't think I conceived of it as let's put it encyclopedia online. I think I probably thought, like, this is gonna be an exploration or we're gonna figure out what a reference work online looks like. It's this idea that in the context of the internet, why should we be constrained by these previous models of production? When we could build on this and create things which are more. In the sense that the World Book, for example, their ideas of notability are at some level constrained by the fact that they're only gonna print. You know, they're gonna, you know, the books, they're only gonna make this many books, which, although it's a lot of books, is only a subset of the topic. Somebody's not gonna get written about. And the concept of notability plays a very important role in deciding what does and doesn't get an article. Contributors struggled with goals that diverged even slightly from this tradition. So H2G2, again, this is the project which was like the Hitchhiker's Guide, right? It was very, very similar to the Earth Edition. But it was supposed to be with a different tone. And the fact that it deviated made it very hard for people to contribute. So this is a quote from one of their founding team members. He said that one of the problems was firstly that people would be writing completely fictional stuff about the universe, you know, about the Hitchhiker's universe. And we'd go, no, no, no, no, you're not getting it. This is for real people. This is about the real world. Then what they did at the same time was that they'd also do stuff about the real world, that they'd try to write it from the point of view of an intergalactic guide. So we get articles about soccer that would start with. On planet Earth, which is the third planet out from the solar system's soul, the humans like to play blah, blah, blah, blah. Shut up, all right? It's like this is going to be read by humans who live on Earth. We had piles and piles of this shit and we had to shovel our way out from under it, right? So the idea is that even though they diverged a little bit, it created barriers to sort of collective action in the sense that if we think of an important part of mobilization, sort of getting people on the same page, this can provide an important sort of barrier to this. Now I'll say one thing that if you look at a lot of the most successful free and open source software projects, almost, and this is just, this is not a scientific observation, this is just my sort of idea that if you look at most of them, almost all of them are reproducing, or many of them are reproducing applications that we're actually pretty familiar with. Your photo application is actually quite a lot like some of the existing photo applications. So. But also if you go back and flip around the way that you're framing that so that part of what happened here is that the project controllers were telling the users you're doing it wrong and if they had encouraged this type of writing they might have actually continued to grow and gather a really large community of people who wanted to write stuff in that type of frame and so I mean that's possible. So everything too is much more than an example of that in the sense that everything too really didn't tell people to do or not to do particular things. They really said we're for everything. Over time they sort of, I mean they reinforced positive reinforcement on things that they sort of appreciated but I think that that's a good point. But I think there are some other examples of projects that did that that also struggled in the long term to get the kind of traction that we competed in. Yeah. You think like salvageability plays a role here like in everything salvageable because you're just publishing it and we could give you like most things that are publishable because you can turn them into but this like these are completely unsalvageable because you can't just make minor bits of them. That's a good, that's a good, that's a nice suggestion. All right, I'll go back and watch the recording. Someone watching the live stream asked if you could repeat questions. Okay, that question was do we think salvageability of the content in the sense that if, you know, the sentence that's quoted here is unsalvageable but in everything too people might be able to improve it or work it into something else and I think that's a good suggestion. That's something I can look at. I mean, I'll be going back to this data so suggestions of things I should look out for is something that's are useful. Yeah. I think another piece of it is that you say you can publish everything here, you're competing with the web as a whole because the web exists and you can publish everything there and you have a big audience. And so I think part of it is of that framing is that the contribution of a project is to be a constraining factor because you actually have a completely unconstrained publishing platform that is going to, that might give you more publicity that's something like everything too. Yeah, so the idea is that if you do everything you're sort of, aren't you competing with the web? And I think that, I mean, seriously, people talked about that. Interpedia, for example, which kind of came about before the web, thought that, I mean, they said things like, we thought we might be able to be the internet, which was kind of an interesting concept, but yeah. I wanted to ask about that part of it because it was before the web, were they based on Gopher or some other technology? They had, they built a series of different interfaces which was a very sort of technologically focused project and I'll talk about that, but yes, they were based on downloadable things on FTP, they were based on Gopher and they also, in towards the end, had to build a web client as the web was sort of coming about. Right, so my second hypothesis is that Wikipedia is sort of substantive focus on content and not on technologies was important because it left its founders, founders were focused on evangelizing and attracting content creators. The field projects often focused on building the technological capacity to support these contributions. So in contrast to a lot of these arguments that suggested that Wikipedia, that peer production projects succeed because it's just like new technology makes it possible that actually the strong focus on technology and making things easier seemed to distract from the real goal of getting people to contribute. All of the projects except Wikipedia and Newpedia which were founded by the same team were founded by technologists, which in the early days of the web was not too surprising. All of the projects, all of the projects, including Newpedia, other than Wikipedia, wrote their own software. Wikipedia was the only project which did not write custom software in the entire sample. They just used a Wiki software which was somewhere on the web. Many of the projects were very focused on what I'm willing to suggest are sort of technical minutiae or redundant systems. The founder of the info mentioned the superior quality of his URLs four times in a one hour interview. And he's right, he's right. Wikipedia URLs are really long and ugly. His URLs were much, much better, but I don't think that that was the deciding factor. The GNE, which was originally Gnupedia, that project had three competing technological implementations, 200 participants and they never got farther than producing test content. They had huge numbers of people working together to build infrastructure and no one actually contributing to the article. In Gnupedia had four software development projects, including the Gopher version, 400 participants and again, very little content. So here's a quote from a founder of the info who described his role in the project. I had this notion that my job was to provide the platform. Like my job was writing the code and not the content. That was the community's job. But since there was no community, it just didn't happen. And so I feel like I kept trying to refine the user interface and things like make it more inviting so more people would write stuff. But I didn't realize that to really get started, I just had to, there's only so much you can do by making the interface easier to use. You just have to get writers or write stuff yourself. One of the founders of GNE described the necessary resources in technical terms. And I said, what was necessary to produce the encyclopedia or into your project? And he says, well, in the way we were developing the prototype, we started to look at what kind of server capacity, for example, we'd need, how much content we'd need, how much space, how much room, bandwidth, all that kind of stuff and loads of other technical challenges. Wikipedia was and still is, according to some of these, to a lot of the people that I've interviewed, seen as sort of technically unsophisticated. A founder of Intrapedia said, a lot of the stuff in Wikipedia is extremely obvious and not very sophisticated. I mean, the Wikipedia is not high tech. I always imagine something high tech. That's my nature. I envision things that are at a higher technical level. We envision for the Intrapedia as something that would be high tech. And you could see the Intrapedia inspiring Wikipedia, but not the other way around. I mean, the question that I asked here was something like, do you think that there's any ways in which your project could have been inspiring for Wikipedia or something like that, or things that it could have learned from you? It says no, not really. That said, the Wikipedia's founders really focused on content. And the people that I've interviewed, I ask every interviewer to talk about all of the other projects, because most of them know it. And to talk about Wikipedia, it's interesting. As you imagine, I mean, this isn't in the talk, but you'll ask people like, these people have all spent a lot of time thinking about why their project didn't take off and why Wikipedia did. And I ask everyone, so how do you feel? Like having basically created something like Wikipedia before, and they're like, what do you mean, am I bitter? It's like, no, no, no. I got over that. And I would say that the common thing is that everyone said, yeah, for a while, I was a little annoyed. But I've sort of come to terms with it. What's interesting is that many of the contributors, many of the founders here, actually now became major contributors to Wikipedia, quite a few of them. And most of the other ones have certainly looked at it. But I can tell you, all of them have thought deeply about why it is that their project didn't work. So it's really great people to be asking here, because they've been basically doing my research project for the last 10 years individually. So anyway, they all saw Wikipedia's founders' role soliciting content rather than building infrastructure, is the archival data supports. So again, one of the founders explained, I think the smart thing that Larry Sanger, one of the co-founders of Wikipedia did, and that I would probably try to do, if I did the project again today, would be to solicit academic experts and other experts and try to get people to write and see the articles. Founder of Everything 2 cited the role of Jimmy Wales. Wikipedia's Jimmy Wales is an evangelist, and the reason for Wikipedia's success. And that's part of what probably held E2 back, everything to, to some extent, was that we didn't have a strong evangelist out front. Now, yeah. But I mean, either of these were as constrained as Wikipedia in terms of content, right? Like everything to head over things. That's right. If it had evangelists, like it wouldn't be encyclopedia evangelist. And I remember being in front of our head like, sort of like, how-to tutorials and why. Yeah. Wikipedia also has how-to tutorials. Whether or not you need to, what? But it also has Jimmy Wales. But it also has Jimmy Wales. That's right. That said, both of these projects had enormous press. In the sense that both of these articles were, they were two of the three featured articles in the USA Today article. Every project in my sample, with the exception of Interpedia, had write-ups in major national newspapers, at least. And were cited in a variety of places. The Founders of Everything 2 also founded Slash. And advertised the project very heavily, from Slash at the time. So none of these projects were, I think, hurting for, were hurting for exposure. These projects, certainly. Do you think it's possible that if they had, I mean like, read like, if the info had people who were writing lots of articles, but they weren't sort of academically engaged? Maybe, I don't know. Michael, do you have anything more to help me scaffold partial success? Because everything to, how do you go from 1,000 to 50,000? And then maybe there's something more about something that's a community, as opposed to producing an object or a platform. But how do you go from 1,000 to 50,000? So I haven't framed this in terms of a stage model, although I have one at the end of my slides. Well, let's come to that at the end. If you don't think that I've answered that well enough, which I think is possible, let's come back and talk about that at the end. Because my other question, which is not your responsibility to answer is, how does this relate to other things like YouTube versus, because it's not all open source. Like it opens the frame too wide, but that's, I'm asking how we can translate some of what you're saying. So I'd like to imagine that these results are extremely general. Everyone, everyone should use it and cite my paper when they do. Let's talk about that at the end, hopefully. I'll get through. I have two more hypotheses to get through. So the third is lower transaction cost. This is the one that is certainly most supported by the literature. And I basically found support for this as well. So the idea is that Wikipedia attracted contributors because it lowered the costs around transactions. So we can think about, and the failed models had higher costs. So if we think about this in terms of like, think of this in terms of you can contribute to, if you see a mistake in Britannica, you can contribute the answer. But you have to go and find the address of Britannica. You have to get a type of a little letter or write it on the back of a postcard. You need to find a stamp. You need to mail it. I'm annoyed. Last week, I think for the first time I fixed, and I edit in Wikipedia all the time, I saw a missing comma and added it. My example is always like adding the missing comma, but I don't think I'd ever done it. And I added it, and I was like, I'm actually adding a missing comma. But there are these people who see the missing comma and things, and then we'll want it. I was annoyed by that missing comma. But was I so annoyed that I would have written it on a piece of paper and found a stamp and mailed it? No, I was not that annoyed. I was a little bit annoyed. I was one edit button annoyed. And in the sense that the costs of that collaboration now or participating in that or contributing have gone down, I became in that particular moment more likely to do it. Yeah. Well, there's another part that maybe you get into later, but it's not part of your statement. OK, if I find a mistake in Britannica and I write a letter, it might be several years before I see the result of doing that. I'm not going to get into that. But that's where it is with Wikipedia. Well, I fix it. It's there five seconds later. So right. Now, the reason that this is a little bit of a bad example, because of course, I'm comparing this. I'm not comparing this to Britannica. I'm comparing Wikipedia to all these other projects. And my suggestion is that Wikipedia made the cost of contributing lower than almost all these other projects. I'm also curious, in these other projects, if I made a change, did it have to go through a curation and moderation process? Oh, there was variation in that. That's a type of transaction cost, too, although it's not my effort. So you can see, how long does it take for me to actually get a result of what I did? So there are two ways of understanding conceptually what's going on in that example of having to go through a curation. I'll put that in two pieces. One is in terms of cost, but I want to make this more like the mechanical cost. And the second thing is more of the social cost of having to go through that. That I will talk about. OK. So the frequently cited costs in interviews have been things like account creation, logging in, learning a markup, as opposed to just having like a more open office-like interface. And there's been some suggestion by Yochai and others that these lower costs create opportunities for peer production like Wikipedia. Wikipedia definitely did offer lower costs associated with contributing compared to some of the projects. For example, one of the founders of HGG said, I think it's the immediacy of it. And certainly one that gets to your point. The aspects of the fact that you don't have to sign up for to edit, you can look at a page and see something wrong and immediately edit without having to do anything else. You can come along and do a drive-by edit and never be involved again and make a contribution. You can't do a drive-by edit on almost any other project. So that's sort of support for this basic idea. Again, the distributed encyclopedia project, founder said, the distributed encyclopedia failed because building encyclopedia using handcrafted HTML, so having to write the code, although it's still mostly like text, is still too complex. Wikis solved that ladder issue nicely. It went on to become a major contributor to Wikipedia as well. Two of them? Two complex. Two complex, right. Yeah. Where's the button? All right. But cost alone seemed limited in explaining the failures of the projects. Although this is what I walked in is what I thought would be the dominant explanation. Only one founder suggested that the lower contributions were the most important reason for success. And I asked people to rank these things as well. Several projects argued that their project was effectively as easy to contribute to as Wikipedia. For example, one of the founders said, one problem was a mandatory preview step before you saved it. We probably wasn't enough to kill the site single-handedly. But I probably would have changed it if given the opportunity to do the project again. Wikipedia does fine without that. So this idea that although this was important, or played an important role, most people didn't think it was the most important thing. They thought that some of the other hypotheses that I've already suggested were. Now the fourth proposition is that Wikipedia succeeded because it hid authorship from editors. And the idea is that it had low attribution, much lower attribution of work than any of the other projects. And my suggestion is that this low attribution facilitated less social ownership over the work products and socially risky collaboration. So most of the failed projects used stronger attribution and had more territoriality over the text. So that would be more like this reviewing stage where you have explicit authors. Several projects allowed no direct collaboration on text. As in in the sense that if you wanted to improve someone's work, you needed to copy and paste their work. And you were allowed to do that. All this work was freely available. But you could take it and put it somewhere else, and then work on it there separately. Some other projects did this thing, which is kind of interesting, which is that they would allow anyone to edit the project. But there would be an author. The first person to create it would just be listed as the author. So they'd have their name on it. So even though it was like Wikipedia in the sense that anyone could press the edit button, there was this there was a sense in which there was a, I mean, try to find who wrote something on Wikipedia. It's actually pretty tricky. You can go to the history page. You have to set a count. OK, there's these things. I mean, there's a few people whose work, I guess, academic work has been built on answering these sorts of questions. But it's certainly true that attribution was largely hidden. Also compared to Newpedia. Ashish? What's specific as to which ones allow no direct collaboration? I've got a list of all of these projects coded up with all these hypotheses. So I'll show that in two slides. So yeah. I'm going to go out of the way and assume that anonymous contributions are more prevalent now than Wikipedia's super prevalent. How prevalent were they? Anonymous contributions are more common than projects that don't ascribe ownership to contributions and to creative things. Maybe. Yeah. So the idea is that I'm not a student. How prevalent was it back during these projects? Three of these projects allowed anonymous contributions of various types. Most of them. It depends on which one you got anonymous. Some of them required that you create accounts, but almost none of them required that that account be connected to your identity in any way. So you need to have some persistent identity, but that identity didn't have to be connected to your real identity in any of these cases, except for Wikipedia. I mean, but also like an open source, how typical was it? Anonymous contributions, very unlikely. So again, you could think of this as a lot of the literature, and actually a lot of my own research has focused on the role of reputation and status connected to contributing. And that seems to go against that. There's some great work by Eric Vanipal and Fashar, who's suggested, who used the example of French chefs to suggest that attribution norms can act as a form of intellectual property, like copyright, in terms of protecting ideas. And the suggestion, but I think that there's an interesting suggestion that maybe they impose similar types of costs. And I think that you can sort of think of these as sort of the flip side of reputation-based systems, and that reputation may come with a cost. And we can think of that. Filled projects created, their attribution systems created barriers to collaboration. So someone from everything too said, in Wikipedia, when you submit content, you don't really get authorship credit directly. You appear in the history, but these aren't necessarily your words. They're just sort of your contribution to Wikipedia. But with everything, the writings were still theirs. They had control of them on the site. And they received kind of direct attribution. I think there was some weakness there in that when people wrote something, and if it was factual content, if they had information that was incorrect, there was no real, I mean, occasionally they would go and change it of the content, but otherwise it was sort of up to them to receive communication and re-add to it. All of the photographs of Wikipedia have photographer credits. Yes. Different from text to site. That's right. So that's sort of- You can edit graphics, right? I mean, many, many, many graphics on Wikipedia would have different edits, comments of it. So if you look at diagrams, you'll see that very often there are enormous amounts of collaboration on diagrams even in Wikipedia. And even if you look at most images, it's very unusual to go to an image and actually see that it hasn't been changed at all. Usually it's been edited and cropped, at least maybe adjusted in certain ways to make the article a little better. So I think that it's true that certain types of creative works may lend themselves to, for technical reasons, may lend themselves to different levels of cooperation. But I think that- I think the images is more for legal reasons than technical reasons. I'll defer to you on that, but- But it's a really interesting point. Yeah. I think that you've- This discussion suggests to me that you're combining attribution and control. So this quote here is about control over the work. Whereas, in fact, these images we've talked about, anyone can be the most recent uploader of an image. And it's still that image. So I am trying to do something a little more subtle, which is that I'm trying to describe attribution as a form of social control. So I agree with what you're saying. I'll- I've written that down. Well, maybe we can talk about that later. Another founder suggested that people would con- from Interpedia suggested that people would contribute articles that would be missing or they'd be somewhere down the list below the default article, but they'd be the work of one identifiable person. Unlike Wikipedia, where it'd be very difficult to track down who contributed some messages. And then many of the people suggested in fact we could be succeeded because of the low-textual ownership facilitated more collaboration. So for example, a different founder of everything too suggested that I think that having one article as opposed to several write-ups on a node took advantage of marginal contributions in a way that everything too is not set up to do. That really helped make it much more strong, strongly many hands make lighter work type of exercise. Another different developer from Interpedia suggested that Wikipedia sort of conquered because anyone could just write anything on the page without anyone's approval. And then another one from everything to saying I think that having one article as opposed to several write-ups on a node took advantage of marginal, oh no, that's the same one. Okay, in any case. So anyway, those are my four sort of hypotheses. I wanna try to bring this together a little bit. So these are the four sort of propositions. We don't need to go through them again that emerged from the coding. And I wanted to go back and sort of put them together and know that we can think about them. So here's a two by two sociologists love this. And what I've done here is talked about we can think about sort of like the innovation in terms of process and product. So you can think about the first one, this idea of familiar, the familiar concept, right? Is the projects which have used the ones that said we're building like an old, we're building an encyclopedia. If you know what Britannica is, you know what we're building. And you can see that there is basically four of these projects which really said we're building just an encyclopedia. And then the info was a little bit somewhere in between and HGG was somewhere in between. And they said it's basic encyclopedia but it's gonna be different in some way. And then Interpedia and everything too had much, much broader visions of what a reference work online would look like. And then along, and so that's the access from like sort of established products to sort of very innovative products or goals. Now in terms of my third and fourth hypotheses, this idea of the sort of lower transaction cost and this idea of sort of textual ownership. We can think of these as actually really describing a different way of building an encyclopedia. And that you see, and some of these people are saying, well, we're gonna build in a really different way, we're gonna get rid of authors, we're gonna do it in like this sort of a radically different way. And some of them said, we're gonna build it basically the way that Britannica has done. So down in this corner, we have these three projects who were basically building something like Britannica, some basically like Britannica, except they weren't paying people to do it. And these three are the ones that I think failed the hardest in the sense that they're the ones that really like we're doing Britannica but they never got moving in some of the way that the other ones did. The other two that were more difficult were these ones that, so this is that we're building the established, we're building the established product in the established way. Similarly, the ones that were like innovative product, but we're gonna do it in an innovative, we're gonna build this new thing and build it in a really different way, these are the other ones that really struggled to take off. The ones that were changing everything. Now, the ones that mobilized people that had thousands of contributors, all of these things sort of, they kept one thing, they basically did some stuff a little bit different but they kept stuff constant. So we can think of it as having like, if collective action becomes more difficult when you have all the balls up in the air. If your goal is to give people something that they can sort of understand, either a process that they can understand or a product that they can understand, those are the ones that got a little bit of traction. So the ones along the diagonal. And the only project that had an established product but did it in a new way was Wikipedia. And in the long term, I think that it's interesting to think about what the effect has been because Wikipedia now deletes more, if a new article is created, more of them are deleted than are kept. In part because Wikipedia has, because most of the things, articles that people wanna create seem to fall outside of what people sort of conception of an encyclopedia has been. So that thing which, at least the people I've interviewed, have suggested was critically important for Wikipedia's success in mobilizing contributors. In the long term, helped create, placed barriers in terms of Wikipedia's continued growth. Yeah. Larry Sink of our new product is a related gene epilox. He created this in 2006, but he kind of went back to this. We need a very credentialed peer-reviewed sort of model rather than kind of open source. So I wonder if you could just place CitizenDM here on this track. I guess I would place CitizenDM right here. Like sort of on that line right there. In the sense that they're keeping more of the traditional process than they're keeping more of the traditional process than say Wikipedia, but they're diverging from traditional encyclopedia, the way people write encyclopedias more than say a new pedia did. So then that speaks to one of the quotes I was on a previous slide by one of the founders that said that if we had a stronger sort of peer-reviewed credential sort of system, maybe it would have worked out better, but the experience of new pedia and CitizenDM seems to suggest otherwise. So I'm just wondering how CitizenDM sort of fits into your narrative or how you consider that. So I'm very hesitant. I think that in a world with Wikipedia, I think that it's hard to compare creating an encyclopedia project in a world with that Wikipedia that's creating a new encyclopedia and a project with one. Because there's competition for resources in a way that changes things in ways that I'm very interested in, which I think complicate my story in ways that makes me hesitant to draw conclusions. So I think it's super interesting and maybe we can talk about it, but I'd be a little, at this point, a little hesitant to speculate. So that's, I'll say one thing about future work and then opening up for questions. Basically, my next step is to, I'm doing some more interviews, but then the real thing is that I'm actually testing these, I built a data set of basically now 10,000 attempts at Wikis. And the idea is that I'm gonna code them up in terms of these hypotheses and some other things from the literature and then actually try to do a quantitative test to see if, in fact, how these things sort of wash out. So the idea is to go back to the work that I'm a little more comfortable with. Some of this quantitative work and test it there. Of course, the large majority of Wikis have failed. I've also got data from a different online community, the Scratch community, created by the Life on Kindergarten Group, Mitch Resnick's group at the Media Lab. And Andres Monorakarnadas is a graduate student there who helped build the community. And he and I have collaborated on a few papers together and we're interested in testing some of this work there. So really wonderful, really wonderful data there as well. So hopefully the idea is to understand how specific these, if these apply to other Wikis and how specific they are to Wikis per se and maybe not the broader world of peer production process project. So thanks to a lot of people, including a lot of people I haven't put up here. Susan Silby is the qualitative methods professor at Sociologists at MIT who's helped me a lot with this and then of course a number of other people have as well. So thank you so much. I appreciate it. Thank you. Thanks. Oh wait, I actually, I removed that, oh I'm sorry. Let's see if I can find it. No. I removed my table with all the list of the, I told you it was coming but I must have, oh I'm sorry. Okay, I can pull it up in an old version of the slides. But yeah, any questions? Great talk, Miko. Thanks. One of the things that I thought might have been on this list would be modularity and granularity as you and I mentioned. But I do understand that these are very similar projects so it's hard to, you probably won't see any variation in those. I'm also curious in that these kinds of things are also gonna be correlated with the inability to attribute things to people. If a hundred people have written an article together how are you gonna put a name on it? And it might be interesting to look for examples where the two of those things are not so highly correlated to see if we can figure out where attribution and modularity, where how those contribute or not to the various. Did you take modularity? Did you think about adding that? So a couple people, I mean I can go back and look at the code. I have a code for modularity in my dataset. So it was mentioned but it didn't come through as a major thing that lots of people said. In part I think for the reason you've suggested which is that actually all of these projects were at least as modular as Wikipedia and some of them were more. Okay. Yeah so I actually had, it was a very good talk. First time I managed to say this, it was very interesting. I had more of a comment when you had the two by two up. It actually made me think there's a book by Anna Huff that talks about doing research. And it talks about joining conversations. Huff? Huff, yeah. I can send you the reference. And it talks about joining conversations and it talks about how you can really innovate both in like method and in research question because you don't really have anyone to talk to. And the fact that you have innovation in one of, like the diagonal essentially happening and not the established process established product or innovative, innovative working out kind of resonated with that. Great. I'd love to see that reference. Yeah. Yeah. So yeah I mean in terms of the way that the paper's being framed it's more of this, like it's more in terms of a literature on mobilization and like drawing. So there's a big literature in sociology on social movements and they have a similar problem to the literature on peer production which is that it's hard to see all the people who tried to create a social movement and didn't, no one came to their protest, right? Like, because it was just a guy on the street. And so I'm hoping to sort of leverage these data to help speak to that and the broader sort of research project to speak to that literature. Dennis, there are already some research on the peer side of what the people who were potentially attributing what they say about this that already exists somewhere. So I've interviewed a few people, it's not in this presentation, but I have interviewed a few people who were contributors to different, interview people who were contributors to different projects who weren't, early Wikipedia contributors, but also contributors to some of the other projects in part because I was interested in their perspective on things. So I'm not interested in making that the focus of this research, but I use that as one of the ways in which I try to understand these data. Yeah, I'm sure. So I was trying to reconcile point two and three, which is most people say Wikipedia was technologically unsophisticated. Yes. But they also say that it has low transaction costs, right? Yes. And so how many of these other things, you talked about all of them being videos, but how many of them are Wikis? Is it the only Wikis? I mean, what's, so in the sense that none of them were called Wikis, but what's a Wiki? I mean, press an edit button, see like some kind of like non-HTML markup. Yes, there were other projects like that. The info was like that. I mean, there's a sense in which Wikipedia today isn't, doesn't really look like Wiki. I mean, Wiki at the time meant things like, every article had to start with a capital letter and have at least one other capital letter, which was not adjacent to it, right? That's what Wiki meant at the time. Now, Wikipedia has since removed that restriction. Some of the other projects were, didn't never start it out with that, but still had the big text box. So in some ways they're more like, in the sense that Wikipedia is a Wiki now, they're more like a Wiki than Wikipedia was in certain ways. But there's variation in that, which I hope shows up. The transaction costs come from something else. That's right. The transaction costs that people cited most often were account creation, like they need to log in. Mechanical costs like confirming an email address, that kind of stuff. Yes? Yeah, I think there's a couple of places, I think probably for timing and bring up some nuances, but I think there's an interesting part within the low attribution that there's a big difference between low attribution and anonymity. And that, I think especially with Wikipedia, because it does allow anonymous contributions, but you can track the difference in how much contribution comes from those who choose to log in and be identified on the site, so that versus the ones who do contribute anonymously. So I think you get to see some, it's a nice way to be able to see what is the benefit of this subtle attribution, that it might not be huge and outstanding, but it has a really big effect. There's also an irony in the case of Wikipedia, which is that by logging in with a pseudonym, your identity is much more hidden than by not logging in, because of course if you don't log in, your IP address, like your internet address, is like plastered on every area that you do, and people can find out exactly who's- So the whole issue of identity on there is more subtle than that low attribution and that within the community. That's right. There's two audiences, so it's low attribution for the general public of readership, so I think part of it is that there's two levels of audience people write for, part of which is the general public, and part of which is the community of other users, and especially anyone who contributes on a talk page is dealing with that community of other users as their audience. Yes. And then, if I'll let someone else else talk about the other point. I don't know, someone who hasn't asked a question? Yes. I'm just curious, I maybe mentioned this tonight, I missed it, but were all of these non-profit projects or are they some for-profit ventures? I'm curious if the financial model of them, they all failed to make profits. What? They all failed to make profits. That's right. Of course. That's right, they all failed to make profits. There's sort of- All of the projects were, Newpedia was a little unclear, they sort of thought they might want to, everything too was started by a company, but they certainly failed to make a profit. HGG2 was also started by a company and then was transferred to the BBC. So I mean, I guess, yeah. State, non-profit. Interpedia was non-profit, the info was non-profit. Yeah. GNE was non-profit, and TDE was non-profit. So there's variation in that, yeah. So a couple of things. First of all, on modularity, at least some of the statement you quoted on low transaction cost, would refer to modularity as opposed to the account setup. So when you tell the story about fixing a comma, that's exactly modularity in terms of the fact that the discrete unit of contribution is low enough that you need very low activation in order to do it. So in that regard- Well, they're connected, yeah. In that regard, when you say, I coded up and the others were more modular, the question is what's the minimal level of contribution necessary to move the ball forward? And at least the comma story is consistent with modularity. The second was the question of generalizability. And particularly, I think your hypothesis four is the one that is most discreet to encyclopedia-like things. But if you were to take something, so two things. First of all, you said you talk about why you think everything true is a failure. But I think relative to any crazy success like Wikipedia, everything is a failure. Does it stop? Does it not continue? 50,000 participants, 100,000 articles, I think you had up there. What does it mean to be a failure? And then how you compare that to things like slash got or daily costs, which are broadly within the same family of peer production clearly don't have the same non-attribution characteristics. And so I'd say that of these, how many of them are directly about what makes foreign encyclopedia versus a successful peer production process? So that's a... Those are great and difficult questions. So in terms of what makes these a failure, I mean like one answer I can say that every single person that I've talked to considers their project a failure. I think everything to team more maybe than anyone else because they were the ones that have the most reason to feel like they could have become Wikipedia. They're over it now, but... So that's one answer. Although interestingly, they're more replaced by the blogosphere than by Wikipedia. And certainly when I was looking at it in 2001, it was clear that what Wikipedia was doing was completely different in the sense that everything to people wrote about whatever they want. Everything to had different parts. I mean everything to had a huge number of people who were producing encyclopedia content. They were just writing encyclopedia articles about things. There was a sub community in everything to. Largely, they hired an editor who really pushed creative writing in a subset. Those two groups were probably, that was the smaller group until we keep until somewhere in late 2001 when basically, and you can see this, the people who were writing encyclopedia content on everything to left to go to Wikipedia. And the everything to founders we'll talk about this. They can point to people that have done it. They introduced me to people who moved. There was a sense in which everything to really identified its like marginal utility in a world with Wikipedia. I believe that everything to is trying to do something that was much bigger and broader than just being encyclopedia very explicitly. But they were also, and many participants in everything to, were definitely trying to build an encyclopedia. Do you mind if I ask you a quick follow up now? And they take the content with them from everything to talk to you? That's a great question. I assume people, some people must have, but I don't know the answer. That's a great question. And then, yes. I very much like your hypothesis, especially the final one, but the research question that you have, when I compare Wiktionary, which is comparatively Baylor's project compared to Wikipedia, I think that the main factor is the annoyance that you just mentioned. When you see the missing comma, you feel annoyed, right? So you want to act. And there's even more conflict deriving from the data in Wikipedia. So you see that somebody's basically wrong. So you have to act immediately. And there is no conflict on Wiktionary. So I'm wondering to what extent this conflict and also conflict resolution solutions present on Wikipedia, like discussion pages and other forms of dispute resolution would be actually distinguishing? So there was spaces for that kind of social interaction in different ways, in the info, in intrepidia, in everything to, in the issue due to, issue due to a very sophisticated system for that. The info, for example, is just comments. Newpedia, there were offline channels, but it wasn't the same place. But there was definitely, especially that, G&E was, and maybe TDP didn't have that. Then one failure of Wiktionary? Yeah, I'm not going to speculate exactly as to why Wiktionary fell, although I'm interested in that topic. I think we're now over time, I'm being pushed to, so we can maybe talk afterwards, sorry. I just want to make one very quick comment, which comes back to the sort of overall significance of the work, because I was thinking like, if you put their Facebook or you put their YouTube, could one do a similar study of saying, was it because the product of Facebook is really friendship? And they kept that very much at the core, making the process innovative. And in the same way, YouTube does something, which is the product, we all produce videos and we like to watch other people's videos, but the innovative process was there, so it's like a genre of studying. I don't even use Facebook, so I'm not going to speculate as to what the product of Facebook is. I have a conception, but I'm probably off, so everyone here knows more about that than I do, I think. So, cool. Well, thank you everybody for coming. Thank you. And my email is on every slide and I encourage everyone to tell me. I'd love to continue this conversation. I'm writing this up.