 Karim and I are going to give this together. And we have a question for you. So we have like an hour and a quarter, I think. And we can either do, each of us, the 15-minute versions of what we do. And then we can have the rest discussion. Or we can do the 25-minute versions in which we have a 25-minute discussion. So which would you rather, I mean, I don't know whether you already know all this stuff and you just want to chat or whether you'd like sort of the planning. So how many of you would like 25, 25, 25? How many of you would like 15 from each of us and then what would it be, 45 collectively? Option to revise and extend. If it's going really well, we can just keep going. I suppose. So OK, so let me just launch very quickly then. You will see me skipping over slides. OK, so the first slide, there it is. Second slide, I want to just say that a lot of people are doing research in this. And so Karim and I are reporting stuff that's really fun and exciting. And other people are to be credited as well. So the underlying thought was where do innovations come from? They really historically have been argued to have come from the manufacturers. That basically, manufacturers, since Schumpeter look at the center of the market and they say, whoa, look at all those users. Manufacturers can aggregate their demand and then they can innovate. And the result will be that they can do a better job than can any individual user innovating for his or her self. The missing piece was that in fact, at the leading edge, the market is by definition small. And so in fact, what you get when you track back to the beginnings of things, what you get is this kind of pattern. So this is a guy who developed the heart-lung machine. And his name is John Haitian Given. And a patient died on his operating table that he thought he could have saved if he'd been able to stop the heart momentarily. He went to manufacturers and said, hey, we need a heart-lung machine. And they said, yeah, have you developed the operations to go with it? Well, no, but I know I can. Well, do you think anybody else will want it? Yes. Do you know that? No. So immediately, market manufacturers say, well, if the need is small and uncertain, we want nothing to do with it. So this poor guy, in effect, had to spend 20 years developing this thing for himself. So this is what happens. In fact, what we then did was we studied it in a range of areas to actually quantify the number of innovations users had developed versus manufacturers. This particular one is in the area of scientific instruments. We did it in a lot of other areas as well. Took the sample of the most important innovations, tracked back the innovations on each of these to find out where they really came from. And ended up discovering that, in fact, most of the important innovations, and I'll qualify that in a minute as the functionally novel important innovations, were actually developed by users, not by manufacturers. OK. So the other curious thing we discovered is that manufacturers did not know this. So the fascinating situation was that it took a lag of like five to seven years between what the user would develop and the first manufacturer. Now, part of the reason for this is that what users do is they make things for themselves. And these things that they make for themselves are clues. You just make them so they work. And so a manufacturer comes along and he looks at that. This is the first automated radio immune assay system developed by doctors who had to do 20,000 patients for a study. And manual methods just wouldn't hack it. You couldn't even get your students to do it 20,000 times. Anyway, so the manufacturer looks at that and says, hell, that's not a product. There's no room for our logo. There's no operator's manual. What kind of a piece of junk is this? And they come up with this, which is the same thing. But in fact, obviously, it's much glittier and better and more reliable and stable. So the manufacturers do stuff, but the point is that inside this is this. So now, this is true wherever you look. Agricultural innovation is done by users. This is center pivot irrigation. And what you see here is that's the first one. Put together from what the farmer had around. What you see here is this is old irrigation pipe. Wheels from tractors, and so on and so forth. And he built it, and it worked. And it's a major agricultural innovation. Here's the glitzy manufacturer version. And what you see is much cleaner, much neater. But the key is that people didn't understand where the innovations came from. For example, I called up these people, and I said, so did you get input from users living or dead? And I said, certainly not. Be sure to send us a plaque with their name spelled right. So the key issue here then is not that people take credit where they shouldn't, because in fact, that's one of the things that makes life very pleasant, isn't it? The key issue is that if you don't understand where the innovations come from, you don't know how to replicate the pattern. This is also true in consumer goods. Everything from all sports equipment, and so on, done by users. Now the key here is that we're not talking about individual users versus companies. We are talking about the relationship to the innovation of a firm or an individual. So Boeing, when it develops a machine tool to process an airplane, that's a user innovation because they expect the benefit from using it. Boeing, when they make a new airplane, that is in fact a manufacturer innovation because they expect the benefit from selling it. So that's the essential, and so it could be an individual, but it also could be a manufacturer. Now the next thing that we find is that it is not a rare phenomenon. In fact, when you go out there and you look at user innovation, it is all over the place. And I'll show you in a moment that in fact, it looks like this dark matter of innovation that is not measured in government statistics yet is vastly outweighing the measured stuff that you get in government statistics. It's really cool. So anyway, lots of user innovation in lots of places. These are people who care most about their innovation. So what you see here, for instance, surgical equipment. It's not all surgeons. 22% of surgeons connected with university clinics innovate. They're the ones pushing the leading edge. Same mountain biking equipment. It is 20% of the people who are in the mountain biking region. If I go to Harvard's bike racks, I won't find 20% modified. Now, why do they do it? Because they have high needs. What happens now, and this is about the internet, is that the internet is enabling individual user innovators to join into collaboratives. Now, this becomes a very powerful competitor to manufacture-based design. So now, typically what we talk about in these discussions of the internet is on is software and things like Wikipedia. But what I want to show you is that it also applies to hardware. So this particular example is about innovation in a user innovation community in sports. Specifically, it is kite surfing. That's the board that you slide on, you can see here. And then that's the giant kite. And it operates like a reverse parachute. You can go up instead of down. And you need lots of controls here because you really don't want to come down more abruptly than you hoped. So it's actually quite complicated. There are five control wires and so on. All developed by users. It became a fad. Manufacturers came in. They said, this is great. They adopted the traditional strategy of saying, well, now, what we do is we will innovate. And we will develop our version of this and so on and so forth. So each hired one and a half engineers wasn't that big a market, and they began to compete. Then what happened was that a guy from MIT named Saul Griffith said, well, you know, I can really post here and create a website where users can get together. Here's an example of a post. And they can exchange information. What a cool idea. So now what happened? Well, this particular posting is like a super high aspect ratio inflatable design for mountain boarding. As you know, these guys now go up mountains with their boards, as well as just along the water. And I do none of that stuff. And then what happened? What happened was that immediately a community formed and people began to bring quite sophisticated tools to that community and develop them. So what happened was, enabled by the internet, you ended up with not just one and a half engineers with a master's degree in each company, you ended up with aerodynamicists from NASA and Boeing and places like this who were also hacking on the weekends or whatever. And they brought along, slow speed aerodynamics is a regime where you get turbulence and so on and so forth. And so they ended up with modeling tools. They ended up with design tools. They started sharing their innovations on the web. So first by sketches like this, but then by CAD. So that what you got was you got the ability to make a kite, encode the information, distribute it across the web, people in other countries all over the place, but download these designs, cut them out on laser cutters that are used in sail lofts, and reproduce across the world. And the result was that you ended up then basically vastly in this collective activity outdoing the manufacturers. And so manufacturers began to adopt the kites from the users. So what you got is this interesting situation where we think manufacturers, this is Carlos Baldwin and I now, think that manufacturers are going to be driven out of product design in most areas by users because of the power of the internet. Specifically what's going on here and the two advantages that the situation enables is number one, what you've got here is this is a design space, let's say for all possible kites. What you've got is more users, all those X's there are experiments and activities, you've got more users than you have manufacturers. The result is they can explore more of the space. Critically, because they are users, they will share their innovations. Critically, I'm not saying as a manufacturer I might, but as a user I don't, I say I invented a new cup holder and you can't use it. Whereas the manufacturers are doing that. The manufacturers are saying, look, I've got an area here and I'm protecting it and you can't come in. So not only can they cover less of the area, but they wall off pieces. The result you all experience when you buy a car, you want the engine from this car and you want the back door from that car, but you can't get it. The power of the fact that it is user-invading, it the power of the fact that it is user innovation that is open and that is widely distributed on the web is that the best solution can be adopted by anybody. And that is why, in fact, it's starting to drive things out. Now there's a lot more to it, like modularity and the rest. But the key thing now is that we have began to say, okay, let's study this in more detail, and then Karim is doing the collaborative innovation and how it works, and he's going to talk about that. Another thing we're doing is we're showing that, in fact, when you really do government statistics differently, it's so cool. It turns out that there's a hell of a lot more user innovation in those manufacturers. Now this comes from Fred Gould in Canada, and it's, again, so cool. What they used to do, governments, because of the focus on IP and on manufacturing innovation, governments were asking manufacturers, well, did you innovate and did you off? And said, well, did you get any useful information from anybody? You're all customers' help. So what did customers do? They never asked that. It could be that the customers actually designed the thing, or it could be that the customers said, right on, do more. So what this thing did, and now what we're doing again in 2007, what this thing did is so cool, what it did was it said, I'm going to ask the users, in this case of advanced process equipment, how they got their equipment. And the answer turned out to be that 46% of them purchased, 26% bought and then modified, and 28% built their own. Now, if there's more than four customers per manufacturer, this means that, and of course there's maybe 400, so this means that the amount of innovation distributed around is actually much larger than the innovation the government's counting. And now, another stuff we recently did was finding that, in fact, this innovation is not only in the manufacturers, but they're not counting it typically as R&D, they're sort of doing it as Slack or something. So we're going to make all that visible now. So what's the implication of this? In the old days, when we thought that basically manufacturers were the innovators who needed strong IP and strong brands and so on economies of scale and the rest of it. And now, with user innovation collaboratives, you need protection for intellectual commons, low networking, collaboration costs, and so on. And so that is all I think I want to say. And then I'm going to just tell you one more thing, which is so cool. So yes, the thing we're doing here is we're looking at the power that manufacturers still have, and one of the powers they still have is brands. Well, you could make your own candy bar, but a butter finger bar by Nestle is far better. And so what we're doing, the guy named John Fuller and I is so cool, is looking to see that user communities also develop brands. But they don't do it in a costly way. They're not there being forced by a manufacturer to think you're part of the Pepsi generation. That costs money to convince you you're part of the Pepsi generation. They're doing an activity on their own. And the activity on their own, they are now making badges for themselves that they put right over the manufacturers logo on their backpacks and stuff like this. So it's very, very cool. So, Karim. So I'm going to take from basically this slide that Eric has on sort of what we see is that innovation is becoming more and more collaborative and discuss sort of the patterns of these collaborations. And in fact, what we see is that it's not individual innovators at all, but sort of collectors working together. And I'm going to sort of walk you through some examples, both in open source and in a strange programming contest where we get to see how these collaborations work and how these collaborations actually result in better performing products than any one individual might have done. So I'm going to just walk you through sort of a day in life of a software feature. This comes from a project called PostgreSQL. They make a database that competes well against Oracle and Microsoft and so on. And we're just going to track one feature and its evolution. And you'll sort of see these different people and individuals come in, do work, go away, come in and do work, and get a sense of sort of this collaborative mess. So this is Mario Balguni from Germany. And he posted on the email list a question about database vacuuming. It's a big thing about how to clean up databases as they're working away. And the way it sort of starts off is I can sort of post on the list. The list has about 900 members signed on to it. This email goes to everybody and says, you know, I know you're all working on the beta of the release, but I'm having some problems with vacuuming. I wish we could fix this. And it sort of gives a solution as well. Perhaps we can find a way to come up with a better vacuuming algorithm within our own database. He does that. And then basically, in the space of the same day as he posted that question, different people come back and provide some solutions to this thing. So Shadar from INDEK comes in and sort of says, you know, I think we're going to have a different mechanism to solve this problem. Mario comes back to Shadar saying, oh, this is never going to work. Rod Tietrum-Cannan comes back and says, actually, Mario, you're mistaken. And here's how Shadar's proposal might work. And he actually gave some code that would show how this might work. Tom Lane comes in. Tom Lane is a core developer here in this community. And he acknowledges the problems that's an important issue for us. Give it a name because, well, we should have auto vacuuming come through. And he came up with a solution that was going to be a server based implementation. He says, in order for us to fix this properly, we've got to go in the guts of the software, rip it all out, and redo it all over again. And Shadar comes back and sort of says, you know what? I'm going to come in and try to solve this problem. Now none of these people have interacted before on this email list before. This was the first time where they showed up together to sort of solve the problem themselves. So here's what Shadar wrote. He goes, sounds like a nice idea. I'm giving you about a week to do this. And I'll come back soon with that. And then seven hours later, he sort of lists out his plan. He goes, I'll kind of work on this. Here's my brief idea. And then he sort of asks a bunch of questions. It's my first attempt to contribute to Postgres. Please let me know your comments. And then basically what happens is that I sensed with our poster that, no, Tom Lane came back and sort of said, you know what? Your approach is never going to work. This is going to be doomed forever. So if you imagine you're in a setting where the head of the group says, you're doomed, what are you doing? It might, you might not keep continuing. Rock Taylor, again, was sort of listening into the conversation. Remember this is going to 900 people that are into sort of observing this discussion. Rock Taylor came back and sort of said, you know, Tom, your ideas are okay, but I've got a different perspective. Tom agrees with that, sort of see that coming through. Barry Lind from Palo Alto comes in and comes up with a different solution. Mario comes back into the conversation and proposes yet another solution. So same problem. Now we have seven, four different solution approaches being proposed. Go this way, go that way, approach it this way. Tom Lane comes in, again, sort of provides some more feedback to the discussion. And then Matthew O'Connor from Philadelphia comes in and also proposes an approach. So what you see again, a problem shows up and then within the space of a day, this is all in the day, all of these individuals start hammering what we added from different perspectives. And you wouldn't see this in any kind of a normal innovation setting, right? Where I have a problem, I sort of sit there and go think about it, maybe talk to a couple of my colleagues, they might give me some feedback, but this type of a distributed feedback mechanism, you wouldn't see. And then two weeks later, three weeks later, Siddharth comes back and says that he solved the problem, right? And here's what he said, right? I've written a small demon that can automatically back to PostgreSQL database. It sits on top of the PostgreStats collector. He sort of tells you what it does. He puts it on the net. Let me know for bugs and improvements. I'm sure there's people that are running some scripts. This is a attempt to provide a generic interface. What he did, though, is he actually abandoned his proposal and took elements from Rob, right? And Mario, okay? So his initial approach, which was sort of lambasted by Tom, was he abandoned that. And, but he also didn't listen to Tom and what Tom was proposing either. He took other approaches that he could do. And his approach was basically one of writing a small demon that was actually external to the system, right? It wasn't actually part of what Tom recommended, which was to go into the guts of the system and fix it up. And then various people ask him questions about it. And then the list goes quite on this issue. But then two months later, Matthew O'Connor comes in and says, I've taken Siddharth's code and improved on it. And the thing that he did, which was interesting, was that, here's what he said. So most of the time I tried to implement a special process back in as an auto vacuum demon. I failed it in my lack of experience. On September 23rd, Siddharth came in and wrote this in C++ and did the job, right? But everything that Postgres is based on was on C, right? So he goes, I've taken the code in C++ and now we've written this in C now instead because Siddharth didn't care about rewriting it, right? This is all the problem and we've done it. And what he also did, which was remarkable, and this is how these communities work, is that he actually gave evidence of the performance as well in the email. He said, look, so on the X axis is the number of transactions per second and on the Y axis is the transaction time. So you want to be at the lower side, right? As transaction times increase, as transaction numbers increase, you still want lower transaction times. And what he showed was that in green, that around the 700 transactions a number of times, the database collapses and the transaction times just increase significantly without his code. With his code, it sort of stays stable throughout, right? So he was a stranger in the community, hasn't contributed anything at all, taken somebody else's code, modified it, fixed it up and then provides evidence to everybody else saying, here's why you should look at my code and here's why my code works. And then Siddharth sort of, you know, he says, hey, thanks for your work. You know, thank you for doing it. I don't have much interest in trying to make this into a C program. And then basically, when we step back and analyze this one small feature, this feature was voted the most important feature on in 7.04 release by users and also what was interesting is that this was on Nobody's Roadmap, right? This sort of emerged on the ether from this community working away. What we see is that altogether, there were 23 people who contributed to developing this feature. Four of them were core developers, but they actually didn't do any coding. They maybe gave some suggestions and most of the suggestions were ignored, right? And 19 AHA, people that were sort of in the community coming in and out and basically there were in the end three people from the periphery of the community that actually wrote the code with the testing and approved upon it. And so this is a dynamic we see now on the internet where users can collaborate with each other with some advice from the core group, but can be almost pretty much independent of the core group and come up with features that make a lot of sense. And what we see, so let me just show you a version of this where we can actually track performance. So this is data from MathWorks. MathWorks is a very busy programming contest which is partial Wikipedia, partial open source, partial American Idol. And the way it works is that imagine if you're on Wikipedia and you made a change to a Wikipedia page. And on that change, it would tell you, congratulations, you are now the person with the best change ever on this entry. This is the best performing change we've ever seen. And this is how the MATLAB contest works. And the way this works is that there are some rules on participation. There's typically a problem given which is sort of an NP-complete problem that you have to solve. There's some standings. You write code to solve this problem. You submit your code and you get scored. You get your total number of code is. But the Wikipedia aspect is that you can look at these standings and from there examine the code that's been provided. And then you can change it. You can add your own things or you can modify some things and then resubmit it for evaluation and then you can rise to the top. So they run this contest for a week every six months. And here's one, Nathan, who was a well-established developer right in a different company. He takes a week off participating in this contest. And he goes, this is my first MATLAB contest. It's gonna be far too much enjoyment. It's one of the most addictive and compulsive things I've tried. Also I have experienced physical trembling while making the final preparation to submit code. Is this normal? So here's what we see. The contest consists of three phases. There's a darkness phase where you just work on your own. You submit code and you see if your performance has improved. The lower the score, the better in this setting. It's a twilight phase where you can send your code in, you get scored and you can see how the people are scoring. And then a daylight phase where you can see everybody else's code, you can take their code and remodify it as well. And in this contest basically there were 111 participants and they submitted 4,000 comments. And what we see is that in a space of a week we have basically between 10 acts and 100 acts of improvement of performance. The red line measures the leaders at any one point in time. So who had the best performing code at any given point in time? And this is sort of this dramatic improvement late at the end by Yikau. Yikau is an engineer working in the UK. But as we'll show you, all this code is a mass collaborative effort. Yikau gets the credit at the end but he actually borrowed code from 32 other people to get here, okay? But now what we can really see is sort of like what happens when you truly open up and when people can borrow and share easily and rapidly. So here's what we see basically on average the leader, those people on the red lines are borrowing code from 19 other people, okay? So it's not just de novo innovations, it is really cumulative and collective innovation coming together, right? And what we also analyze is that 89% of the leader code is borrowed as well, okay? So you need this huge amounts of transparency and sharing for us to get here. And what we see is that there's a rapid iteration between new code and reuse of code. So what this maps out are comparing each leader to the previous leader and examining how much code is deleted, how much code is conserved and how much code is novel. So the overlap basically shows you the conserved code, right? This is the deleted code as compared to here. This is the new code. And what you see here are that there are these punctuations where somebody comes in with a brand new stuff. It makes a lot of difference. But then other people show up and make small incremental improvements and you sort of go up the frontier of performance, right? And there's a trade-off between how much new stuff you bring in and how much new stuff is reused along the way. And what we see again is that once you have code that is performing well, it gets massively reused. So this is just showing you one line of code and it's evolution. So Jim McCullough had this one line, he had actually a bunch of lines of code. And the best Jim ever got to you was a hundred place with this line of code somewhere in the context, right? Then somebody else took that, recognized that code as being good, right? Made it part of somebody else's code base, shows up here as it comes up at the top. And as soon as this code ends up at the top, this is explosion and reuse by other people, right? But this recognition from here to here had to be done by somebody else who could recognize it, make sense of it, make it part of something better and then from there improve it. So let me show you a little video which will outline the leaders, okay? And this is a snapshot of every single leader. The colors inside the box represent different authors, okay? And then the red lines are gonna show you the novel code coming through. So we'll basically go through the whole contest in about a space of a minute, right? And what you see is that as these colors are changing, you can see new authors being added to the leading entry, okay? And the red represents novel code being inserted. Right, so there's this big change where all this blue came in, right? And then you'll sort of see it over time, shrink out a new stuff being added as well. But each color here represents different authors participating, okay? And so this is again our ability to take elements from other people, recombine, remix and put it out there and then other people finding better ways to use your code than you might have yourself along the way. And so with that, we'll stop here and then open it up for discussion. Very cool. I have found that there are some things that volunteers can do and that it's fun for them to do and other things that you really need professionals because either volunteers don't find a fund or they don't want to do it. I work for non-non-profits and that's true. I also work with computer documentation for people to get online. And I also noticed that people like to muddle through. They say, well, we'll get through it. No, we'll work a little harder and it's hard to persuade them that we have to pay for the professional and maybe that's okay, they can train each other but sometimes it'd be good if they just break down and say we'll hire somebody. So, others have comments on that or certainly we do. Yeah, please. I'm just gonna say it was interesting that you said professionals that it doesn't seem like some of the way you were mentioning presentation is that you allow professionals who contribute on a volunteer basis. That's right. That's right. So distributed innovation, if you broadcast widely, you often attract really the top folk. So we have to think about why in your case this is not so, I mean, we can do that. So, sorry if I can add. I mean, so a lot of the Linux code is driver code, right? Driver code is pretty darn boring, right? Not exciting, not the leading edge of computer science but that gets rid of my volunteers too. So I think there is something about what we might define as fun to one group might be born down in the group but that boring stuff may actually be fun to do in the group or necessary and they might do it. They want the device to work. Yeah. See, that's critical, yeah, because if you are a user, you need it for yourself. It's not just entertainment, yeah. I just want to add that in professional groups, you very often complain about structures that will event various forms of interaction which could be, essentially, beneficial. And in some of these other groups which are volunteer groups, you find the same things. In other words, you find patterns of interaction going on which restrict what they could possibly do. So they're getting secondary dean or even social leadership which prevents other forms of interaction. It should be better. First question. Just, yeah. I'm intrigued in some of your earliest assumptions and how we could sort of industrialize them or put them into practice. If we speak of user driven innovation and the fact that the user needs to do things they see around them, like you were showing us sort of a two-stage innovation process where somebody came up with something because it was solving a problem that a very open source response user thinks you've got around you, solve it. And then somebody else took that thing and made it relevant and scale it up. But that was the intent in some of the physicals part of that suit of. I have. So here we have final practice that we can take that and the world of software and systematize that. But if we get in such a form that we get some intellectual things to work with, then we've actually taken out the two phase and made it one phase. Do you have research that would help us understand that? Do you accept the condition? No, see that absolutely, this is about toolkills. And you've got the 15 minute version because you wanted it, man. It'll induce you. And so in fact, yes, what happens now when users innovate and you see them innovating, one thing you can do is like that farmer, you as a manufacturer doing hunting and gathering. He innovated on his own, you provided him with no tools in advance and he did what he did. Now in many fields, one of the things that you can do is provide a toolkit that makes easier for users to innovate. Often these toolkills have constraints so you're innovating within a box of some sort. So it has pluses and minuses, but absolutely. The fascinating thing here, and this is all happening in our watch, is that historically, find a need and fill it has been there as a strategy. And 10,000 researchers have done work on this. And now we're sort of filling in the whole space as to how on earth distributed collaborative innovation works like Karim is showing you how it works. You know what the economics of it are. And the thing that's so exciting is that it appears economically across all of them and I keep on writing these titles to papers like, manufacturers on the dust heap of history. He says, no, no, no. He says, well, manufacturers in the ash heap of history. You're missing the point. But anyway. He's at the business school too. That's right, last. But anyway, so that's, yeah. So it's a very exciting concept. Yes, that's part of it. I think to your point, I think the toolkits, we're seeing that so much on the web now with these bridges and the actors. And I think, if you can go beyond just a one way relationship between the toolkits of buyer and the consumer, but in fact, open it up laterally, other people can participate as well in any kind of a tool creation. And I think we'll get this kind of effect that we sort of see in game, like metal today. And it's physical stuff as well as software because nowadays, you design everything sort of as an information product first. So it's just really amazing. I was curious if you had any comments. I looked a little bit at the Living Foundation report about the contributors to the kernel and the dramatic number of commercially-employed programmers who are contributing to Linux. I think that really is a migration away from volunteer effort to largely commercial. But the implications for the product, what happens when you have a split world? Yeah, I think that's actually good news in the sense that somehow these developers are getting paid now to participate and that their hobbies are becoming full-time paying jobs. I think two things are going on. One is that- Are you finding out the same developers? In many cases, at least historically, these guys were volunteers and they got employed by these companies. I think, so 10 years ago, people would say, well, Linux doesn't scale up, it only goes to four processors or eight processors. It doesn't work on Supercom- And which hacker had a supercomputer and a basement that they could get access to? So I think as you think about the types of applications that Linux is now going into, there are certain cases where you do need these heavy physical assets and heavy physical things to work with and then the companies are providing, are seeing in their best benefit to have these individuals work on these projects. But it doesn't preclude the volunteer participation. In fact, I think it actually helps it along the way because now they can actually have a conversation across a whole spectrum of use cases, not just the basement hackers, but the very professional developers as well. So I think in the end, it's beneficial. But also, I think if somebody's, I don't know if anybody has an analysis to do, how many of these people are working for user companies versus software publishers? Yeah, that's the point. Korea and I always finish each other's sentences. So the thing is that you remember that users are also companies. So if Microsoft is making server software and there's also Apache server software out there, it's not so that has to be individuals in their basements. Amazon and all these people hire one or two engineers each. It's just that the innovation effort, although paid, has moved up into the user room. And I think most of the analysis in the academic world, the game-threat analysis has been sort of a community publisher and a commercial publisher of software. And we sort of try to see when does a community fight against a company and what are the dynamics? But what's missing in that analysis, and we hope to fill this out this summer, is that in fact, there's a third party, which is a user company, participating on Google or Amazon or on Orbitz. They are not in the business to sell software. They're selling you something else. They're selling you services. But their participation and their release of some of this code back in the community completely changes the dynamics. I was interested in the math cream of your example. How many people it needs to be in a community to how many people participating to how many are actually innovating? Yeah. So I think, so in my analysis of the Postgres, what we see is that there is still a core group in 10, 12 people that are doing what looks like most of the work. But when you analyze the work and you sort of see what are they doing, what you see is that they're actually doing work that's what we call sort of dimension of merit improvements, making software harder, faster, better, right? But then the most of the functioning novel things, things that are adding new functionality in software, those come from these peripheral users. And they keep coming in and out. So in a community like Postgres, there were 12 core members, and then the rest of the 900 folks were all peripheral users. They just do one little thing, a small thing, then move on. It doesn't change much, but then software being run, it doesn't change much about the type of class. Yeah, I haven't done that kind of analysis yet. I think the pattern is fairly robust, and I've shown it to other community leaders and other projects. But we haven't done the, this took, this was sort of my saving over dissertation. I just sort of read 15,000 emails, manually matched, source code commits to people too. So until we have a more automated mechanism, this will be the problem. But you as a user will develop. All my graduate students might develop. Yeah, there you go, there you go, there you go. Found your work really fascinating, and I was just curious about two things. First of all, do you have, can you name, or give us an example for a company that sort of tried to take advantage in a positive way of this new innovative reality going on, that you know, people with bad practices allowed for other companies going forward. And I was wondering also, what patterns do you see across nationalities? For example, I know Israel has a large, large people who have code development. I was wondering if, well just in general, what do you see in the media? So, yeah, I mean, what you're doing is you're saying, users are the innovators in the design space, okay, but there's still other things that have to be done. Complimentary products and so on. So one of the things that manufacturers are doing is saying, how can I link my users by, for example, a toolkit? So if you look at this, for instance, in democratizing innovation, you'll see a firm called Stata. And what Stata does is a statistical software company. What Stata does is says, fine, we know we do not invent the statistical tests. Users invent them. We will watch the user websites because they have community websites in this thing. And we will pick up the best innovations. Now, Stata has however, a proprietary core of code that they keep proprietary and that's how that works. The same thing is going on with Lego. Lego is looking around and saying, my God, we have 50 designers in-house and there are 20,000 users outside design code, in this case, models. Why don't we publish their models? And so what they set up is something called Lego Factory where you can actually design your own box. You can post this thing for sale on the Lego site. And again, what it's doing is saying there's a distinction between distribution, manufacture and so on. We can do those things while intelligently yielding innovation to the user. And citing news, Denmark has now made this sort of national policy. They're trying to push this. They think they cannot win, they feel, on the side of R&D push. Countries like the U.S. will win then. So what they're saying is maybe we can be very friendly to the sort of internet communities, users and so on, and produce their stuff ahead of others. So it's, and the UK is coming along on that, too. I'm from Microsoft. You're going to need to apologize. It's such a mess. It's okay. Heel, heel. Well, I can actually point to an example of what's being used at Microsoft. The X and A creators of 12.4 people are allowed to create games for the Xbox and on the vote of the community, they actually have the opportunity to distribute their games that they produce very high-end markets for the Xbox. So it's being successfully used there. We tend to believe, and I'd like to check this with you, that collective production of content is possible, and users don't seem to mind that you're going to profit from it, as long as they also have a stake in the outcome. As long as they know that they're going to benefit from what's being produced and they have a share in it, they're willing to participate. Yeah, I think, so if you go back to Eric's example, it's data, it's data that makes the statistical software package, right? If you look through the manual, right? So who is producing all these statistics like Kareem's Kappa or Eric's Eda would be done by an academic, right? And we care about citations and recognition, right? As creators of these things. And so they're very smart as you go through the manual, right? When you get to K, right, Kappa by Kareem, right? They'll say that, yeah, this was done by Kareem McConney, you know, and the reference, even the paper that this person came out of, right? So close to about 40% of the manual has these citations that sort of come from it. So users do, well, there are many heterogeneous motivations to participate. Some of it can be, I want a piece of the action. And depending on the institution you set up, you might have to give a piece of the action. So like a company like Threadless, which does t-shirts, right? They give you money for it, right? If you have a winning design, they give you money for it. In other settings, it's like this recognition is good enough, right? And so you have to sort of, I don't think it's free labor in the sense of like, you know, we get all this great free stuff. It's actually a lot of work to make this happen. But then it's the institution that you set up that other people agree to participate with that comes to it. Right. These are early days as to how this is going to work. You know, companies, most, somebody asked how often this is, or whether they're successful models. We're citing rare successful models. Most companies are still in the deer in the headlight stage of what, you know? So, John. So one of the questions that I want to throw out for these, just what are the essential preconditions to these things in terms of, because if you look at it like Postgres, you've got source forage, you've got the internet, and you've got places for the code to go to standard places to discuss them. And the pharma industry is struggling to figure out how to use some of these. But there are all sorts of fundamental infrastructures that don't exist yet that seem to need to exist to enable some of these explosive user recommendations to come out. And so has there been any research on what are the five infrastructure elements you need to have to deal with it? Yeah. You know the answer to that. Right. Well, I wonder what you guys are doing with research management, maybe. Oh, but why don't you tell us your answer, then we'll tell you who you are. Wait a minute, you're the one who's like, oh, okay, so modularity, you know, heterogeneity of knowledge that people have, and an opportunity to get a benefit from each little increment. So for example, in one of the issues with respect to our pharma is you have to do such a huge thing before you can benefit in some way from it, that it hasn't been modularized given the piece of fact. So, but it looks like when you do get modularity, when you get low communication, which everybody now has, when you have good tools, then in fact, and benefit from an increment, then in fact it appears to be everywhere. So we did a look in the SIC code with Ethan Malik just to see how many user communities there were and just everywhere, you know, I mean, it was very hard to find something, I mean, you know, where there wasn't, you know, termite washing, I mean, you know, ear, weight training, you know, I mean, we made it, yeah. But also, I mean, I think the other component, we told the tools is sort of, can you share and assess knowledge and what's the basis of sharing and assessing knowledge? So, in software, you have the compiler and the ability to pick anybody else's code, but only your own machine understand it and figure it out. In pharmaceuticals where you have such wide disciplinary basis and we don't know what this might do in the human body later on, that those things, until we come up with better simulation models, until we can come up with different communities in pharma that can sort of say, okay, I understand what the physiology guy is saying and the mouse guy can actually create this in the mouse, until that happens, I mean, I think we'll sort of see attempts at it, but not the same kind of a rapid speed that we see in code. Now you want to give your answer now that we've done all right. Yeah, all right. All right, yeah. Okay, so please, yeah. You have the statistics as to the profile of the collaborators, for instance, in the leadership scandal or any other office or some kind of, because it seems to me that you're saying that you have 12 collaborators that are full-time non-payment and until that you have nine guys that are being paid counter to the product. The question is, if you analyze the profile of these guys and how it has been developing through time, perhaps you can discern that, wow, it is changing so fast that the original mode is changed because perhaps, again, based on your argument, I can't assert it doesn't seem to me that collaborative right now, perhaps it's still collaborative but the nature of the collaboration there has changed. So do you have any statistics, actually? So I'm raising moving. Yeah, I'm raising different questions. One is, so even the 12 four guys are actually being paid for this, and the Tom Lane works for Red Hat, he's just being seconded to work on post-drafts, right? Bruce Longton used to work for this company called SRA, which is a Japanese software generator, now he's working for company called NRIDB, which is getting inventions out of post-drafts. So even the four guys have either the publishing jobs or user-focused jobs that are, one of the guys who works for Philius, they're a big internet registry, they use post-drafts quite a bit, and they buy it at the core for them to be participating in. Right? So it belongs to the model of sort of the hard, sort of the hardest entertainer kind of model where you can sort of say, we know you're doing good work, it's an event that our company, one way or the other, and we're gonna let you do, we're gonna rely on independence for you to work on this, on these things. So in fact, I don't think it's sort of this, the volunteer aspect of it is from the point of the project, right? But the project isn't paying anybody to participate. Right? But the motivations can be pecuniary, I've been told by the boss to participate, I'm paying my to participate over this, I need it for my code, or because I'm enjoying what I'm doing, I'm never learning. And I think that is what is the core of what I've been studying the motivation that sort of stayed around. I think the numbers are probably more increasing towards firms, bonds from people to participate, and that will probably change, I think the nature of how the output will be produced in the end, but it doesn't prevent the volunteers who are doing it for whatever reason to also engage with the organization. But your opinion doesn't, economic motivation change any aspect of the collaborative environment? I'm tested for that, I don't know, I can't give an answer, right? I don't know, can you think about what the change might be, so what are you doing? I know, it seems to me that when this process started off, you had a fully commercial model and a fully open-source model where everyone was here, home, child development, so it wasn't being asked for anything, it was being asked for something. I don't know, it seems to me that there's something that was true even in the beginning, so maybe Linus Perlman would have been paid for, certainly the folks who were writing the code were working in institutions, but the code was being used, and again it was not a publisher model yet, but there was some degree of connection between the user of the code and Linus, right, and the guys who were contributing. So I think there was this view that although probably the Spare Time volunteers were going to like, the project, yes Spare Time volunteers, but at the point of the person, they had real needs for that real code, and they were being subsidized by their institutions, but the economics of the dependent part of being free and revealed, they don't depend on how they pay. Yeah, okay. Well, maybe it's got a little related, I'm curious if you, I thought much about management ownership structures, I mean, we've got, I don't know, Linus Foundation, we've got Sun, I don't know, Java, sort of commercial, we've got about the volunteers in just two projects. Is there, are there other models that I haven't thought about, and are there trade-offs and how it's kind of owned? Come on, come on, but I mean, you're even on your own, I mean, you know, the world is so full, this is such a new area, the world is so full of low-hanged groups, you know. I mean, all of us can just have such a ball here, you know, I mean, it's just wonderful. So one of the things that we're looking at now in our current paper is, well, you know, maybe you don't need the core at all. In other words, as we look at the kinds of innovations being done, you know, we mentioned that there are 700 or 900 people in a periphery participating and often not paying attention to the core. Well, if you substitute a toolkit for the core, maybe it truly is totally democratic, you know. So this is another model to add to your piece, you know. It's like, well, this is, you know, it's like science and something like that, where it's totally open and nobody's running with it, but there's sort of collaborative building, it's like a coral reef. So the coral reef theory of innovation. So what you can imagine, I mean, so what you think about Mozilla and Firefox, I mean, there's an explosion of extensions in Firefox, right? And those are almost completely independent of it. As long as the API is published and people don't have the right to extensions, people can go out and come on. And they don't need to interact with the Firefox for developers, right? And so if the more we can create this extension, that every product will have an extension layer, that we can keep shrinking the clock over and over and over and over and over, then we'll be able to process that. And institutionally, we are seeing sort of these non-profit foundations. So Mozilla Foundation is kind of an interesting world with Mozilla Corporation, you know, to do, because, you know, there is this concern around API mismatch, right? So, at least in the world of software, the biggest consumers of software can take to the big institutions, governments and corporations, and they want to write, they want the CIO wants to write and check somebody, right? He wants to build his empire and have it big and big righted. And open source models don't help them with that, right? I mean, the API might be able to change the tech market. And most CIOs want to blame somebody, want this view that, oh, I can blame somebody for the problem. And I'm gonna talk to a sales guy to take me to a nice two marketing lunch and do all these things. And what are my employees who are being green on this code without us paying for it? So I think there might be this impetus mismatching too, and that you might sort of find the sort of student institutional arrangements where there are the foundations or these corporations that help sort of mask the impetus between what the community needs and what the problem is. Like in new democracy, y'all, you worry about what's the second president going to be trying to get out of succession. Yeah. A whole bunch of these systems out there, we've had a succession. Not yet. So I guess you're applying to one's core goal, applying these steps aside, you'll feel actually an unleashed bunch of creativity that can be positive that even, you know... Again, we have to see. Now, another thing's happening is the corporation's understanding of it. And so one of the things that's a network effect, you know, look at eBay or something like that, what you do is you corral people. It's not your content, I mean, or YouTube. You corral people, but you own the corral. And everybody else sort of can't walk away very easily because they have to walk away on masks to get the same sort of network effect going. So there are ways that, you know, even if users are doing the innovation, which is what we're talking about, there's all sorts of debates and war is still going on as to who's now home in the fact of the space. So, of course we're doing it. First comment on that question, comment is that look at the commercial space, and we talk about the manufacturers that sort of, they're gonna dominate you or whatever, but if you look, I fund startups, you look at the startup world, a lot of the problem with this innovation is that the less sexy problems don't get solved. And so, manageability and predictability and all of these things that make these innovations more valuable in the long run don't get addressed in the same way and commercial entities come in and say, okay, fine, let's solve this problem. So, you know, LifeSQL doesn't become the cheapest alternative necessarily, even though it's free. So I think the question is whether you get to see this thing called get satisfaction, which is kind of open source customer support thing and the guy's starting to face it and say, look, you know, instead of trying to convince people to do a better job with customer support, we're just gonna create communities and ground it up and we're gonna encourage you, the company, to come and understand what your customers think and strikes me as a very much an open forum for consumer innovation or, you know, consumers say, their poster child example right now is the Timbuktu bag, which, you know, the zealots of the Timbuktu who go to get satisfaction of the Timbuktu community and say, you know, why is it that your zipper compartment in this messaging bag sucks and whatever and they have, and then everybody said, oh yeah, you know, it should be made out of felt because then it would slip and you know, there's this back and forth and then the company comes in and says, that's an awesome idea. And it's incorporated into the text bag. So you know, we saw this, you know, this is the history of Usenet, right? Usenet support groups stood out a long time ago around all, not just open source products, but Microsoft still has this big MVP community and with Microsoft it's been an ongoing debate as to what we do with these people that we don't pay, right, but they're answering our support questions for us, right? It's been a long going, some are gonna kill them, some are gonna embrace them because firms don't know what to do but all these people who are like talking about their products and saying what's wrong with them and suggesting improvements when the marketing people sort of say, keep it all hidden and away from them. Well that's funny, in fact, I was having a conversation with the CMO of HB, he said, I'm trying to embrace blogging, I'm on the course, as part I was kind of, he said, what do we do with blogging, what would we blog about? And I said, well why don't you take your biggest support problems and blog about that? And he answered, you know, you get these questions where people say, oh would we stop because whatever, he said, that's what you blog about, he was like, ooh that's not scary. I bet he didn't do it, I like that. So, yeah, let's push it a little bit. So, one of the things that I've noticed patterns, so I'm curious if you've seen different patterns in the kinds of communities that have a, to me it feels like they're often driven by the kind of motivations of the participants in the community, for example, one that can solve them a problem and whether or not you've categorized some of those, I'm specifically thinking of, like Firefox plug-in example, I've participated in an electronic hobbyist community and there's a lot of values in those communities around novelty, originality, and ownership. So, people are often, and same with the plug-ins, people are often doing a lot of the repeat, like a whole bunch of people will go, oh well I want ownership over this, so I'm gonna build my own version of it, and a lot less than the current example we have, people were sort of doing, working in the same code. It's very much not like what we saw in software. So, in those models I see replicating different places, I'm wondering, do you see patterns, do you categorize those at all? So, Carlos Baldwin has a theory about this, she's our colleague at Harvard with us, and what you see is that, when you get this kind of, like, there's 115 text editors on SourceFord, like do we need, we did Emacs, and then we did, do we need, and what it is is that the design space is being explored, right? So, Emacs has done one trajectory, and for some other reasons somebody else wants to do something else, and somebody else, and that the design space is being explored by all these different, similar types of the same thing being done, and then there may be, you know, sort of IP reasons for it, sort of Carlos and our colleague, Joachim Henkel are doing this notion of IP modularity, that sometimes you modularize code because of IP constraints around it as well. And so, I get where the beginning of this thing where what we see is that the design space is being explored out, and because the costs to enter are so low, anybody can set up their own extension, their own plugin, that we sort of see this flourishment of this design space being explored, and then one will pop either because of path dependence, because it was the first one, it was the best one, and so on, or that over time there's a space shift between the old one and the new one along the way. So maybe we just do a quick summary, can we do that? How much time do we have? Well, we have to stop a couple minutes. So quick summary, users are unique in the sense that they don't have to benefit from selling something. As a consequence, they can share freely. The power of this is such that it is going to dominate basically the traditional model of the closed manufacturing relation. And what we are all engaged in now is figuring out the new rules, figuring out the new models, and so on and so forth. And this is really enormously exciting because it's an area where really the old way has been evolved over many, many years, and people are just beginning to wrap their heads around the new, all of us are obviously. And so, I mean, we're lucky to live in this kind of thing. I'm always interested in further details or whatever. Go to Eric's website. Well, I have a free book. You can get two at the same price. Just go to my website and you can download it. And I talk a lot about the whole communities activity. Karim also has a free book on there as well, right? Yeah, I'll open source there. Yeah, yeah. And if you're going to open source something, I think that would be you. There's a bunch of papers, some spots for the world. Yeah. Do you want me to do this? Open source.mit.edu. So thank you very much.