 Great. So I have the opportunity today to speak to you about a research project of mine, which I'm doing at the University of Limerick in Ireland, and it is about Debian. It is a PhD research, and I've been carrying through with it for almost four years already, but the actual research on which I'm reporting today has been conducted over the period of the last year. The topic is tool adoption in the Debian project. Well, that's sort of a working title at the moment. What I'm looking at and what I'm interested in is finding out what makes you choose a new tool or what makes you reject a new tool and not adopt it. This is what I'm going to talk about today. First of all, I want to give you an introduction to the method that I used for the research before we actually start talking about the research approach. Just give you a general overview of what I did, and then out of that I can present the results, 24 influences that I found, and if we have time towards the end, I would like to give you a couple of gems, a couple of really nice quotes and smart things that have been said, and then end with talking a little bit about what the Delphi method, which all of this is a lot about what the role of that method could be in a Floss project, specifically the Debian project. So let me start with the Delphi method, and it goes back to a very simple question. You want to be a millionaire. You all know the game show, and you probably all wondered at some point in time what happens if you're up there and you don't know the answer, right? That's fail. But you have three jokers. The first joker is you can phone a friend. The second joker is you can ask the audience, and the third joker is you can have two of the four choices eliminated, the 50-50 chance. Which one of these three do you think is the most powerful? Ask the audience? No, the last one. Or 50-50? Phone a friend? All right, 30% say this, and 30%, well, you know, 30% of you are right, and the rest is wrong. Research shows that, ask the audience is by far the best method to find an answer to a question that you don't know. And this goes back to the principle called the wisdom of crowds. That's actually a new book. The method is a lot older. And it's based on this principle that once you have enough diversity among several heads, they are better at making subjective conjectures than a single person is. And the important point here that I like to highlight is the enough diversity. Because obviously if you just have a couple of people who all have the same interests, then they are not going to be any better at finding a conjecture than every single one. But as soon as you have diversity, you get to argue in between and hopefully find consensus. So the Delphi method is a method that was developed in the 40s by the U.S. Corporation Rand, which is a defense corporation or established by the Department of Defense to answer the question what the impact of technology would be on warfare. This was the building up to Cold War, and people were getting interested in that, and obviously everybody said, look, let's just ask experts, a couple of them, and we'll be able to find a better answer. The Delphi method grew out of this, and it's an iterative method that starts with a question. And this question is fed to a number of experts in this initial round. And you can see that the question goes to each one of those experts independently, and as a matter of fact, in my study and in many of the Delphi studies, the experts are anonymous. So none of the other experts know who else is serving on this panel. The experts then go ahead and answer the question and send it into the moderator, which in my study was me, and the job of the moderator is now to concatenate and integrate all of these responses still anonymized into a coherent piece that I can then, or in my study, I could then pass out into the second round. So in the second round, you had a coherent piece of concatenated information, which was basically the aggregated opinions of all the people, and now they could respond to that. And there are many different variables that you can set, you can either ask them to respond and change their minds, or you can formulate a new question in the context of all of the findings that so far have been found. And then X2 repeats two, three, four or five times until you have a synthesis, which could be consensus or dissensus. This is what I call, or what people call a moderated group communication technique, and you'll see very shortly why this is probably of interest, possibly of interest to the Debian Project. It's anonymous, which means that everybody gets judged for what they have to say for the merits of their input, not for social status. Or there are also no interpersonal conflicts getting in the way because you don't actually know who you're talking with. So all you can deal with is the actual content. It has controlled feedback. I really like this, especially in the context of the Debian Project, because if you remember the graph, all of the experts fed their responses back to me, and I had to concatenate them and then send them back out. There was no way that these experts could have engaged in a flame war with each other. And there's the group response. So there's a couple of people, experts that are there who have been selected onto the panel because they seem to know what they're talking about. And through this method, we're establishing a sort of group opinion in the whole process. So if this is, if this is clear so far, I don't want to go too much into details, then I'd like to move on to the research approach. And I'll, of course, tell you a little bit more about how I actually did it. Are there any questions up to this point? None. All right, then let me move on. Research approach. So my objective for this research was three-fold. Actually, there is a main objective, the first one, and then there are two sub-objectives that have sort of crystallized out later. I started doing this because I really wanted to know why people would choose Git over BZR or why people would choose Dev Helper over CDBS. It doesn't matter about the individual tools. It really occurred to me that there must be something that makes these people tick, us people tick, and influences our decisions. So my objective was to identify what the salient influences are that shape our adoption decisions, adoption or rejection decisions. And while I was preparing for this and while I was using the Delphi method, it occurred to me more and more that this is really cool stuff for Floss because it works very much in line with what we are already doing, mailing list discussions, but it takes some of the energy out of it. So my second objective became to extensively document and explain the entire process to facilitate its future use. Because when I started, nobody had ever done a Delphi study in a Floss context, at least none that I know of, and nobody had ever done a Delphi study that was comparable. And even if there was a Delphi study that was sort of interesting in the context, then it was a four-page paper without any explanation about how they actually did it. So I had to piece all of this together and I decided that once and for all, you know, let's just document it. And then obviously, since this is about Debian, all of the data are going to be available under a free license, and that includes the data that has been submitted by the experts on the panel for future research and also maybe because you're just interested. So I'd like to say a few things about the research just in order to set a scene. First of all, what I'm doing is exploratory research. I'm trying to gather breadth. I'm trying to find as much information as possible. So please do not expect from me to at the end of this session tell you the one thing that explains everyone's behavior in our project. What I have are 24 things that kind of explain it, but that was the goal. Go into breadth. Also, those of you who have been in Gauden's talk just now, he was doing something called quantitative research. He had numbers. I don't. I do qualitative research, which means I use words, I use opinions, I use people, and that's inherently biased. So keep that in mind. There is a lot of bias in this study. The important thing is that I'm trying to make it as explicit as possible so that people can always say, yeah, well, this is biased and he identified it as such. Identify all the sources of bias. And then thirdly, this is sociology. So actually, I'm most of this work is not mine. I'm trying to facilitate it. I'm trying to hypothesis in the end, but most of the input that went into the results that I'm about to show comes from the panel of experts. Keep that in mind as well. Don't throw anything when I say something you don't like. It's not my fault. So that gets me to the panel. The important thing as I identified early on is that the panel is diverse. And then there are several other considerations, of course, that you have to make when selecting the panel, including the size, the communication medium and so on and so forth. I'm going to get back to that a little bit later. But let me show you first what I did in order to ensure the diversity on this panel. And this is already biased, but I think it's a pretty good approach. I identified four different dichotomies that where then people would place themselves on. For instance, the first dichotomy is whether you have a very uniform or a very diverse package catalog. Are you doing package Pearl and every single package is more or less the same? Or are you doing, for instance, package games where the packages are kind of related in spirit, but they're not really the same? So you have a very diverse package catalog. The second dichotomy was are you using a uniform or a very diverse tool set? Are you using the same tool for all that you do in packaging or are you working in X different teams and everyone uses a different tool? The third one is simply are you a team player or are you a solo worker? And the fourth dichotomy was are you interested in improving the process of your work or are you just here to get things done? And among those four dimensions, let me just check what the next slide says. Keep those in mind for a second. Now I'm going to show you what the sampling approach is. In order to find people that would be, that I could ask to categorize themselves, according to these four dichotomies, I wanted to have as many people as possible be potential panelists. So I went out to the mailing lists and bug trackers and all kinds of forum and I identified 162 people who are visible in that they are not just one time contributors, but they have been with the project for several months, who are also innovative and that means that they are ready to try other things. They are not completely closed off. Now I realized that this is somewhat in contrast to what I said just before. Are you interested in process improvement or do you want to get things done? That's not necessarily the same as innovative. Innovative simply means are you earlier in the adoption curve? Are you someone that understands the sort of bigger picture? Are you engaging in discussions, basically? And I asked those 162 people for nominations. This is what's called snowball sampling. And a total of like 460 nominations or something is what I received. And so I processed those data and I figured out or identified 43 nominees that had been nominated more than two times and there were some other some nominees couldn't didn't want to participate so I couldn't include them in the set. But anyway, 43 of these nominees I asked for the self categorization. And self categorization is please place yourself on these four dichotomies. Are you a solo worker? Are you a team player? And so on. And then I imagined these four dichotomies as a 4D feature matrix. The first comment from my PhD supervisor was could you please draw that? I didn't manage yet. But you all know what a 4D feature matrix is like. It has 16 corners, right? So these four dichotomies, that means ideally I get 16 people, one of them lonely sitting in each one of the corners. So this is qualitative, right? This is qualitative. This does not mean that I did, you know, brain surgery and found out exactly which corner you belonged in and whether you fit 79% or 90%. This was sort of like, you know, just trying to make sure that this is diverse. So there's another source of bias here. But as long as I make it explicit, this is sociology. I also picked three people from the center. So a total of 19 people went onto the panel. And we are ready to go and engage in the Delphi discussion. And the Delphi discussion that we did was a three round discussion. And I'm just going to quickly give you the questions that I asked at each round. I don't want to go into too much detail. You can obtain that detail from my web page later upon time. The first question is exactly that. What are those, where does that space come from? That's weird. Please describe at least six influences that you take into consideration when you adopt or not adopt a tool or technique. Please identify at least six such influences in the Debian project. And then I put in there also witness again on future occasions. So I kind of wanted to make sure that this is not Debian as we used to be, but sort of like Debian as we are. So something that you expect that will happen again is more or less a trait of what we are. And I received replies. I collated them. I sent them out again in an anonymized way and collated way. And then I asked people to go through and comment whenever they disagreed with what was stated here. Remember, they didn't know who stated it. They could only look at the content and had to say, yeah, this is something I agree with or no, totally not. This is completely wrong. And then in the third round, after receiving those statements, in the third round, I asked them to select the three strongest influences, positive or negative, that they have experienced. Sorry, I'm jumping ahead of myself. Sorry. The second round gave me a lot of data, a lot of replies. And there was a big, big process in which I had to make these data into something that I could present to the panel. Because one of the problems of this study, as I did it, was simply the volume of data. There was way too much. I might get back to that later again. But because I couldn't expect the panelists to help me manage this volume, I had to come up with my own ways. And there was a lot of, it took me four months or something like that to go through all these data. And I ended up coming up with 24 influences. And those influences are backed up with statements from the panelists. So it's not necessarily that I just came up with these 24 influences myself. I tried to extract them from the data, but again, this is an inherent source of bias. So in the third round, once we had these 24 influences, I asked the panelists to select the three strongest influences they have perceived for themselves and for the project and to share details about what happened. And this was basically the final step in an exploratory research, further enhance the data. I was, you know, I was already drowning in volume and I was asking for more, which is kind of silly. But on the other hand, there was literally nothing else I could do. And at least I didn't know of anything. I hope that because making it all available, all the data available, and maybe at some point in time, a future researcher can do something better with those data. Anyway, I'm not saying this failed. I think the output that we established is very valuable and I'd like to present you the 24 influences that we found. There are those. All right. Now I can move on because you're not going to be able to wrap your head around this. So I had to come up with ways in which I could present this to you. And it just so happens nicely that this is also the way that I'm going to be presenting it in the thesis. So I kind of did two things with one job here. So just a general image, but let me explain to you what all this means. And before we do that, I have to go a little into what is called classical diffusion study. Diffusion study is basically the idea of what makes ideas spread. And so far, this is a field that's about 60 years old and there have been literally thousands of studies done in diffusion research. And those studies are usually always, or you can separate them, again, according to four differences here. For instance, the studies have either happened in peer-to-peer settings or where there is top down, sorry, top down communication. The studies have either happened in voluntary settings or authoritarian settings. The studies have either happened in low dependency settings where it didn't matter what your choice was, it wouldn't affect anyone else, or in very high dependency settings, where what you did affected everyone else. And the studies have either been done in individual or in community slash organizational settings. It just turns out that the studies are either those four or those four. But we, in the Debian project, are somewhat spread. We are peer-to-peer voluntary, but we have very high levels of dependencies because we are trying to cooperate, and we are definitely a community and we are operating at an organizational level. So now, in order to make sense of this, I had to look at multiple things. I had to look at what happened in the past, what was done on the individual level and what was done on the organizational level. And one of the important contributions by this guy called Everett Rogers, who is basically the diffusion studies god, is his innovation decision stages model for the individual. And he says individual adoption is a process. It starts with your knowledge. You are discovering something. Then you are forming an opinion. Then you are making a decision for or against the tool. Then you are implementing it or reinventing it to suit your own needs. And finally, possibly, that's what the asterisk is supposed to do. You might reach a confirmation stage where you go, yeah, I did well. This was exactly the right tool. You can always go back in stages. And even at the persuasion stage, obviously, you already look forward into the implementation stage. It's important to remember that this is really just the sort of framework that helps someone to talk about data. This is not exactly what's happening, but it's a framework, a helpful aid. So this is on the individual level. And there's also another model. This is called IS implementation stages at the organizational level, where organizational adoption is obviously also a process. Here it starts with a pressure to change, agenda setting, or someone like a CTO saying, we're going to do this from now on. Then there is resource allocation, basically a commitment to do something. Then there is the period where the innovation and the organization converge. We call it adaptation, and it's very similar to the individual reinvention. Basically, you might not be able to put the tool to use as is, but you might have to make minor modifications to it. As soon as the understanding grows, the innovation is set to be accepted within the organization. And then there is the use performance satisfaction stage, which is where benefits are actually becoming noticeable. And once those benefits are becoming noticeable, then it goes really quickly into the sixth stage, the incorporation stage, which is where the innovation loses its identity. It's nothing special anymore. It has become part of your regular activities. So individual adoption and organizational adoption, those two, neither of those can really explain what's going on in the Libyan project, because we are a free software project. And we don't actually know what happens first. There's certainly no agenda. There's certainly no individual decision that then ends up in the standards in our policy. So in order to explain that, I looked at needs and innovation, whatever comes first. And I find that in the individual case, needs tend to be generated from an innovation. Everybody nowadays is supposed to have a phone that can send crappy quality pictures around the globe, but nobody really needs it, right? But that need is being communicated to you, so the innovation is there, and then the need grew out of that. That's the individual level. And again, it's only a tender. On the organizational level, however, it is mostly that the need exists, and then an innovation is sought to satisfy that need. And in the afterward, I think that the only reasonable explanation I could come up with, at least, is that there is an entire, or almost entire, individual adoption process happening multiple times. A thousand people are doing individual adoption, and that, those different processes, then at some point in time, merge into an organizational need, and out of that comes the organizational adoption process. So let's look at these two models that I've identified earlier, and put them next to each other, and then see what's actually happening, or what could be one explanation of a timeframe. At the organizational level, we have initiation, at the individual level, we have knowledge. Those two kind of seem to match up. One is creating a need for change exists, and the other one is knowledge. Maybe that's also a need for change. Then you have adoption, the resource allocation, and on the other side, on the individual side, persuasion and decision. Basically like, let's do it on both sides. So those seem to match up. Then you have adaptation, implementation, where you are putting it to use, and you're possibly converging the organization or the individual with the innovation. And then something happens that I had to put in yellow, because it's not part of the individual adoption process. Then I think what happens is there's a shift now from the individual adopt and the many different individual adoptions that are happening over to the organizational adoption, which is critical mass. As soon as enough people are using it, there will also be acceptance that is built in the organization. Acceptance is understanding that grows. It doesn't mean that everybody agrees with it. Acceptance means understanding is growing. So by the time critical mass is reached on an individual level or many, many individuals, then the organization starts to understand what's going on here. And maybe if it's positive, the organization starts to put it to use and get benefits, reap benefits out of that innovation, and that is in some ways a confirmation for the user who did it first. On the other hand, I'll get to the influences in a second, and then the last one, incorporation, that doesn't individual level. So my proposed sequence in this is that we start at an individual level. We start over there with knowledge, and then the individual goes over to persuasion and decision and implements. And then once many people have done that, once we reach critical mass, then we have an initiation at the organizational level going on, followed by adoption, acceptance, adaptation, acceptance, and then use on the organizational level and confirmation on the individual level sort of happen at the same time. And finally, the last step in the processes in corporation. So if this sequence seems kind of acceptable to you, then I'd like to move on and give you the 24 influences rolled out along that time frame. All right, we'll start with the first one, individual knowledge. And this is where cognitive knowledge is being created. Cognitive knowledge means you're knowing about something, but you don't actually have an opinion or feeling about it. And this stage ends with the first impression. So you can linger in the knowledge stage forever because you don't really want to be dealing with it. It can be there, but only as soon as you actually created, like consciously thought about something and created a first impression, then you're leaving this stage. Before the first impression, there are three important influences that are happening. One of them is the entire big area of marketing, creating awareness, awareness, and meeting needs. And this includes communication within the project, block posts, basically establishing or providing the ability for people to have multiple glimpses at something. And these multiple glimpses is what leads me into the next point, sedimentation, the nature of spread of ideas. Nobody is just going to accept an idea. The idea kind of has to happen several times. And then it starts to sediment. And once it sediments, then you're starting to accept it. And then you can understand it. And on an organizational level, where this obviously also applies, because we're dealing with many different individual adoptions here, on the organizational level, this means that, for instance, many different people have to write about the same thing from completely different angles. Because whatever I write, if I describe my Git workflow, some people might be like, this guy is on crack and rightfully so. But somebody else might be able to explain it a lot easier. And then suddenly you have the two documents and suddenly the idea settles. And then this relates to the third influence at the individual knowledge stage, which is chunking the question or distinction between revolution and evolution, piecewise adoption. If you can somehow spot something revolutionary, something that is supposed to change everything, then maybe you can chunk it up into little parts, that each one of them can be adopted individually. Each one of them can be adopted a lot easier than the whole thing. And each one of them you might even be able to sell as an evolution. So the process that everyone is using evolves because someone is adopting it. Rather than the process that everyone was using is completely overthrown and replaced with a revolution. So once we created that first impression, we have a foot in the door. And at this point in time, we are at the individual persuasion stage. This is where effective knowledge, the feeling, the opinion is being created, positive and negative. But this is also where a lot of forward-looking happens. Because we are all geeks, we are going to look at the software, and before we like it, it has to be implemented in the proper programming language. And it has to have spaces instead of tabs. And whatever it is, we have a lot of different opinions there. And it's personal. Every single one of us has their own preferences here. But once that first impression is formed, there is a foot in the door. The innovation is settled in the person. And now, basically, let's see what determines whether the person will be able to form a positive opinion or not. Now there is something here that is possibly one of the most important points of the entire study, which is a lovely little word that also was created during the study called peer collation. And the idea is simply where a peer-to-peer network where a lot of different people, we talk to each other through a lot of different channels, and there are very different levels of respect. And if somebody you respect a lot says something to you, you're going to like it or incorporate it into your own opinion a lot easier than if somebody you dislike says something to you. And it goes the other way around. If somebody who dislikes says something to you, you might just form the opposite opinion to be a rebel or something like that. So this is a very central one. And this is also one of the points that needs a lot more research. Gaudens has done some in that direction. Your hypothesis about bugs that are filed by people who have higher social standing get fixed faster. That's something here. That's the same level of respect that I'm talking about. So I just kind of like mold it all together into what we call peer collation because I couldn't separate this out even further and increase the data much more. Now, out of peer collation, obviously, there's at some point in time, there is a need for consensus. And consensus, the influence of consensus doesn't mean that once there's consensus, it is basically authoritarian and it has been decided. Now you have to do it. Consensus is also a lot about the process, how the consensus has been found. If the process, if the consensus has been found through way too much email discussion, that might put people off. On the other hand, if the consensus was built very nicely, that might just make people look at it because they're really interested in something that was accepted, you know, there was consensus. Like in the Debian project, there's consensus, you better look at it. On the flip side of things, there's resistance. And again, this is not only negative, even though it might sound like that, but it could be positive. Resistance might allow you, like a sort of like general resistance in the project against something, might allow you to say, hey, I'm just not even going to look at that, I have other things to do. Let's wait until that resistance has toned down and then I can start looking at it. So it's a little bit of a pre-filter and it also makes sure that only the good stuff manages to come through. So maybe at this point in time, you can see that there's a little bit of a leak from the organizational level, adoption level into the individual process. It's not possible to separate them, but again, this is really just a presentation framework, a way to describe these 24 influences. And now the last point at the second stage is sustainability. And this goes back to forward looking. A lot of the influences that we will meet at later stages could also be in this stage right now. For instance, there's one influence that I'll talk about later, which is modularity. We all like modularity, but I enlisted it later on. But at this stage, we make a decision whether a tool is sustainable. And part of that decision involves us looking at how it's programmed. A modular tool will be more sustainable to us. And so at this stage, we're forward looking, but we're forming an opinion. And because it's modular, we might decide that this is something that we want to look at closer, possibly even decide to adopt. And that gets us into the individual decision stage where we basically decided we like this thing. So what do we do next? We like this thing where hackers, we want to try it, right? Trilability of a tool is a very, very important influence that has been shown. I need to be able to get at that tool, do something with it within 10 minutes. Otherwise, it's just going to be, it's going to require a lot more persuasion. Somebody telling me that this is the best of the tools in the world, but you will have to invest two hours in it for me to even go further. If I can't get my hands dirty at this stage, I might not move on. At this stage also, people tend to look at quality docs. And I call it quality docs because documentation itself doesn't count or isn't everything that we want. Documentation needs to be maintained. It needs to have exactly the right level of verbosity and we need to make sure that we are talking to the right target audience. I'm sure you've heard all of these things before. Documentation is a topic and every one of us had to think about it once or twice before, but this is actually important stuff. If your documentation isn't maintained or it's too verbose or it's too little information or it's talking to developers when really you are a user, then this influence might not lead an adopter to choose your tool. Related to this is examples and the role of examples and templates because apparently this is one of the things that I found out and became apparent in the study was that a lot of us actually go and copy a template and then modify it. We don't read the man page and then draft our configuration files from scratch. We take a template. So what's the role of examples versus documents? I mean, examples make a tool much more accessible at first, but of course they don't replace documentation. And I think that's an important message in here. You can't just put a wiki out there with like, you know, Bluetooth for instance, Blue's audio is great for that. They just kind of put a wiki out there with examples how you use the new D-Bus stack, but nobody actually tells you how it works. So they need to work on that because otherwise not many people are going to start writing software for the new Bluetooth stack. I don't want to sidetrack, so let me go back here. Cost benefit. This is now the question of whether I want to be investing time or whether I'd prefer to get things done. Yakshaving happens at this stage or the decision for or against Yakshaving happens at this stage. And it's an important one. And one of the things that I think we found out in the studies that in the Debian project, we're all pretty much aware of Yakshaving. I mean, there are still times when we just still fall for it, yeah, because it's just too beautiful and conducive and everything. But we're pretty much aware of it. We're pretty much aware that we can sink hours of time into something without actually getting the task done that would have only taken four minutes. And we're careful of that, but sometimes it happens. So this is something that we investigated at this point in time. Is this worth it? Is it worth to invest the time that I could use to actually get work done? Did you have a question, Paul? And then the last point here at this stage is compatibility. What is the amount of effort required to actually make that change? Something that is finger compatible that just kind of like replaces your current workflow, but is a different tool will be adopted very quickly. Something that needs a complete overhaul of everything you've ever done will not be adopted. Okay, so at the end of the individual decision stage, the individual has made a decision for or against. And against means, well, rejection, we don't have to deal with that. We're going to move forward. So let's assume that the individual has adopted and now we get to the implementation phase. And this is where I thought that the UNIX principles come out, transparency, modularity and generosity at the implementation stage. And there are very many ways to explain that. One of them could be that it is a lot easier to put a modular tool that has little tiny modules into your workflow than it is to have this monolithic thing that needs space in your workflow and doesn't really fit. So that goes back to compatibility. Another one might simply be that at the individual implementation stage, you are doing a certain level of reinvention. You are converging with the innovation. You might change your workflow a little bit, but you might also change the innovation a little bit. So transparent tools that you can easily understand, modular tools that you can easily piece together in UNIX façon and generosity, which means you can use that tool not only for packaging but also to manage your home directory and to do your email and to do graphic design. That's a good one. That will actually make it a lot easier for you to implement it because the benefit rises, I suppose. You'll see those three again at the organizational adaptation phase happens at all stages. But now after the individual implementation happens, we have this critical mass thing happening, right? So a lot of people are using something and suddenly there's the initiation at an organizational level because at the organizational level we tend to think that uniformity is something that is important to us. If we have 20,000 packages maintained in 20,000 different ways, we are hardly a distribution. If we have 20,000 packages maintained in five different ways, we are closer to a distribution. So uniformity is beneficial to the organizational need, but it's also something that could act as a driver for change. If there's actually something that has established critical mass, then it's a perfect opportunity for outlying factions to realign with a project because they might just say, look, this tool is kind of discontinued. Everyone is jumping on this bandwagon. Let's just do it. So this uniformity influence at the organizational initiation stage is where this need gets created. And until this need is created, we can't really get out of the organizational initiation stage. And then something interesting happens. Once we establish that uniformity is good and this is sort of like a general need and needs to spread some more, then we hit network effects. Because then we get to the point where we really get faced with the dependencies between the work of people. Whatever you choose to do might affect a lot of other people. And so the network effects are somewhat dampening at this stage, unless you can resolve them, unless you can actually create a way for everyone to start using it, you won't be able to leave this stage. The network effects will keep you back. The adoption will not be able to go through at the organizational level. On the other hand, network effects also have enabling effects. Namely, you might be working in one team and they just switch to this cool new workflow for packaging. And you will just take that workflow and take it to another team that you've been working on and implemented there. So through these teams and through the dependencies, we can actually also spread ideas. Now again, organizational adaptation. This is when the organization and the innovation converge. If we as Debian adopt something, then it's really helpful, modular. So I'll just skip that because I already talked about it. Let's go to the organizational acceptance stage. This is now the stage where basically a lot of people have adopted it and there is an understanding that is growing in the organization. And at this level of understanding, also the use increases. Even more people start using it. So critical mass still plays into this. Scaled use, which is defined as the various degrees of use of a software and related to that graduate migrations, make it possible for everyone to use a tool. If there's this giant workflow helper out there, which can do everything under the sun and cook you breakfast, then if you can just use this one little part in it to make your workflow just a little bit better, then that's also a foot in the door in some ways. You can start using that tool without changing everything. You can do a gradual migration and possibly still stay compatible with all the people that you're working with. At this point in time, obviously another factor maturity also plays into this. Maturity being can I actually use the software without tracking development and without rewriting my script every day because the API or ABI have changed again. For a tool to reach organizational acceptance, it has to be mature to the point where deprecation happens if something needs to get changed and also mature in terms of its use must be stable. You must be able to use it without having to be tracking that development. Now, if you might recall, use and confirmation, that's sort of like a level on both stages. This is where return on investment happens. Return on investment means there's a value, there's a future benefit for you, which could be, for instance, more users filing bugs that improve your software or more users writing documentation that make it easier for you to use it or just in general more users meaning the project grows and it becomes more innovative. There are more features, it gets much better. And then finally, in the last stage, now this is the organizational level, there's nothing anymore at the individual level. This is the incorporation stage where an innovation loses its identity and becomes part of the standard process. This is where standards are. Standards get defined and the Debian policy, as our standards document, is a document that doesn't get defined in the beginning as a sort of agenda setting device, it gets defined at the very end to document the current and ongoing practices. So at this stage, the influence of standards is what the effect of creating a standard and how do you create one. And obviously not mandating any sort of change is important in our group, but on the other hand, this plays back into uniformity and also makes it easier for the individual not to have to make decisions. A standard enables you not to have to make a decision, you can just assume what everybody else has done. And then related to this is the last influence quality assurance automated tests. Linzian for instance, this is of course a very strong way of communicating a standard because Linzian, I haven't actually looked at the adoption of Linzian itself, why everybody started to use it, but I've gone there, I've talked to the original author and we're trying to recreate this. But obviously if Linzian starts to complain about something, a lot of you will do it. But this then again factors into consensus and there are a lot of interdependencies between these influences. So that was the 24 that I presented and I kind of like try to list them here in a table. I also have another slide that has the little summaries. I hope you can read that. But at this stage, I still have some more material that I can skip though. At this stage, I'd like to see if there are any questions about these 24 influences. Gaudenz. Did you actually try to validate your process, you propose how it works on an actual example, say adoption of version control systems for packaging or whatever else? That was my original plan to validate the results on exemplars, like look at Depp Helper, look at whatever it is and validate it in that way. However, I found out that in the data that we had, there were already so many references to all of these adoptions that it was really hard at a later point in time to say, yeah and let's now look what these 24 influences, what their role were in Depp Helper and then basically reproduce all of the quotes that happened. So I'm not doing that, but it is part of the result presentation. I'm referencing back to adoptions at other stages in time to explain them. Can you show if you have found a bit of text that you found particularly enlightening, just a bit from the sample that you find enlightening in fishing. All right, excellent. Let's have one more question at this point and then I can give you some of those gems because they're certainly very interesting. Okay, I'll move on right away and then we go back to this later. Gems, all right, so I couldn't, I was like let's figure out three, yeah, let's take three and so I kind of sat down yesterday and there were people that were holding beer in my face and I still had to do three and now I have seven or something like that because it was just not possible to say you're not interesting. You know, I kind of just scanned it and I had help from other people as well and then found that there are so many gems in here. So I'm going to give you a couple, but I'm going to do it fast. For instance, this is my absolute favorite. Automation must be motivated by the desire to do more, not to do less. There are certain tools whose motivation seems to be to alleviate the pain for you so that you have to do less work, but that's probably the wrong way to think about it. The right way might be that you actually want to use your time more efficiently and that's why you use a tool, not because you're lazy. So that's my favorite, but there are other good ones. For instance, many of you have heard me talk about this before in the VCS PKG thing and even in the hallway track. It's important to standardize the right things, interfaces, not processes. If you actually have your VCS layout defined, it doesn't matter anymore what the tool is that you use and that's very much in an alignment with the Debian policy. Debian policy never tells you how you have to do something. It only tells you where you have to arrive and it leaves it up to you how to get there. Incredible recommendations. This is also a very good one. The most persuasive advocates are those that can clearly explain that have significant experience and who, this is really important, who seem to have done all the research and experimentation that we all like to do but usually don't have time for. This really resonated with me. I was like, yeah, that's me. Related to this is the next one. That's an interesting one as well. So there's this hype thing in Debian, right? And sometimes there's overhype. Sometimes everyone starts to speak about something and many of us will just go like, leave me alone. I don't want to deal with this. This is a bad tool. It's been overhyped. And one of the panelists said, well, this is actually not really an ulterior motive. It's not that you're negative against everything. It might just be that you would really like to use that tool. You would really like to have the time to look at that tool, but you already have an overloaded queue. You have no time to look at it and so you feel guilty and because you're feeling guilty, then you're building up hostility. And quite possibly that is true for a lot of things in Debian. Many of you have seen me lately on IRC ask about anti-corporate feelings and actually as part of this DEB CONF, I talked to one of the panelists in order to figure out what exactly he meant with anti-corporate feeling and we reached or he said this and I think this is, you know, it's one of those things where you're like, yeah, sure, that's obvious, but actually just state it and to actually believe it. Why do we have anti-corporate feelings in Debian? Is it really because we hate canonical? Is it really because we're ideologists that don't want to deal with anything to do with money because we're anarchists? Or is it because some of us may have some sort of distrust in corporations and because Debian is a non-corporate distribution, we just tend to flock together and that's why all the anti-corporate sentiments might be accumulating in Debian. Again, this is not a truth. I just thought it was an interesting way of looking at it and it totally like, it opened new worlds for me to think about this. What's another one? Squandering the next generation. Social influences, pure collation within the project and people on the Debian Mentors mailing list saying I won't sponsor you if you're not going to use Quilt for this. Well, that's a great way to push Quilt, isn't it? But it also means that you are effectively imposing your own workflow on the next generation and that might be okay because maybe your workflow is generally accepted. Quilt is pretty good. It might be okay to say that but what you're doing is also making the next generation taking away their possibility, their opportunity to innovate themselves, to find something that is a better way of doing it. Fresh blood in the project might be able to improve it and through this we're actually squandering that. What's another one? Aesthetics and efficiency. Aesthetics are part of efficiency because I am more prone to be efficient and willing to modify something that pleases me than something that's horribly horrible and broken. Resonates very well, I hope with most of you too. Finger compatibility. The tool must not get in the way. It must come naturally. If that doesn't happen then that lowers my productivity because as we could see for instance in subversion and in the case of bizarre as well, those two tools have benefited immensely from the fact that they were basically finger compatible to CVS. CVS lock, CVS commit, CVS add, it's all the same command in the other one so there's finger compatibility at this level and that really helps. So those are the gems. There are a lot more, I promise, but you'll have to wait until the thesis is done and the data are available to get to them. I do have about five more minutes so if you don't object I'll talk a little bit about the Delphi method in Floss. A lot of it may already be plain obvious to you because I already said a lot of things about the Delphi method but let's have a look at it again. Matingless discussions kind of suck, right? They explode in all directions and at some point in time nobody is able to have an overview anymore. Nobody can actually say what the threat is about. The same argument is being made in a thousand different places. It's kind of annoying. With the Delphi method there's controlled feedback. It's called moderation. Yeah sure and some of us don't like moderation but what if moderation is part of like taking the next step? What if you actually use this method to make sure that the mating lists don't explode and it doesn't even have to be a bad people in the project that make things like that explode. It could just simply be that the email does take two minutes to show up in someone else's inbox and so you didn't see the reply by the time you already sent yours and there you go. We forked the discussion. Delphi appears well suited to this and if you compare, if you remember what kernel traffic used to be like, we even had a Debian traffic at some point in time. Those were people that just kind of summarized the entire mailing lists. I put them into a central place as a basis for discussion and so that actually helped a lot to move on. It was also so much work that both of these projects are discontinued and then the last thing I guess I want to say is that Delphi is a toolbox. It's not a prescribed method. There are a lot of different variables. Some of them are here. I have at least 13 more that you have to set but I also tried to make sure that in my thesis all of that will be explained so that in the future it's easier for people to set these variables for the research that you need and then the last thing I'd like to say is that because of the volume of data that was one of the big problems I used the wrong method in the end or maybe I asked the wrong question because to ask someone, to ask a panel of 19 people to please identify at least six influences, yeah that didn't work so well. It was a lot of work on behalf of the moderator. Many, many months that I didn't want to spend and as a conclusion I can say that this is good for single issues. It's not so well suited for exploratory research. So I think that's it from me. Are there any more questions? I think I'm very close with the time. I'm very sorry about that but it is also lunch so if you don't rush out immediately we can stay a little bit. There's Sam and Gaudens. Quickly but wasn't that the sort of exploratory stuff? Wasn't that exactly what this method was originally developed for? Well yes it was originally developed to be exploratory in order to be able to get as many different opinions but then to be able to converge those opinions through further iterations of the Delphi method. So you would actually want to converge at some point in time at a consensus or a dissensus but this was completely impossible. There was no way that like given at least six influences from 19 people I would be able to converge within 10 years or something like that on all of these issues and that's what I mean like if we had taken one single influence like let's say let's discuss the importance of consensus in the project then we would have been able to converge but not like this. Gaudens? There's currently discussion about the DEP proposal and I think it's one thing that could structure at least the organizational adoption process a lot more than we have it now in Debian. How do you think do this or does DEP fit in with your proposed Delphi method? In some ways it fits in perfectly. DEP is as the way I see it is a sort of central location where you collect the arguments for and against and people can argue with arguments from others on the page and you're basically building consensus in a central place. The DEP process is not anonymous but it doesn't have to be to be a Delphi process. An anonymity is only one of the variables. So the DEP process is very nice. I think that maybe at some point in time we might need to get to a point where we have actually an anonymous Delphi method to resolve an issue just because it's too loaded or something like that. Thanks for the presentation it was really interesting for me. The way you expose the individual processes and the organizational processes it seems that they are logical but most of the time we simply see resistance to change. I'm wondering if how would you see the simpler resistance to change not for technical reason, not for maintainability, not for any logical reason maps onto these processes at which level? Sorry, resistance to change. Resistance to change. How do maps there? Well I think that a lot of the influences that are here explain this resistance to change in many different ways. I mean one of them might be one of the most obvious one might be cost-benefit. I just don't have the time. But I think what you're insinuating is that there is a sort of like generic level of resistance to change in Debian because we've never done it that way before. I'll have to think about that. I don't know if that can be explained. On the other hand I think that something like the DEP process or the Delphi method could actually help to jump over that because if there's a resistance to change and you'd ask these people to pre-provide factual arguments against it and they can't do that but you have a very good case of like a DEP formulated with all the arguments for that. Well, sorry to them, right? They put up a strong fight but that fight is over. Igor? You have identified 24 factors influence. Do you think they are all equally important? Is there some order? Are there some top three influence factors? The top three that were sort of identified in the last phase where percolation, quality assurance which means Linzian and that kind of stuff and yeah I don't know what the last one was. I think it might have been uniformity even. I don't know what the last one was but I also didn't like those data at all because a lot of the times I was faced with a challenge of trying to like get the consensus among the group of panelists who were very diverse and so you might be working in package Perl in a team with the same tool set and for you two or three of these are much more important than for myself who is working as a solo player with like all tools spread all over the place and doing crazy stuff all over the packages and I have completely different set of influences and I didn't find any way in which you could actually then take your set of influences and take my set and weigh them and merge that and then come out with like the sort of grand thing. I think that though, percolation, I mean the level of respect that is being built and those things like people who have done the research that I would like to do and to explain clearly what is going on that those are very very important influences and then also Linzian and tools like that that simply enable us to adhere to some sort of standard are very very important as well. Those were sort of like percolation and quality assurance ranked high up there. They were named by most people but I wouldn't necessarily say that that makes them the most important ones. All right. Thank you anyone for attending. Thank you Martin for your talk.