 Yeah You can hear me now. I feel I'm having Right, okay And I can project a little bit more too. All right Okay, thank you. All right. I'll turn this off Okay, welcome everybody to Sunday here at scale 14x and for our first speaker today in the In this track, we're going to have Brian profit Brian will be telling us about Finding the value here in open source projects Brian is currently principal community analyst For Red Hat open source and standards team He's a longtime member of the open source software community and he's authored several books on Linux IOS and other odd operating systems He's also a former technology journalist for read write IP world Linux planet and Linux today. Let's all welcome Brian Thank you. Okay, so The the the topic of this talk is going to be talking about Something that's probably near and dear to a lot of people in the open source community this idea that Community and business do not necessarily always get along This is a common thread throughout a lot of companies that we see this in new startup Entrepreneurial situations. We also see this in even established open source companies like Red Hat and and Suza and In canonical, this is like a common thread where there seems to be this little push-pull pseudo adversarial sometimes not pseudo adversarial Relationship between the idea of community and business and the nature of this talk It's a sort of dispel this notion that they have to be at odds with each other at all And this kind of has come up in my tenure at Red Hat because surprisingly even though Red Hat is regarded as one of the you know Purists for lack of a better term open source companies out there There's still a lot of this internal struggle on a daily basis within our different teams A lot of this is because community can be generally fairly confusing to a lot of people the approach that people can take with community can vary as Widely between like okay, this is a group of users that we're going to respect and we're going to take care of and we're Going to give them Lots of nice things and make them feel good to an actual participatory community Whereas where we need these people to help us with our project But even then that becomes a little bit of a nuanced relationship because sometimes you can do that and say well We're going to we're using this community as a resource But that's all we're doing they're not we're not really going to give them anything back We're not going to give them the same what's going on To like whatever regard is better forms of community Which are participatory and on all levels? It's a two-way street So kind of nailing this down Is something that's very important when we all look at community and Try to figure out what it is we want and what it is we should be doing And I think that even if companies have different goals with community There's still some basic rules of the road that they should follow and make life Better for everybody even if it does seem like it's a drain on resources or we're giving something away and not getting anything back Because this confusion can create a divide and this divide shows up in Any open source company out there? I named a few earlier But I would I could probably Universally say that if you could think of a commercial company that is working with open source in any way shape or form There is going to be at some point And probably more often than you think a divide between what is regarded as the traditional business side Versus the community side and how that divide is bridged is an interesting case because in sometimes what we see is that That the the vibe will actually be bridged by let's say, okay, well, let's make community part of marketing Okay, and if it's part of marketing we understand marketing marketing is a business function We will make it part of that and we will treat it like that as a marketing line item basically Okay, that's a way to go But it might not necessarily be the best way to go because at that point You're really you're getting into the point where you're you're treating that community as a as a resource to be mind And that is probably not your best approach to handling community When you think about how Businesses think of of community, you know businesses traditionally are very much lined up Along the bottom line, that's what a business does if you're a public company in most countries You you're required by law to make as much money as you can and if you're a private company You know, you're you know pretty much going to be feel the same way You're trying to try to get as much revenue that you can squeeze out of your product and your company and your team as you can Okay, it's it's craft, but money is a very big driver here Okay, and I'm not really saying anything that's like hugely new but You know when you think about it from a project point of view And I've kind of been being business a little bit and putting this all on them and saying well, they don't understand community. Well There's the opposite problem too. Let's be fair a lot of community projects that work with corporate entities Often feel like we're not about the money man You know, we don't want to be you know dealing with that. That's not our problem We're here to create something good and pure and solid. So they they they sort of They can almost be from the opposite side adversarial and stiff arming, you know Commercial entities and trying to keep them away. So it's you know, it takes to the tango So it's not just the businesses now Treating the communities the communities can kind of be the same way about the businesses as well This is candy and adversarial relationship at times Okay, and it doesn't have to be okay Community It can be regarded as a resource But it doesn't have to be the kind of resource that is like set up in its own little area And self-contained and it doesn't do anything else beyond that People don't always get community, but people know that they want it so many companies want to be To have a community obviously red hat we want to have a community going and we do With a lot of many of our open source projects Well all of them that we actually directly support but it's not just pure companies like pure open source companies like red hat McAfee They have a community infrastructure Google does ENC Canonical just going down Charles Schwab Charles Schwab when I was researching this I'm like okay They want to have a community manager, but their community is around Their users and that's okay. That's a viable community But you know, what does that look like as far as participation? What does that look like for growth? What does that look like for people? How did they you know? What are their users get out of this and what does Charles Schwab get out of this, you know How many people here remember? When Saturn came out the cars do you remember that they had they used to have the cool Picnics and everything like that and you would have the commercials on the TV and everything and and you would literally I I was too poor to buy a new car at that point in my life But I was like literally sitting there thinking I want to get a Saturn so I can go to these picnics You know and show up That was actually I I got to give GM props for that that was a cool idea for community And it was a brilliant strategy for getting people Embested and inclusive in this community of Saturn owners and I You know I have a soft spot in my heart for Saturn because of that then then I actually drove one and maybe not so much But you know whatever um so Tom the huge company, you know buy one give one they have a community infrastructure as well But where did they get the value? Each one of those companies that I mentioned has a different form of community red hats communities Are about developing code. They're about building projects. They're about You know contributory free and open source software projects That's vastly different goals than Tom's which and and and Charles Schwab who are both very much in the user community point of view So how are they going to get value out of their communities? Why are they bothering with this? And and the cynical part of us can sit here and say you know what they're not really in the community They're just doing this as a marketing strategy They're just doing this to try to get people to feel good and feel nice when basically what they're setting up Is a bunch of targeted users that they can get On line on and monitor and track and get their habits and whatnot and and figure out how to sell things better Yeah, that may be true Okay, the problem is this when software companies do this with free and open source software projects That becomes a problem There's very little chance for me if I you know my daughters buy toms all the time They love the shoes. They love the idea of the buy one give one and and you know, they're good shoes So they love them, but they're never going to have like contributed design for a new shoe to Tom That's not what that community is about Um, they're you know, they're they they can they're going to vote with a pocket book So if a shoe line is successful, it's not because people participated in the creation of that shoe line It's because they you know People were buying and voting with their their cash So we have to be very careful here because there's been a trend lately and that's sort of the origin of this talk where a lot of companies in software land who are distributing open source code are taking that idea of You know community as a marketing arm and they're running a little bit far with that I want to be very careful when I continue with this talk As you go you might be hearing threads of things that like. Hmm. He doesn't like the Apache license. He doesn't like um, he doesn't like Permissive licenses because that means you you know companies are trying to steal things that is actually not what this is about I have no problems with restrictive licenses. I have no problems with permissive licenses Okay, how you act with a license is how is what we're going to be kind of judging you by Okay, you can you can still have full participation with permissive license or not Okay, but you still have to deliver value to your community and and and the way and when you do that You're going to get full value for your your company as well So let's look at one side of this. Let's look at the business value And where they get what does the business need when they talk about value? Well, okay, so the business value or as far as community sorry So again, and I've said this several times already The business value for community. It's obviously okay one It's good marketing and even at red hat we do this the fact that we're you know over at the next building We've got a booth over there. We've got stickers. We've got t-shirts. We've got swag We've got brochures, you know, we all these things and all of our companies that are here are doing this We are advocating our communities Okay We are we are holding them up to the world and we are saying these are the communities that we work with And we are happy to work with them and we want you to be a part of them Okay, you will never be able to separate marketing from community in a commercial infrastructure Okay, it can't be done nor should it be done. So I'm not here saying Let's just kill off this idea that community is marketing is a bad thing It's always going to be there. That's part of my job as a community analyst It's part of any but any community manager's job if they do any form of advocacy that is marketing And to do that when they do that they're building loyalty I'm building brand loyalty with stickers. I'm building project loyalty with you know, hopefully a low barrier to entry for participation And and getting people good documentation so they can jump into their their project more easily Brand awareness mentioned that already support And I'm sorry my slides here are very tiny. So I'm having trouble reading ahead of time You know, that's another value for community and a lot of software companies are actually this is a big part of what they do With community as well It's easier to support your software when your community is there helping You know if I could you know save a little money and and if I have an active and vibrant user community And you know somebody comes along and says hey, I'm having a problem with feature x y or z And it's great if there's a user mailing list or a forum somewhere That where I can point them and I can say hey You know go here first see if somebody's asked that same question and it's wonderful that that question has already been answered Your community has done your support work for you So you've reduced your support costs So that's a great business value for community On the product side You're going to get a higher adoption rate And and this is actually something I'll be I'll be honest This is something we do at red ad all the time and this is something that susan does and you know canonical suing as well The more people that use overt Which is a virtual data center management platform and I kind of pick on it because that was my last community manager role The more people that use that By mere attrition alone We're going to pick up some commercial users of rev which is the commercial downstream The more people that use fedora or sento s the more people that we're going to see pick up rel Okay, which is where we you know we sell the support for and this is where we generate revenue I could say the same thing about the relationship between open susan and susan And there's nothing wrong with that. We're all open about this This is the thing if you if you want to run open susan in the enterprise and you can do it and make that work Sus is all for that You know And and if you need some help along along the way, they've got the infrastructure and the ecosystem That would they like to invite you in and be a part of their commercial product Same thing with canonical same thing with red hat and so forth and so on Having a strong and vibrant community Brings a definite bottom line value add as far as higher adoption rate of your commercial product Just by mere attrition alone And you're going to you're going to see more spending on product as you go So you're going to adopt the product later and long term if a community is strong, we've seen evidence that Um You're going to end up getting a lot more Spending on the product long term now. These are actually not my ideas erica call from sales force and I've got this URL it probably won't be very visible on the video, but I do invite you to Go back and look at the slides that were with this presentation and pull that URL up because she does a very good detailed Explanation of the value of community from the sales force side You know and and this is coming from a company that is not necessarily regarded as an open source company really Um, but a lot of what she is saying is true for software in particular and pretty much any business Whether it's open source or not Now the community reaction to a lot of this And I hinted at this before that the community is not always Friendly about this is not the most positive thing in the world Okay, it's gotten a lot better But in in the past when I remember when 2001 when ibm started making the rounds and investing billions of billion dollars in linux software development And red hat was becoming prominent and oracle was a big coming in and being a big player People were very wary Of business people They wanted to stay pure i'm going to stay with debbie and because I don't I do not want to sell my soul to you You know and we still see that and i'm not making fun of that attitude at all that is a totally valid You know if you don't want to have any dealings with commercial entities fine You know that's because commercial entities I work for one, but we're not perfect You know and because of our size We can do unintentional damage sometimes And I can I can respect that people do not want to be involved in that But so the community reaction to a business approach to community is usually one that is regarded with suspicion And and definitely wearing us if not outright suspicion So what is the So we talked about the value of community as far as As products are concerned Okay, how how do we get the what does the project get out of community? Well, obviously growth when we When you are a community manager one of the first things you're going to try to do if you haven't done it already Is measure the growth of your community? And that can be measured in a whole bunch of different ways Because the idea is basically the the more your project is growing the stronger it is If you have people leaving your project in droves there's a problem And it might not even be your problem. It could be that there's better technology down the road You know, I don't want to really contribute to this project anymore I want to go and contribute the project why because I think their their stuff is cooler And then that leads to a cell or or You have people who leave where they're saying I want to I want to leave because I don't like the way this project is run Or I'm having a personality problem with the people at the top. These are all valid reasons why people leave So obviously then growth indicates that your your project is going to be stronger Okay, but you can't count on that because that can that could turn on you in a second So you've got to look at trends more than like day-to-day things You got to make sure that things are still growing at the right at the at the rates that they need to be And and keeping that growth rate up and running The more you grow the more resources you have the more talent you have in your community The more diversity you have in your community as far as talent and the backgrounds and And and everything gender race everything like that the more resources you have cultural differences make a difference People from different cultures and different backgrounds and different education will bring something new to the table all the time Okay, you don't want to be like the little gated community of all one ethnic group or income level That's boring. All the houses are the same You know, there's nothing there's what's the value there Those people seem to be happy because they're living in their little gated community But what other than you know, you want to be a part of that What's the value of that community? Really? How does that community grow? How does that community? Maintain loyalty these are problems And i'm a big believer in the more you grow the more resources you have and you really need to have that for you Advocacy Okay, that's another Value that community brings to a project the more people who are using it the more people are going to just be advocates for it You know, we all well not all of us I know people who don't but a lot of us and hold this up as a visual here. I'm kind of taped down You know, we've got stickers You know, I love pulling this thing out on an airplane Because people like normal people, you know, and I don't mean to disparage, but sometimes I call them muggles Um, you know, they don't have that going And they look at it and they're like, what are you because you know, I'm kind of an older gentleman, you know And everything like that, you know, I'm I'm pretty like, you know, leave it to beaver kind of guy looking And you know, they're like, what are what are you a hippie? What is this? What is this going on here? You know But this is this is what we do in community all the time This is this goes back to ancient tribal, you know, I'm not a sociologist, but I'm sure it's just some sort of tribe I'm part of this tribe. I've got a sticker You know, we got people redhead to get the fedora tattoos. I'm like, no That's that that's not happening But but they do that because it's part of advocacy and that's a very powerful reason to have a strong community and a project That helps people understand the project Okay So if somebody comes up to you and says, I don't understand what this does The better, you know, the stronger more vibrant in your community. The easier it is. It's going to be to explain it to people They're going to have this sense of community and they're hopefully going to help people understand the project better The other thing that you want to do with advocacy is communicate how useful it is to participants It's not just enough for me to say that overt is a A virtual data center management platform I need to be able to as a community member communicate that how useful it is to use someone outside of my community who hasn't used it Okay, how is this going to be useful to you? Okay, and and build that up and use community resources to help get that message across And then and then this is really nuanced because it seems like it's the same thing So now I've explained to you how useful overt is. I really haven't but we're short on time But hypothetically, I've explained it to you Okay, and what it will do for you now. I actually have to tell you how to use it How do you install it? How do you get these virtual machines from one host to another all the all its documentation Okay, and that's something community also needs to bring to So these are the three areas of advocacy that a community can do for you and should do it for you If you you're a project a software project And then the benefits of this and this may seem a little bit familiar Just like on the business side for the project will have a higher adoption rate if the community is strong Oh, you know so-and-so my friend over here. He's using debbie in how you like that You know, he really really likes that it's gotten really strong It's not it's hard to install anymore because does anybody remember how hard it was to install debbie in Yeah, I almost shot my computer the first time I did it back in the early 2000s. It was very painful But that is no longer a thing Deming is like easy as easy to installs any other distribution now Except maybe gentoo because I still don't understand that and that's my fault not theirs. Um more project recognition These parts tie in very very well The value of community is much closer To an alignment with business side here Okay, you get the higher adoption rate. You get the more you get more brand product rigid project recognition. Sorry Okay So community versus commercial the age old problem that we have Um because they do tend to be at loggerheads sometimes and we see this a lot It's the upstream downstream model and different companies are going to deal with this in different ways It is very important and and I'll deliver this lesson from a red hat point of view And you take that with the grain of salt as you want because not everybody does it the way we do But one of the things that we have found that is very very Very important to differentiate between the work that is done in the upstream project Versus the work that is done in the downstream project Okay, and and how that work is done Okay, so Give you an example one of the thing um Oh shoot, you know look I gotta peek at my phone here for a second because I just had a brain Okay, slack. Sorry I was trying to think in the name of the app. Okay, so has everybody heard of slack As a chatting app and things like this. Okay, so actually this is something that we're actually talking about a little bit at red hat Because a lot of people especially on the business side to sales side and everything like that They don't really like irc And you know I kind of don't blame them because if you looked at irc From somebody who hasn't been using it forever It's a really simplified Arcane system, even if you're using something pretty like xchat or something like that, you know, I you know, I'm running I'm hardcore. I'm an irsi. You know, so I'm doing command line irc, you know, but but I get that You know xchat is very limited. It's just sort of like hey, how are you? Hey, how are you and you're you're watching channels and things like that Slack is more interesting to people Okay at red hat And and and there's been a movement lately of saying hey, we really should think about adopting slack as an internal communication tool and blah blah blah blah and it's like, um, yeah, and and and I I've got slack on my phone I use it to talk to some team members usually when we're here When we're out at conferences because I can't really get irc very well on my phone like persistently So it's nice to have slack as the side channel where I can say hey, we we need to go to in and out burgers now Right and yeah, because we're so sophisticated, you know, but that's what we need it for So we're using it. It's fine. But the problem that we're having and this fits to the upstream downstream thing Is that if red hat were to adopt and start using slack Almost exclusively for chat Then we would see an adoption fall off very quickly of irc And the problem there is if we're running red hat a domain red hat slack dot com Right, what happens to the communities? Okay, because we have we use irc predominantly to talk to our community members who do not work for red hat They are not behind our firewall You know, they're communicating with us on public channels if we use if we move to a private channel That's inside our firewall That's great for us. That's that's terrible for our our communities So that is a current Kind of real-time example of one way you have to make sure That you keep resources Dedicated and focused dedicated on the upstream and do it on the upstream first Okay, one of the one of the challenges we had again overt Because i'm familiar with it one of the challenges that we had over and we still do is our documentation was terrible Okay, and what was happening was Or we have a team at red hat that works on the documentation as part of our support structure So part of the value add that you get if you buy a commercial Support for any of our products is you get training you get support and you get documentation Okay, but what was happening was the upstream documentation and overt was uh terrible Okay, it still kind of is we're working on it. I swear to god Okay, but it's bad and then what would happen was the commercial people would basically be writing the documentation for the commercial product only from scratch Okay, and and it was better and they were delivering it and they're delivering it to Their customers. That's not the what our our customer. Sorry. That's not the right way to do that And they know that okay, they know that But then there's a couple of things one is they say well they turn to the upstream and they say well You kind of really need to get something better because we're using overt.org is based on media wiki They were they're using, you know doc book based documentation So through publican media wiki does not deal well Because media wiki is a wiki and this is a semantic text language down here and the downstream These two things do not talk to each other At all it's it's it's really awful and ugly and evil. So they're first off they're turning us today. You guys need a better website And we're like, okay But then we're but then we're like to them We're like we need you guys to actually stop writing for the downstream and start writing for overt And do that in the upstream work on that first And then you should if we've got the right platform for you Because admittedly we don't now but we're moving to it Then hypothetically you should be writing all the documentation and overt And then when it's time to release a commercial version You just push a button rebrand everything and say boom. There's your commercial version You know So even companies like red hat are not perfect here So the important thing is two things know where the boundaries are between upstream and downstream Make sure you are dedicating resources to the upstream Make sure they're separate from the downstream And really this is personal bias, but I think this is the right way to go Dedicate those resources more to the upstream less to the downstream. Let the downstream take care of itself I think personally for me 75 to 80 percent of the work should be done in the upstream project The rest should be done in the downstream As an afterthought where you rebrand everything and you secure everything and you lock features down and do what you need to do another thing another thing that we see In community versus commercial as a way of bridging this gap is the notion of open core Open core basically says our software is open Okay, and you can have the free version, but it's not going to be as full features as The the version that will sell you And we don't do this at red hat at all There are differences between You know fedora and rel obviously, okay But there's not those differences are related to the fact that we're trying to like stabilize things We got a we got to use stable more stable code that is enterprise ready Fedora is great. It's not enterprise ready Okay, but we're not handing when we give you rel. We're not handing you cripple ware Okay, we're not saying, you know, this is something that's less than fedora. We're saying no, this is more stable and sturdier than fedora Okay open core These companies will be handing you things that are not as They're deliberately not as feature rich If you want these features, you're gonna have to pay us money for them Okay And and a lot of people look at that and they say well, okay, but if I know that going in fine That's cool. At least they're upfront about it. They're not really trying to trick me or anything like that But the classic problem there is this Say i'm offering in my software the ability to have you know Talk to a mobile app You know And that's part of the full featured sellable version of the program And then along comes this new hot young developer and you know real hot shot and she says something along the lines of Hey, I know a way we can make this work And I can upload I figured this out we can upload the code Guarantee you what's going to happen is that the people who you know the company that runs that that project Is going to reject that code Because even though somebody's come along and said I could do this and I can do this Much more efficiently and we can talk to the mobile app and everything like that They don't want that to happen because they don't want that in the free software version Because then what would they sell? You know, that's the classic problem. So what's that developer going to do? She's going to feel devalued You know her not going to feel like her work was taken seriously And worse for the company involved ironically she'll modify the code and take it to a competitor who has a more open project You know and suddenly now they have that feature oops community diversity And and and and I want to be very very very clear here I support diversity in all forms race gender religion whatever okay But at this point the context of diversity Is how many people work on your project that work for the company? And and then don't work for the company Okay And we have stats on this we have dashboards on all of our the projects that are open source and standards group and red hat support okay And I will tell you with absolutely no pride whatsoever that for overt again the one that i'm the most familiar with The last time I looked at the stat, which I think was in november 88 of the contributors on overt work for red hat That's not good That is not diverse Because that's a problem because that basically means if i'm if you don't work for red hat and you want to contribute to the project like overt You're going to feel like why well, why would I even do this because if they don't want to see this feature They're not going to accept it Because they have a plan for avert and they have a plan for rev and they're going to go with that plan I will tell you that it's not true, but that's just one guy. You don't know me from adam You're going to look at 88 percent of participants for red hat and you're going to be like he had no he's blowing smoke you know So you need to try to make your communities as diverse as you can and get outside people in As much as you can and that is a very very very hard thing for commercial entities to accept Because they feel like that is a loss of control I was talking to somebody from chef about this last night and he actually brought up an example where because there was because of their governance structure And and some things that had happened and I don't remember exactly how this worked But basically there was a period of time where chef was In their governance structure was being managed by people who are not employees of chef You know, I I don't think that lasted very long and I don't think that was like alarming It just sort of happened and everybody was cool with it and everything was fine And then they had another round of elections later and it kind of shifted back But think about that for a minute the level of trust that chef had And I wish I had more details to share with you and make that that make the story more real But just even hearing at a cursive level the level of trust that chef has for their community to let that happen was amazing You know, that would be great I'm sure they were probably it was probably scary You know, but wow the fact that they they the community was that diverse and let them happen It's really really good stuff So the conflict we go back and forth with this a lot Um another example I wanted to kind of give you was the the age old argument that we have inside red hat Which is talking about, you know, our salespeople look at open source as an obstacle Okay, and and they don't really have anything against open source It's just that if I'm going to be selling you rel Okay, and and we now have direct support of sento s which is a free version of rel Okay, and our salespeople go out into, you know potential customers and and they're they're the potential customer says You know, I don't really need rel because I have sento s And that's the same thing and I know how to use it drives our salespeople crazy Especially when you know in the old days it really drove them crazy because it was like they had no Influence over sento s at all. It was just purely adversarial. Well now sento s is part of our company You know, we we have hired the some of the key maintainers of that project How does that work? Why can't we make this thing go away? You know, why can't we stop what they're doing? Well, we don't really want to because that's not how we do this. It's a nuanced approach salespeople And this is where my physics background comes in so be prepared to go to sleep at any point in this this presentation from now on Salespeople think of this as a sale as they're hitting the basket Actually the original metaphor I have was shooting a gun that I thought nah, I'll stick with sports It's still projectile motion Okay, it's pretty simple you hit a bath you hit a basket. Yes, there's skill involved I'm not an mba player and I will never be confused for an mba player But you know intuitively it's pretty clear that after a little bit of practice Most of us can hit some masters. Maybe not consistently well, but there's no real skill I mean there's no real deep skill there You're going to need practice to do it consistently and do it better But we intuitively because of the way our brains work and because of spatial relationships It's not hard to figure out if I throw the ball in a certain way. It's going to get inside the basket The physics are relatively simple Okay And that is a game that commercial entities not just sales but commercial entities in general are trying to do They're trying to hit that basket And a proprietary software world that's fairly straightforward. I have software it has features x y and z it costs this much You want it or not? You know I'm not a salesperson. So that's really not how it goes But basically that's it and then you know if they make it boom They've hit the basket if they don't make it they try it again or they go off and try to sell it to another customer in open source land When you have an open source project that is out there in the world with you That you're trying to sell into So i'm trying to sell cloud forms in a world where there's managed IQ or i'm trying to sell Open i'm trying to sell susan linux in a world where there's open susan Okay, this is true for everything not just red hat Then it becomes much harder You know it becomes more nuanced actually More challenging This is pool Okay, this is billiards Okay, this is just it's a billiards and this is not even all of the diagrams that I could pull up This is this is this is this is how Velocities and and and and transfer of energy works when you hit A cute, you know a cue ball hits a ball and you're trying to hit the side pocket Okay There I don't have the reduction in friction Things that you get from rolling it across the felt or making a bank shot or I don't have the sweet spot physics in there There's a whole bunch more physics in there. I was like no i'm not going to throw that all in This is one Because billiards is a more nuanced approach It's not shooting the basket. It's great Now you got to think about bank shots Now you got to think about putting english on the ball Now you got to think about all these different things that you have to do to get The objective achieved Okay So you have to think about it in a more open and And and and sort of a smarter way You can't think linear anymore Think about the value. So for us And again to follow back on us the value of centOS is not immediately clear to our salespeople It's an obstacle Except yeah, but now you've got people who are using centOS. They're contributing, you know, they're doing bug fixes for centOS Okay, and and they're they're And even if they are saying now i'm never going to buy rel That could change Because they're using and again. I'm not trying to disparage canonical or or boom to or open suzer or any of those others But let's face it if they're using centOS. They're one step closer to using the enterprise system that we're in You know, and they're not going to jump ship and go over to suzer You know, so let's that sounds terrible and pragmatic, but it's true And they're there You know, but but those those users are contributing something the the centOS community Because they're involved in bug fixes because they're involved in feature A request and things like that that will eventually trickle back all the way upstream to fedora Okay, which is the real upstream for all of these projects So now you've got a circle going it's not just upstream to downstream now. You've got actually a way back up to the upstream You know, it's nuanced okay So this is sort of where we have to kind of take our commercial companies This is where we have to lead them We have to say to them. It's not a straightforward linear progression from build a product sell the product Now it's like we have to we have to get these communities engaged and we have to make sure they're engaged very few Oh, sorry, wait. I had a little bit very few of us I've like built a town Okay, but Hypothetically, let's say we build a town We build roads. We build water pipes. We build electricity. We build, you know, we get everything we've got schools We've got the post office. We've got town hall. We got Town jail, you know, whatever we build all this infrastructure and say that If you look at your community right now, and I'm sure all of us wherever we're from and whatever nationality Would be able to say this is true that You would be able to say in your community. There are a whole bunch of people. They just want to live there Don't bother me I want to live in my house in my apartment. I'm going to pay my taxes I expect the police to show up when I call them. I expect the fire department to show up I expect the water company to fix my waters, etc. Okay That's how most communities are. I'm just going to be a user of this community But I guarantee you you all know people in your community who are much more civic minded And they see something and they say, you know what? I really want a dog park here Or I don't want that factory here. I'm going to sign a petition. I'm going to get people going They're more civic minded Okay, they want to change the community They want to be a part of leading that community and not be politicians necessarily although that is a way to do it too But they want to be involved. They want to be active and we As if commercial entities have got to provide an outlet for being involved in that way Yes, most people will just say I want to just be here hang out and be cool Right, but some people are always going to be wanting to change it and improve it And you may not like the direction of those improvements, but that is sort of how it has to go You don't want to manage community like this Okay, you cannot have hierarchical top-down You know rigid management structures and governance you can do that on the technical side for you know maintaining and approving patches Okay, yeah, because sometimes some it's always going to be somebody who decides I'm not talking about patch maintenance and and technical stuff. I'm talking about governance Okay, you can't do that Even in the linux community, which is regarded as a benevolent dictatorship people have voices people can be heard You can change meanest or evolved mind I ain't saying it's easy But he's open to it because he's smart He you know, he knows he knows a lot about linux and he has very set ideas about some things But I also understand that he's I'm seeing changes mind You can do it okay, so Commercial entities get really over controlling sometimes don't be this person You want to inspire Although please don't do this because this is I saw this and I was like no, this is just too happy You know, whatever, you know if I this is team building and this scares me But it's still true. You want to inspire your community because the difference is you build a community with inspiration And you're going to inspire people to build a community like this This is not uniform. This is not necessarily pretty, but this is a community that works Okay, if you build a community through management And strict controls, this is what you get you get disneyland Disneyland is the most Scrupulously managed community of people on the planet Epcot center very few people know this Epcot center was originally designed to actually be a town It was the most planned community in the world And it would have all these cool future features And yeah, it would have an amusement park in the middle of it to sort of kind of showcase these things and generate some revenue You know Disney is all about building the strongest, you know the the most overly managed community as possible And those of you who are familiar with disney know that there's not a lot of freedom there Yeah, there's a cool mouth, but that's all you got You know, so governance is key The key to the best value For a delivering value For your business and making sure that your community growth is not a grain on your bottom line Is to let it go You know, it's the line from the song if you love something set it free Okay, and that will work and that sounds cliche and corny But that is definitely the most true thing you will have about community In the business world And with that I believe I've rambled on quite long enough And I have a few minutes for questions So If everybody's awake. Yes, mr. Wally I'll repeat your question right Okay, so That and that is value. I I'm I'm not trying to argue that point How is that different from loyalty that I was talking about before or I'm not sure I understand the nuance Okay Okay Okay, I I I believe I and so just to repeat it for the camera in case it didn't come through Mr You know Stephen's talking about how inertia is part of community too and stickiness and people are not the willing the unwillingness to change Is a factor and and actually yeah now I get that because I live in south indiana Where it's ridiculously cold in the winter and I'm on the wrong side of lake michigan And I get dumped with beets of snow feet feet not inches feet of snow Usually every winter and yet a same person will move. I would live here This is really nice here by the way Those of you are local both cali southern california. So I'm rocking it out, you know, but I why do I not move because If I understand steven's point correctly, I have a inertia It's home You know and I'm familiar with it. I'm going to stay there, you know Until I win the power ball and then I'm out of here So we're out of any other questions or comments And comments are very welcome because I'm still this is sort of a We're building this presentation Okay Okay, that's a great question and repeating the question A lot of what I've said has applied to businesses that are directly involved with Open source projects like this and building communities The question was can that also apply to other businesses who might be on the outside of such communities? And how they relate and what's the value to them to interact with communities? I think that People on the out businesses on the outside will get as the same kind of value if not more Being a part of a community even if they're not directly involved in it okay, because You as a user you get a lot of value being part of community you get support You get people to help you you get you get that the brand that being part of a tribe and everything like that Whatever you want to do And I think Outside businesses will get the same value if not more You know if I'm a small business and I'm running You know My sequel Okay as a database, you know database server or something like that or if I'm running, you know An email server or whatever like that it's awesome if I can be a part of a community And just ask for help And not pay anybody for it. It might not be the best help I might not be the it might not be timely Like it's you know if I paid somebody to help me I want an answer and you know I want an answer in an hour because I got to go But if I'm a part if I've been part of this community as a business entity You know and they know I'm part of it They're and they're going to be more willing to help me because then I'm just not coming in and saying hey I need help you know if I've been part of that Then they're going to answer me faster because yeah this small business they they've you know They've hosted our our meetings or something like that You know that there's There's a lot of give and take with that kind of thing And yeah, you can call it being favorite or whatnot, but There's a lot of you scratch my back. I'll scratch yours So that's one benefit I could think of off the top of my head if you are a business participating in a community like that And you use that software You know if you're active and people know you're active they're going to help you a lot faster That's human nature Kind of out of time one more question real quick. I mean Right exactly and and and it gets it gets back into the you know that that argument and and that's a very straightforward Machiavellian argument, you know and I I'm sure there's a more nuanced way of saying it, but I think yeah, I think that In for penny in for a pound is is a valid way of doing this Well, thank you all for your comments and questions in your attendance today. I appreciate it Go eat So can we hear? Yeah, that's sounding pretty okay. So the mic's in a good place. Uh, good Yeah So once it comes up, you can you can hear me at the back. Yeah Not not that we're really worried about the crowd at the back, but yeah All right. Good afternoon everybody and uh, I'd like to introduce our speaker for this session Steven Walley Steven is the distinguished technologist from Hewlett Packard Enterprises He has 25 years of experience working in the open source world and He's also had many different roles. He's been a technical executive a founder a consultant writer international business person Systems developer software construction geek and also a standards diplomat Um, Steven is also a blogger and you can catch his blog once more onto the breach Please help me to welcome Steven Thank you very much folks now, of course after testing the hardware as has gone so well with so many things at this Conference I am now in the position that we do not seem to have things properly on the screen Nope, give me just a second here Literally, we tested this minutes ago. See what it does now Now we're cooking with gas. Thank you Everybody can hear me at the back. Okay Perfect. Thank you. Um So I actually thought about a number of titles that were more interesting than the one that got the talk in I mean Lots of people worry about freeloaders in community now typically that's Companies that are trying to do open source and all those people that aren't paying us money But still freeloaders. I'm going to try and demonstrate to everybody freeloaders means you're doing it right um I also was thinking of what the fuck, you know, because it's one of those things that again, we've been doing this for what 25 years now 30 the free software foundation is 30 years old now And so one of the things I want to try and convey to people though The reality is we've been doing communities Since you had a campfire and I wanted to sit beside it This we we understand how communities inherently work. We are a social species One of the interesting talks yesterday, john o bacon said, you know, he really wants to get us to the point where Nobody has to buy the art of community anymore. But the reality is No, we're actually a bunch of technologists. We aren't a bunch of political science majors we did not stutter study Governance systems. So we're actually in a position where we need the book We do need some instructions and some formats and some templates for how to think about these things But I do maintain That as a social species We've been writing we've been sharing software Since we've been writing software The reality of this is this goes all the way back when I say princeton there von neumann's original work on the original programmable computer in america They were sharing the software. They were writing as quickly with other a couple of other projects that were time sharing at midnight with them Um Share from ibm started in the 50s deacus started in the 60s usnik started in the 70s We've been sharing software since we've written software I remember going to my first deacus and shelling of 75 dollars for a mag tape full of software That people were sharing with me, you know, so the network we had to share in those days was tape-sized packets On a you know, biannual conference kind of bandwidth We did not yet have the internet in the worldwide web But we've shared it for a long time and the reason for that is writing good software is hard work And you all you can almost get a feel for that with github right now, you know, you go to github I can't remember how many million projects have repositories there Of which it's how many thousands are actually important of which how many hundreds are actually really golden places that you want to go Participate in two stories just to again try to set the tone of some of my experience I was at a usniks conference around 1999 And saw a great talk from a guy that was doing an early community around linux video drivers And he made the statement during his talk that there's maybe nine developers on the planet that actually understand what you need to know to deeply work on a linux video driver That did not prevent them from running an open source project and encouraging contributions in but It was a very small number of people and most of them were core contributors already. They were committers. They had right Privilege on the repositories And he said it's orders of magnitude for every thousand people you have using your software You know, maybe if you're lucky a hundred of them may give you some feedback Of which if it's a bug maybe ten of them will give you a patch Of which maybe one of them actually read your contribution guidelines and your coding guidelines And gave you something that was well formed. So it's orders of magnitude That's why I try and go back to this idea that freeloaders are important Uh brian profit who was up on stage just before me gave a great example of a community We all live in towns or neighborhoods And most of us choose to live there and we just live there But some people are civic minded from time to time. You'll want a dog park in the neighborhood And some people are a little more civic minded all the time and they actually run for city council But a lot of us just live there second story I worked for microsoft for five years 1999 to 2004 and I have the privilege of being the first guy to ever ship gpl code in a microsoft based product Which was kind of fascinating we we brought A product that was assembled out of open source before we called it open source Into microsoft we were we were the unix face on microsoft on windows nt This was not the sig win project. This was something that was actually kernel integrated And so it was an interesting time to be at microsoft because this was also the era of the halloween documents And enormous paranoia at microsoft around this free software and open source Stuff But I got the the privilege of working on a project called rotor and this was about 2002 We released A cut through the dot net framework That the runtime all of it so the base class libraries We basically built a reference implementation for what was the ecma standard for the shared source common language infrastructure So they you know, they made they made a lot of noise last year. I think it was about they open source dot net And rotor to be really clear was not an open source license This was for academic research, but it'll give you an idea of a community building We were really really really careful It was about a half a million lines of code about a half a million lines of test suite and test harness We built a single installer that ran on Free bsd mac os 10 and whatever version of windows it was at the time. I think it might have been xp And in doing that we ensured that it was basically with a single command You could set up your environment correctly With a single command you could build the entire software to a known state with a single command you could end up running the entire test harness And you know, you expected to see these three bugs So any developer downloading The gzipped tar archive Could get to a known state in a matter of an hour of three commands that they had to type in The only documentation we provided for them was to literally sit down and in a few screens You know a couple of pages of documentation kind of thing We told them the three commands You know how to set up your environment build it and test it to get to a known state And then the last thing we did was actually give them a rough idea of how the tree hung together So we did not describe the internals of you know, the cli We did not describe the the internals of the base class library or anything else. We just provided the source code 24 hours later We got a righteous contribution a developer who deeply understood the intel chip architecture And deeply understood jit technology Gave us a 15 line patch that gave us a 10 boost in performance That's fantastic And then 24 hours later We got a second righteous patch where they had found a bug They fixed the bug but not just the bug they actually read our You know, they looked enough at the coding style. They fixed the bigger bug not just the spot weld And they even provided a test case to say look look I fixed your bug and here's the test case for the test suite So that is the second righteous bug. So if you do certain things if you take a certain attitude into This is not a would hey come and help. This is actually about We want people to get to a known state quickly You can get amazing things very quickly Now here's the dark side of the story We were not building a community and this is how fast You can pour cold water on that After we got that first righteous contribution 10 boost in performance. We said that's fantastic. Thank you. We're not accepting contributions Because this code base was still tied to the product. This is 2002 halloween documents are still coming out microsoft is still shriekingly paranoid about open source Sorry, uh, legally very conservative in their stance towards open source And so we were not about to allow As brilliant a patch as this was into a core code base that we were still producing for product, you know for product setup so And then the second bug fix came in that you know 24 hours later and we said that's great. Thank you But we're not accepting contributions And we never saw another contribution It was that black and white. We just had to be very public twice And We shot any opportunity and we did it with malice of forethought that was the crying shame of it The core group of us that were part of this project We all deeply believed in free and open source software My background was building products out of it. So I understood that you could do this and it wasn't the most Riskiest thing on the planet or any of those things We cared But we also understood that our marching orders were we are not taking we are not building a community And we are not taking input and the thing was an enormous thousands of programmers in the microsoft.net world still loved the project Because it allowed them to actually go and do some spelunking through the code base to understand exactly how the dot net libraries worked It was an enormous tool for learning. It had its purpose Uh, lots of academic work was done on it for bringing other languages To the dot net world. So again, you know, it was useful, but it definitely wasn't open source And we definitely weren't building community. So it's you can build your own tragedies So the point is really when you when you start with a project You're starting with a community that really is a conversation in code It's not a conversation about code and you may grow to a point where you want lots of other contributions But at the core of it all It's a conversation in code And you've got a core set of commuter committers. There might be one of you. There might be 10 of you Probably not 10, but you know a few of you And what you're really looking for is you are looking to find developers. You want at least to encourage other people to use it Because ultimately what you want is that idea of you want an inbound contribution flow Well, the thing is the question you have to ask yourself is where are you going to find those developers? Well, you know Hanging out a sign that you know if you build it they will come that's probably not true But you know if you actually have a set of users Well, some number of those users when we go back to orders of magnitude some number of those users are going to be developers And if you can get them to a known state quickly, they may even start to do things for their absolute Own selfish interest. Hey, I wanted to you know hack this in. Hey, I found a bug when I was trying to do that Whatever it is They're still not contributing anything to you But at least you've made it easy for them to learn and get to a position where it's a known state for them And so ultimately you need to be growing your user base because you want to be in a position that you can find the developers And then encourage the contributions in So this is really about there are three on ramps that you need to think about if you're trying to turn your project on your repository into a community And that is how are you going to find users? How are you going to make it brain dead easy to install? And in that same way that with rotor it was trivial for a developer to get to a known state They did not have to fuss around figuring out how to build it Um, how are you going to encourage developers? So if they're there and there will be developers in your user base How do you get them? And then how do you encourage them to actually contribute their changes back? Now some developers are absolutely we we live in a social media world All that's taking advantage of is a lot of us really do like contributing to the communities in which we sit So how do you make it easy for people to contribute? So I argue there's a set of practices you have to start looking at and these practices. Um, so I I am a software construction Nazi. I'm sorry I believe there is one true way to build software. I don't care what tools you use But you will have configuration management You will automate your builds and you will automate your test harness Which you have and you will have a review process because these things are important um So it is literally things like um From a from driving user base Did you publish executables? Is the executable environment available? I guess is the better way of saying it because we have very broad communities now We're doing plugins for things and the rest of it, but are the executables actually available? Or are you expecting your users to figure out how to build them? Um, have you automated the install in any way shape or form? Uh, I I love the fact that they made it easy on github to do that kind of home page or to build your own page And you can sit there and a lot of people do you know here type in these five commands folks Or computer people It's a script Don't tell them to type in these five commands tell them to type in this word this simple command It's part of your source tree And it doesn't mean that you're promising that's going to work on every architecture Tell them what architecture it works on tell them to type this command or that command and have it Install in a meaningful way have it in configure in a meaningful way So these are the kinds of things you can do to start to build from a software construction point of view A software discipline point of view you can make it easy to start building a user base There's also some community building things that you need to do Put a license on your work and we're going to talk about that in a minute and I'm going to bore you the tears But it's important to have a license on your work um Put some kind of FAQ in place put some kind of mailing list or some kind of a a place for uh People to either ask simple questions Or to capture those simple first things that everybody got wrong and the script didn't quite cover Those are the kinds of things you need to have in place. I need to make sure everybody understands. I am not a lawyer Lawyers are very special when I say this is all advice for building communities. It's advice It's the best experience. I've learned over 25 years of playing in this space. It's not legal advice If you ever want to see a lawyer turn white Say I think I've been given bad advice Because that They're they speak a different language those words the words that you and I use casually mean different things or more specific things And so this is not legal advice. I am not a lawyer But we're going to take a brief diversion on the licensing so Software is protected by copyright The people that wrote the software own the copyright The fact that you publish the github repository and you don't care what people does doesn't change the fact that software has a copyright and you're the person that needs to Have a say in how they use And so you want to put a license on it Because the license is the way in the open source community or the free software community is however you want to define us Is the way that you tell people what they can do with your software There's a handful of licenses out there Um, it doesn't matter which one I have certain Preferences that I'm not going to share right now But it doesn't matter. There's a handful of good licenses. It you know, it covers the gpl the apache license Free bsd. Sorry the bsd license the mit license The eclipse license if you're working on java in the eclipse community. There's a small number of licenses The free beer license as clever and cute and funny as it is is not helpful So use one of the common licenses and these common licenses these were written by serious lawyers that have been part of this broad community for 30 years now They deeply understand the nuances as lawyers speaking legally to one another In how to think about these concepts for other lawyers to understand No lawyer that you hire is going to do better It's just not going to happen If you care about making money if you think this is an open source project But there may be some corporate implications to it and it may serve you well You probably want to learn a little more about the law There's a really good book out there for developers written by a lawyer that can help you get your head around these things Um, if you care about patents and by care I mean you either think they're a good thing or think they're a terrible thing you care about patents You again want to talk to a lawyer. It's that simple because This is it's a messy space and there's nuances out there that normal people like us don't understand Can I ask you to hold the question? Thank you I They finally on github last thing I want to hammer home on github. They finally made it easy to add a license A couple of years ago. Please do that Because right now the even two years after Uh, they had a github stood up at oscon and said hey, we finally did this There are still millions of repositories out on github that have no license on them And so it's it's ambiguous copyright, you know It's or rather we know exactly who owns it, but the user terms are ambiguous So if you want people to be able to easily use your software put a license on it Okay, so how do you make it easier easy to encourage developers again? There's a number of construction things that ah wrong button Better there's a number of construction things that you can do Um, this is where you start to get into it does the build automate So when I drop the source repository when I clone it to my machine Is it again trivial to turn it into an executable environment? Is it trivial to test it to a known state? These are things you just need to automate you shouldn't be telling them to type these five commands because People fat finger things, especially at two in the morning if you're screwing around with something and you're trying to get it to work So automate as much as possible make it easy From a community perspective Again, this is where you want to actually encourage that discussion in code So you want to you know consider things like possibly a code of conduct Maybe a mission statement if you're so inclined if you're building communities You actually have to think about this stuff The art of community as a book is a wonderful template for the things that you need to think about in this space um Another smaller version so there are two ratios that Define the software space And that is the number of lines of code the average developer can write in the day And the number of bugs per thousand lines There's been lots of reasonable studies on this space. I understand how fraught lines of code is I don't think it's a good measure anymore than probably anybody else in this audience But it is the measure that we have and we can at least say certain things about it We all understand You know ripping out code made it better. I get it. I know I like there's no discussion We can have a lot of lines of code that we all don't Eventually end up in some kind of an agreement, but those two ratios have kind of held since we've written software And they've been measured by people that care about this stuff a number of different ways And those ratios haven't really changed. That's the that's the crying shame of it So every everything we've done in either programming language platforms or software engineering practices Has been about delivering more software and better software in fewer lines of code That's what we've been trying to do for six decades now So it's One of those things that community and product scale And when when you talk about some of these things the big aha for me 30 years ago now was It's not about managing complex code The reason you care about your configuration management and your automation Is you can't scale without it What I mean by that is back in those days the only you know 35 years ago the only people that used configuration management version control and make an automation and things like that These were product companies in an early software industry Because that's the only way you can scale your product to the point that you can support your customers Because when they you know when you get a bug report and you will You need to know what that executable thing is they're running And the only way you can do it is to be able to reliably rebuild that time and again To know exactly what went into the recipe and what came out the other side And the same holds true for communities when you start looking at an expanded community Around an open source project like it's the exact same problem It's how do you reliably rebuild it because you're going to need To be able to fix it you're going to be able to know that the software on your machine is the same stuff running on my machine when we have an argument over the best way to fix it or the best way to fix that bug Or the way to enhance it with something So that's the only way you can actually reach the scale you need is to have that kind of reliability The one other observation I'd offer you is is lenis's law It's actually about reviewing And there's all All bugs are shallow with given enough eyes That wasn't talking about debugging because of some bug that was found if that was true We would not have had all the problems with things like ssl and the rest of it over the last few years No, that was a the failure in community there was the community was too small the review culture wasn't there Lenis's law applies all bugs are shallow you discover them before they become bugs in the executable thing that's distributed That's the joy of it. There's a number of papers out there that have proven time and again. You'll always find more bugs in review Then you do in test And that that's kind of a startling thing But you just do and it doesn't matter they've even the one experiment They even varied it with things like was was this formal, you know six people in a room together when they all had roles Or it was a desk check. No, you even with a desk check by you know a co-worker You will still find more bugs in review than you ever do And that's that's the strength of some of these communities, especially when you see one of the things that open stack gets right And for anybody that's ever heard me rant against open stack, you'll know that there's not many things they get right But one of the things they get right Is they built a culture of review Into the front end of it and so that's huge because you're finding some of these problems early So lastly, how do you actually go hunting for contributors? So this is that space where again, you want to make you want to publish your coding guidelines You want to tell people what to contribute and if you're looking for people to help with the docs All of these things apply, you know You need to be able to get to a place where you have a known set of docs Any person that wants to participate can get to that known state But you're making it easy for folks to get there when I talk about kind of a second stage automation I'm talking about that kind of thing of when you see some of the very modern Projects right now go into this whole continuous integration continuous deployment tool chain So it's no longer kind of that first stage of automation of here's the make file We're now into a second stage of automation where we're making it trivial for you to test your software In a no one big environment rather than just doing it in your own small environment That is just a side story. That is one of the classic things. I heard at the open stack summit in tokyo Some people are starting to question maybe the idea of allowing a whole lot of developers To test their clouds on dev stack on their own machine Isn't the best way to test for something that's supposed to be managing large multi machine clouds You know, it's it's that kind of thing of how do you how do you do this thing? Well Again, there's some community aspects to those kinds of things When we start looking at these things you can you know, you get into event management You get into governance discussions because at this point You're probably getting to the point where you want to start laying some of the foundations of this early Now no point am I suggesting that you need to get all this stuff done on day one But you do kind of need to walk through it in order because that's how you're going to build these pipelines of people Because really what you want to get to If you want to start with your project and ultimately, you know You want to advance from committers into contributors into a community of folks around it And you want to ultimately build an ecosystem Now there are projects that don't there are projects that are very happy to be small But a lot of people really do want to foster that community that Growth of their community and that ecosystem is a rich ecosystem. You know, there's products based on this project There's services around it. There's books. There's training. There's consultants. It's a big thing. There's certification projects So that brings up a problem Because if we're going to get to an ecosystem with projects and services products rather and services and all that stuff what happens Is you now have corporate contributors and you need to figure out what to do with them because these are different animals And there's a fundamental thing that companies need that most projects in their infancy don't have And that's where you get into there's a whole pile of things that you need to do Around IP management because that's the one thing they care about So again the license your first license was a social contract That's how you and if there was any other core committers when you started that that was the agreement you had for the the kind of the The governance you've wanted to put into but it was more of a gentleman's agreement This was your early community and how you wanted to see it grow the fundamental things you believed in the social contract in an early community Now we're talking about a license from the perspective and i'm not suggesting you change your license. I'm not suggesting you change Um the values that you consider important. Just understand that corporations have opinions about licenses Now again, if you stick with that small core group, you've got lots of flexibility and Half of them were written by corporate lawyers and half of them were written by academic lawyers So you're mostly protected in that space already They they want to know about IP tracking They want to know about provenance They want to understand what your dependency graph looks like so there's a pile of stuff there That if you've been doing things well You've already you've already probably put it in place But just understand that this is a different set of practices for a different reason because companies care about this other thing Now there's a benefit here. There's an enormous benefit If you put these practices in place You can see an order of magnitude growth Because now you have dedicated resource Dedicated investment from corporate people that want to use this software in their products and services And you're not selling out. I have not in any way suggested That you want to manage differently change your committer structure. Nothing like that But it's it's an it's an interesting place Um the way a lot of people talk about this Is from the perspective of foundations I was technical director at the otter curve foundation For three years uh up to a couple of years ago It was an interesting foundation because this was the foundation that microsoft was the primary Founding sponsor for It was a proper 501c6 so a trade organization um There were some awkward problems with it Because they kind of saw it as well we're microsoft and we're going to tell you how to do open source And you can imagine the community reaction to that and I was certainly on I I walked into it with eyes wide open But it was it was always that kind of right you're going to tell us how to do open source This will be great But there was a really interesting thing that coincided with that uh a guy named hennrick ingo Uh, he's in finland. He worked for mysql. He currently works for mongo db He was I think he ran maria db for a while As the ceo when monti was trying to build a corporation around it He did a study And it was a really interesting study and what he demonstrated and he pulled all the numbers and he's a good engineer So he he listed where all of his data came from all the assumptions he made and he was careful not to suggest this was causative but The number crunch he did demonstrated That the nine largest most vibrant open source communities are all hubbed in foundations The tenth is an order of magnitude smaller So while he's not claiming causality here He's carefully suggesting that there's clearly something going on here. There is a correlation And that that becomes a really interesting thing. And so My executive director and I were stuck in there and we were kind what is the theory of foundations? And the one thing that every foundation has to get right in the open source space And this goes all the way back to the free software foundation is this idea of neutrality a level playing field for all of those corporate participants That care about you know, I don't want to be building your product kind of thing Or I don't want you controlling my product kind of thing all those go away if you have a foundation wrapped around it properly Properly is an important word. I have other rants that I can do about foundations Um, and that is where you need this idea of very bright lines between what is the project and what is that community? And what are the products over there that come out of these projects? That have customers because those are different things And so here is a large collection of practices You do need to think your way through this if you want to build a large successful community Like I said, these are not everything that has to be done on day one Um, I do not presume That anybody could do that. I mean, that's that's one of those things We have so much architecture in the way you never get to actually doing anything interesting in the project um but There is this set of patterns and they really do Fall around these ideas of driving you know encouraging users encouraging developers encourage your contributors And then what do you have to do to do that next step up when corporate players show up? Because if you're doing the first things well Corporations will show up there'll be developers that might have been part of your community who work for companies that go You know, we could do this in a product or we could build this into our product or we can contribute this You know, we can build a service around this for our customers And so having that is that additional pattern. How do you think about those things? Because ultimately you want to take this graph and just like I was trying to do with the arrow I keep trying to I keep trying different visuals. I'm looking for input Um, you want it you want it to grow into this space You want this to be a rich ecosystem where there's lots of stuff happening In that ecosystem and lots of people want to participate for their own selfish reasons And their own needs and you can support all of those things from that core project community Going back to this diagram So this is an interesting diagram because the really terrible thing is once you get corporate contributors Once you have products based on your community once there's services out there once there's you know books and training and everything else You end up with customers But here again is this problem. They're not your customers You're the community you care about the project. That's what you started and you might have a vibrant Yeah, your lenis torbles and your linux has taken over the world But lenis is still lenis and he still runs a community. That's a really interesting thing He never gave up the trademark. He never gave up the source copyright And he still runs that community in the way that he defines us best for the community overall And then you step outside of that and you look at this massive ecosystem of other things around it And so what you really want to be careful of is getting sucked into any discussion about customers Because in the same way that projects are not products Your community are not customers And you got to be really careful and again i'm a big believer in bright lines because when you talk about these things We can have a discussion around Community building and there are excellent excellent resources out there There really are we like I said, we've known how to build communities since you had a campfire and I wanted to sit beside it absolutely And the thing is in the product space There are excellent excellent You know you can go get an mba. There are excellent resources in how to build and drive Products market them all the product management work that goes with it But these things there's a really bright line between the way those things work And there's a bright line between the engineering work that has to happen There's a bright line between how you have a conversation with your community versus how a company might have a conversation with a customer if you're building if so i've been talking as if you were a you know an independent Community project that has grown up into this space Everything i've said actually applies If you're a company trying to build that community Because you got to walk through all the same discussions But the the trap that so many companies fall into Is they make it that customer and community conversation they think is one Marketing folks think for our community that's kind of like a big part of the funnel and it's not And the engineering resource engineering teams if they get this thing wrong if they don't have good open source experience They think this is great. The community is going to be writing all kinds of things for us And again, you got to that's not actually the way it works. So you need to be able to get those conversations Really crisp with very bright lines between them and everybody inside the company needs to understand what those things look like And that's the same thing that you have to do if you are an independent community You need really bright lines for how you talk about these things So i've got another talk that i'll do next year kind of thing part two of the talk is really how to think about those things with bright lines Um, i'm actually finished at this point and i am happy to take questions comments Sir, uh, so i'm just going to repeat this for the the video. Um, the statement is that i was being a little soft on patents And that you do need to think through them. Um I actually agree with you very much. Uh, what what always Fascinated me the thing that is different about the linux foundation Because linus wasn't giving up control the actual ip was unlike the apache software foundation or the free software foundation Or the the eclipse foundation. I guess it postates though When they set up the osdl when the big vendors got involved They had to ring fence the ip And they ring fenced it through exactly those kinds of things. So they they immediately hired linus and andrew to make sure that they were individually protected In the same way that the eclipse foundation and the apache foundation protect their community contributors They turned around and they started there was some of the early work that the osdl did was actually really pretty interesting stuff It predates the open innovation network and their patent pools and those are The oin is a fabulous example of what can be done in this broad community of open source related companies and open source related projects It's definitely something we have to consider My personal view I don't have an opinion on patents good or bad per se, but like you said patents have to be managed It's a reality in the world we live in Go ahead absolutely patent rules do exist the The the interesting thing the way I've tried to explain it to people because I Historically I did some consulting with small businesses that were doing open source related things is patents are tickets To negotiations They are and that and until you're big enough to be making money kind of thing and that Again, that's where the foundation if you if you're growing into a space where multiple companies are involved A foundation can actually help you think your way through some of those things um, if you're A company doing these kinds of things and you hopefully already have Council on on board that is advising you and it's sometimes painful I think a company that does interesting things in the space is red hat Um, I think they found a strong balance for how they want to protect their community and protect the company at the same time um, so But thank you Other comments questions excellent question How as you go through the transition From a community that was kind of small and a little more organic into that space where Corporate contributions are are inbound. How do you manage that growth to not freak out your core community? So there's a couple of couple of things there that I that we've all seen so if you look at linux No, linus really did run it the way linus wanted to run it and so corporate contributors showing up um, you know, we were lucky he was either young enough and arrogant enough or Finish enough and strong willed enough. I don't know. I don't care He continued to evolve that community the way he saw fit So have it knowing knowing yourself so to speak having that that's that strength and belief And maybe having a gpl license on your software provides a very bright line For how con how corporations have to think about you I think that was an enormous strength of that community And so that was interesting when when you go back to henrex nine projects which dates up to five years ago He and I started a different discussion just recently so My cto martin fink Cto of all of hula packard enterprise got up on stage at linux con and dublin And his keynote was really about Everybody has assumed for a long time that the apache software license was the business friendly license But that we shouldn't Not think about the the gpl as the other business friendly license There's lots of things to do with that and that's where I think the gpl provides a very bright line and simplifies that discussion As companies come to play because they're going to be very careful, but the other thing so when you go back to henrex nine projects Six of them are gpl Two of them are weak copy left There's only one of them that's apache licensed And it happens to be the apache group So I had the discussion with the president of the apache foundation because it was like What what's different there? White and the best conclusion we had that goes back to Your question about how do you scale that community and not freak out the the original community? Apache has the incubator and the incubation process is 12 to 24 months that you have to come through the incubation process to be declared a full apache project Now while it's always been stated from the outside as this is We're we're teaching you how to be good community Developers working together in the apache way. So it's always about the apache way. It's always about the software engineering The reality of it one of the net benefits of it as well is it drives all commercial discussion out of the room It builds that very bright line around what is the project and how does that community interact around what good things? and Yeah, what you do in your project or what what you do in your products and your services is your business And so it's building that bright line and I think it's so you can see a number of ways of doing it But do it with the gpl do it do it with a strong copy left Just declare the gpl do it with a strong copy left or provide a process that drives that conversation out of the room Which requires either a process like the apache way The eclipse way is and you know they do a similar kind of thing in the eclipse foundation Or lenis was simply strong enough as That benevolent dictator to actually lay down the law as to how this would happen And because he never gave up the ip at the end of the day, you know, it was his way of the highway That's a good question because those those are the kinds of things we see regularly Other questions, uh, what about forks? That's a great question as well. So what about forks? Um So as a company who was who did not want to live on a fork in the late 90s Um, I worked very hard even though we were pulling software Out of the open source community to ensure that all of our engineering contribution went back upstream So again, I was very crisp in what was my product. How did I package it? How do I license it? What what do I do for my customers? But I don't want to eat the engineering cost of a fork as a company Because that'll only get more and more and more expensive over time. So you want to contribute everything back upstream if it's not your project um so big believer in not living on a fork big believer in There is one code base for the project if you're a corporate player the thing that destroyed the simbian foundation Utterly destroyed it is when they flipped it from Symbian into the symbion foundation The code was still being written by nokia. They were first amongst equals and they published into the community for the rest of the community of vendors to pull from So the vendors left That's simple So you there needs to be that one true place no forks there was there shouldn't there should not have been the nokia symbion Release plus everybody else's um And then in the reverse we've we've seen the occasional catastrophic fork Gcc versus egcs back in the late 90s that was a huge war over the future of the gcc compiler suite And that lasted god, I think better part of 14 to 16 months. I mean it wasn't over quickly that argument There was two parallel growing uh trees of code And they finally understood themselves the engineering economics as open source projects Was ultimately to knit it back together. So there was one true gcc You can look at things like Jenkins versus Hudson Hudson thank you That again I to me that's the kind of failure you saw with symbion that that that's oracle Just upsetting the planet and not caring now in that case they could in the symbion case it killed them But nobody talks about Hudson anymore And that that was a case where I think it was an interest a really interesting case that as strong as the apache way is There are still failures along the way Statistically not that many Now when you look at the the growing number of projects that continue to graduate the incubator um You know the entire big data base big data space based around hadoop All the all the excellent work that continues to come out of apache. It's great. But no forks are not good So that was the long answer forks are not good is the short answer Doesn't and it doesn't matter whether you're a corporation Or a community you do not want to have forks happening So in terms of the different approaches just to repeat the question the different approaches to dealing with patents in the community Different communities handle it in different ways Patents tend to be places where Corporations care deeply Microsoft screams about patents all the time and what you guys have to remember Is you might be the generation that remembers microsoft wielding patents against linux. I have more gray hair I remember when ibm screwed over microsoft in their initial patent discussion So microsoft suddenly became very adamant about patents that were important um I've seen a brilliant Journalist article on the first time ibm approached sun As well and back in the late 80s and kind of kicked sun around for a while until sun signed the cross license deal So patents tend to be a big company thing Um, I like the approach that oin has of of defensive patent pools I like the approach that red hat finally as red hat grew I mean, it's now a two billion dollar company by revenue and a ten billion by valuation um, you know It's they have had to think through a patent strategy as a company That does not upset their community And there's some you know, there's lots of hardcore anti patent anti ip folks You know, you you almost have the ip anarchists. There's there's always one or two in a room that are way out on the fringe and That community the linux community still functions because red hat has found that balancing line Um, it's it's a rich enough community to support all those points of view. So it's I I don't think there's ever one right way Because it's your balance if you were balancing your own community if you're growing it well You're going to have everybody from you know extreme anti patent folks All the way through to if you're very successful and you have corporate players people that want to understand your patents um, that the really uh interesting thing that I Have not spoken to enough lawyers to understand Why more companies aren't playing this game? um Web them when google did web them They did it under the berkeley license Because the berkeley license is a well understood open source license That has zero discussion and zero ambiguity about patents And then for that project they said by the way we have a different license for all the all all the other ip around it And they use that as the way to clarify their state around patents with respect to this space I thought that was a really interesting way to handle it as a corporation Because they weren't you know it left them a certain amount of freedom of play But it also allowed them to very be very clear with their community of exactly what they thought of patents with respect to this software so I wish there was a clean right answer And it's I for a long time there. There's something called a provisional patent, which is very cheap. You don't have to write all the claims It's like 130 dollars or 150 dollars to file. I always thought that would be great You file a whole lot of cheap provisionals. They last for 12 months and then you lapse them So you're just salting the fields around you Because you've now put a stake in the ground that said, you know, this is this is no longer an invention I have talked about it in this And you didn't have to go through all the legal expense of getting you know really well crafted Well, it turns out the lawyers have caught on to this idea and the way they want to do Now is they still want to charge you two or three thousand dollars for the patent search And you know, they still want to if they're not going to write full claims for you They at least want to start to draft them in a little bit for you because obviously you're going to want to turn that provisional into a real patent application. So it's Again, I am not a lawyer. You do not want patent advice from me But I've seen some interesting plays from different communities trying to tackle things And trying and trying to be honest with their community. I just want to expressly say I have no No opinion on anything going on in the Ubuntu commercial world right now Other questions at the back. Uh, what do we what do I think about contributor agreements? Um I'm going to do the two step. You know that because I've two steps along here with everything to do with legal all along Yeah The time when they become important Important is again that crossover line when suddenly corporations want to understand provenance And I think there's an argument to be made that if you want to keep the bar very low And you want some confidence That you could grow into that space Is just do the dco for now if you're starting small and you've got you're using the developer certificate of origination You at least have you can finger point back to a list of people In the contribution stream that you've been committing into the core code base for however long that's been At the demonstration at look, we do kind of know who this came from and we do have kind of at least a first pass statement That this is interesting. Um, and you know that kind of thing can work Because the worst case that we ever saw uh in something that became big was what happened with apache You know the early days of apache, nobody knew who was committing what in the where kind of thing It wasn't complete wild west. There were good engineers doing good things, but nobody was tracking it And again, you got to remember it was only 1980 that we started applying copyright to you know to software And so it became a space where when they formed the apache software foundation Because again, there was there was big companies that had big big lawyers that were concerned and they wanted to use it in products That's why the the the play they did was more of a well, it's an aggregate of software Now and so we don't really worry about where it came from in the early days Um, so you you can deal with the ambiguity, but I think being crisp up front Just doing a dco on the way by keeps it a very low barrier to entry Uh, it's if you go look up developer certificate of Origin I believe is what it stands for just google it. It'll take you right to the page That's what they use in the linux world Um, now I will push back. I used to get into a debate with richard fontana Uh, who was legal counsel at red hat and he used to say no, you don't need contributor license agreements because gosh The dco works great on linux. Nobody's challenging us Okay, linux is a snowflake linux is special There is so much big money floating around linux that we all saw what happened when somebody tried to challenge that space once before You know at the end of the day Very large companies with very deep pockets and very large benches of lawyers are just going to ride that storm out for the rest of us So linux gets a bit of a buy But I think it it is a responsible way to think about it if you were a small project and you were just You know moving along you you will always be able to go back to say that developer with that name Was responsible for that contribution see it's right here in my You know versioning system and so I can even point to specific versions if we ever had to rip it out for god forbid Whatever reason you know exactly when they hit the code base and you can go back It's it's it's a useful way to track it I've heard lots of folks have opinions that it slows things down. I think that depends on how awkward you make it You know there's To me if you're if you're staying small and moving fast, I'd go with a dco Is there a way to slow down corporate involvement? That's a great question What so and they might even be delivered in that steam rolling? Not mentioning open stack by name um Fred Brooks And the idea that you know adding more People to the discussion building a bigger team slows things down because there's more lines of communication So I saw brooks law applied as a weapon In the standards wars back in the 90s around unix You see if you add lines of communication It slows things down And there were certain big vendors that really did not want certain things in the unix spec space happening too quickly So they just suggest yet another liaison point to this other group over there And all of a sudden you keep adding these They almost ground us to a halt. So that was how big corporations were definitely slowing down a small group of engineers trying to build some interesting standards um There is that problem with Too many cooks in the kitchen Slowing things down. So I think a way i'm making this up as I go. I've never been asked that it is a great question um I would think about How you maintain your governance before you get into that space if you if you've got to a point where you actually have a few committers Then you should have a way in terms of encouraging other contributors That there should be some statement somewhere on how do you become a committer? How do you get right privilege on? thanks And that way you can start to think about ways early That you might have to deal with the problem of a whole lot of contributions flowing in or people demanding that we're giving you all these contributions Why aren't you patching them faster or or or If you can be really crisp in how you become a committer Then That gives you if you have that conversation before you need to have it You're in a place that you can feel confident either telling them why they don't get a committer just because they've sent all this code Or while maybe as a collection of committers You think they do deserve because you've been accepting all of those contributions They've been clean and strong and this is the person identified that was sending them all in So that's the way I try and think about that problem Uh, what have I seen in the open source hardware space? Um, I am interested in that space and I've seen nothing because I haven't gone on a looked yet So I'm afraid I I I am fascinated. I actually just started conversations here this week To start to go learn a little more about that. So it's to me. It's a really interesting space. I know that The kinds of places that I suspect there's going to be interesting questions is Historically all the body a lot that came out in there It was very patent heavy because we're talking about things that were you know They they were atoms in the real world not bits in the software world And if you look at even some of the history of companies like IBM intel healer packard enterprise You know my employer We're we're hardware companies. We have a very different way of thinking about hardware patents versus software patents If you one of the fun things if you go back all the way through the history of the eclipse license When it used to be the common public license when it used to be the IBM public license, there is a small phrase Buried in there that has never changed even though it was You know the IBM license and it was a common public license as administered by IBM And it's now the eclipse license as administered by the eclipse foundation But that phrase In the ip section on patents said this does not apply to hardware patents They were very clear there and it's one of those things where I think it that's going to be an area that is hugely Interesting and different from what we've wrestled with in software And it's it's also the manufacturing costs even though things are plummeting With some of the new technologies we have in terms of 3d printing and just the ability to build physical things and prototype physical things for Normal people like us without spinning up entire you know supply chains There's still going to be a lot of legal opinion And and legal can you know conservative opinion left over from the days where no that it costs real money Just you know build a fab Kind of thing so that it's it's going to be interesting in that space, but that's a guess That I have seen nothing. I've looked at nothing There's no other questions. I think we've hit time Thank you very much folks All right Okay, good afternoon everybody. We're going to start our session I'd like to introduce Deb Nicholson Deb has an incredibly unique background Where she combines both her interest in technology and social justice She's been a free speech advocate economic justice organizer and civil liberties defender But surprisingly, she's also been involved with massachusetts politics for 15 years And then became involved with the free Software movement She's a community outreach director at the open invention network and the community manager at media goblin And she's also serving on the board of open hatch Which is a nonprofit dedicated to matching perspective free software contributors with communities and tools and education Please help me do uh welcome Thanks All right, uh, so i'm going to talk about how to use more free and open source software at work Uh in particular how to approach the legal department So, um How many of you are using some free software at work? How about 100 free software at work? Okay, there could always be more, right? So This is you know that sometimes when you go upstairs and you're like we should be doing this They they imagine this is what's going on in it And so we're gonna we're gonna give you i'm gonna i'm gonna talk about some of the ways you can figure out Where the lawyers and management is coming from so that you can answer their To their concerns and hopefully win them over to more free software at your workplace So this is just kind of like what i'm going to go through. Uh, why uh, what are some of the great arguments for using more free software at work? Uh, it's how lawyers and coders are different There's lots and lots of ways We'll focus on the important ones and uh, and then a little bit of patents and copyrights Which are some of the things that lawyers tend to worry about when you say Oh, I want to use this thing i found on the internet And then a little bit of a content tour of what's out there for the particular legal issues And then we'll finish with some strategies and resources Uh, so why is free and open source software good for your workplace? uh, of course Most of you have been here all weekend so you probably know that not having to write stuff from scratch is pretty great It's a time saver and um You know being able to change things when you want to change them is also pretty great But uh communal debugging. I don't know if people can see these slides because of all the fluorescent, but um They're all pretty much all going to be monkeys to see you know Uh, so uh, the communal debugging makes for a better code. It makes more robust code. Hopefully, uh, it makes some of the security vulnerabilities shallower Of course people have to be looking at it, but um, you know for uh for Projects that have a lot of people looking at them and working on them The communal debugging is a huge help as opposed to having like one person in your organization Spend like five minutes a week looking at something So uh trust and transparency. This is pretty big. Um If you are in any kind of an industry where you would like for your customers to trust you at all Maybe with their safety or or anything important Uh having the ability to say like oh, you can always look at our source code and see what we're up to Or uh, we're actually never going to do that crappy vendor lock-in thing that other databases might do It may say that easier to sell to folks like hey try this for a year And then if you don't we won't be holding all of your data hostage You can actually just move on and you might decide you like us in that year So trust and transparency is a great selling point Um for businesses, but also if your workplace is a school or a government agency or anything like that Where you feel like you have a responsibility to your constituency to be transparent and open Free software is an obvious choice there too Uh so uh more capacity so for uh large projects like open stack are people familiar with open stack kind of okay so open stack is uh a huge Linux-based cloud that's like a very very top level But it's a huge huge project and it's worked on by a lot of different companies Each of those individual companies that they were to work on it themselves Would have to hire lots more people just to work on that one thing So instead you have like a handful of people working in hp a handful of people working in red hat A handful of people working at a couple other companies and they are able to make this Much bigger project that everyone can share in The other thing that's great about that is that they're not even all using that for the exact same things So something that you're not doing with your business today is something that One of your partners in an open source project might already be doing so you might think like oh One day we're going to have to translate everything into russian But maybe by the time you get there one of your partners has already done it and you're like oh sweet So um this enables you to do more these grand shared projects give you more capacity So you don't have to hire one of everyone in every single company So so this is one of those things where you want to say um To you know to your your boss may be like oh Yeah, those things are all great and we all know that like freedom also means like a better world and and happier employers But uh your company is still trying to run a business So you when you're talking to your boss depending on who your boss is About using more free open source software at your workplace You want to lead with some of the amazing business ROI value ads as opposed to the You know freedom man Kind of things uh because at the end of the day they still have to write paychecks and expect them to not bounce so um That's sort of a little bit of the Occulturation so there might there might be other talks where we can talk about the freedom and the you know If the idea of a proprietary singularity keeps you up at night and stuff like that But this is probably not good things to bring into your manager or to your legal team Maybe over beers some other day Right so quarters and lawyers. They are not the same right so when we say free They're thinking oh um Has anyone ever had like a roommate bring a piece of furniture in off the sidewalk Yeah, okay, so I have I wanted those roommates who was like it was free I'm like oh sweet and then like three days later I realized we had some little fleas in our house and I was like I think I I don't think this was free um This is uh, this is definitely not flea free So we had to wash every piece of fabric in the entire house coat our old hardwood floors with like a Hyper solution of borax to dry out all the eggs The cat was very very very unhappy for a long time So when you tell lawyers like it's free. I found it on the internet. This is what they're hearing Right so you want to figure out how you can allay those fears because Just because you didn't know about it doesn't mean that it doesn't exist. So You know, there are legal ramifications for using code that if you're not aware of them They can still trip you up. So just keeping your eyes set is no good In particular, and I'm going to go into more detail a little bit later on the pattern landscape is like a Cartoon of a million banana peels. So So this is another thing that when you say like, oh, I found it on the internet and they're like, yeah People always ask me like, do you know if the software that I'm writing is infringing on patents? And I say yes, and they're like, wait, I didn't finish. I didn't tell you what kind of software I'm writing Or what it does and I'm like, no, but the answer is already. Yes, you're writing software It is almost certainly infringing a patent and lawyers know this too. So So they tend to be a little bit more classes so in In our line of work, like if you do something creative you do outreach or you write code or those types of things Permission versus forgiveness like if you want to stay up, you know pretty late and break things or clean out the work fridge without telling anyone or Any number of other things where you're like, oh, I I broke it. Maybe you sandboxed it before you broke it. Maybe you didn't This tends to eventually be okay for code. It's not okay for legal issues And so that's that's kind of the different direction. It's like, oh, well, we'll just try it and then we'll work it out It could be a very expensive mistake to screw up on legal issues So lawyers as a as a group There may be exceptions tend to have extreme caution as their default setting Lawyers never get fired for saying no So that is like their first answer is going to always be no or like could you not? Um, and that's it's uh, it's just sort of the way that they are So you you go to them you ask permission to do something if they say yes and something happens Then they go back to the lawyer and they're like you told us we could do this They never want to be in that position So you when you when you when you approach lawyers with something that has some inherent risk or has some potential risk You need to be aware of that setting um That said uh understanding world of patents and copyrights and understanding where there's a reasonable around a risk and where there's An unreasonable around of risk is going to help you out So this is we're going to go into a little bit of how patents and copyrights work with uh, The great news is is that we're uh, we're not doing new uncharted territory here The uh, free software has been around for 30 years There are hundreds maybe thousands of companies and projects that are have employees and everything using free software So, uh, we're not we're not going off into uncharted territory here If it was um an actual physical space, it would actually be a little crowded these days Fortunately, it's a virtual space. So, uh, you don't have to worry about people blocking your view of the water so, um so first off all Software licenses rest on top of copyright. Do we have lawyers in the room? I'm just curious No, okay lawyer free zone So this is I am not a lawyer and this is of course not legal advice, but also this is um, this is sort of quick and dirty not not legal advice, but uh All free software licenses rest on top of default copyright copyright is the all rights reserved. Uh, it's um It's invoked as soon as you write something unless you say otherwise So it's intended to protect people who have written something That's uh, so because we talk about writing code is how we sort of ended up expanding the idea of copyright to software code Because it is a written thing Uh, but copyright also covers books. It covers sheet music. It covers maps And a number of other things that you would write Play plays musical scores that type of thing So in order to use a free software license, you must first invoke default copyright All of those three software licenses are additional permissions and sometimes responsibilities on top of default copyright Most notably copied left. So So Then the question people always want to ask is what license did we choose? Of course that depends on what you're doing But these are some of the popular ones that are more well known And better understood and have a lot of other projects using them And and can be compatible with a lot of the other code that's out there So if you go to github and you're looking at like you're going to find Probably not a gpl so often, but the rest of these you'll find a lot of code under there um So uh apache and free bsd are what you would call permissive. They don't um, it's like do whatever you like with the code Uh, don't sew us for patent infringement is the additional responsibility in the apache current version of the apache license And then free bsd doesn't have any other additional Permissive some versions actually say that you have to attribute us, but that's you know pretty small It just means in the code and it's usually part of the license tpl three and gpl two I didn't want to overload you guys with logos, but uh are both what we call copy left So those are you can do whatever you like with the code, but if you create a modified Uh, some of my code plus some of your code and then put it out into the world you have to provide source And uh, you you often hear like oh well, you can't make any money with a copy left license, but um I know github just hired like three more people so they seem to be doing okay Uh and red hat from all accounts is doing okay. I think we're up against like Two maybe four of their employees on this slot alone So um, so you can make you can you can base a business on copy left code But you need to understand what kind of a business you're going to run So and then the lgpl and the agpl are variations of the gpl the lgpl is sort of uh a relaxed idea of like what can be linked to gpl code And then the agpl is sort of an enhanced version of what can be linked People often call it gpl for the web So those are all of the different licenses so um if you're talking to a lawyer who knows nothing at all About free software licenses You're going to want to get them to understand that there's a pretty wide spectrum of uh free software licenses So some of the things they may have heard is like oh, they're all this or they're all that or um, you know They were written by hippies that might be true, but uh Depending on which license we're talking about uh what they have heard may not actually line up with the license You're asking them for permission to use So that's important to keep in mind and i'm going to share some links and resources at the end where they can go and educate themselves Um and you too so what is your business model? It depends a lot on like which Which thing you're actually trying to make money on? If you're selling phones it may not matter at all what is what kind of licenses on the software Because you're making your money off the hardware if you're selling services that Just happen to include some amount of software it also may not matter at all what kind of licenses on your software So you want to understand what your business model is if you're in a startup situation And you have a lawyer and please don't tell me you're going to wait a couple years until you get a lawyer It's a terrible idea But if you're in a startup situation and you're starting to talk to a lawyer and you're thinking about business models You want to include them in that conversation from the beginning? Because the license that you choose has a direct impact on the way it works to do business model So let's take a look at two different companies. I think you guys refer to these folks, right? And actually both of these use a lot of gpl2 code So red hat They they do sell service and expertise, but that's not their main thing What they've done is they've built a huge platform that people use and really like to use and they have a community that helps build that platform and What they sell is I'm going to figure out which iteration of that platform makes the most sense for your infrastructure and your business and what you're doing and I'm going to make sure that we made the nicest user interfaces have done all the quality assurance and have made sure that You're not getting like a old weird buggy version of whatever it is that you might get from You know the open internet So what they do is they do a lot of work to clean up the You know big pile of available platform code out there and then sell you the proper slice of it for what you're doing So for them if you can have access to all the code, that's fine It's the combination of the code that they offer that is what they're monetizing So that's a really different business model than hardware or just selling a service So that means that they they're like, yeah copy left bring it on Community like they cannot with it all the code. We don't care. It's it works out great for them And like I said, I think I think they're doing okay Twitter has a different business model. They are a web service, of course so, you know, they make some money on Promoted tweets, which is sort of a fancy type of advertising and I'm sure they aggregate some data but one of the things that is really important for them as far as Getting people to promote the use of their site as they have an open api So it's this grand social experiment that they happen to be providing the infrastructure for But they're not doing all of the research and nuts and bolts stuff on it They're opening the api so that other people can and then of course they're also like if you're writing an academic paper on twitter It's you cannot buy that kind of advertising. They've basically like here use all of our information and then just you know Uh, you're gonna have to say you got it from twitter because there is no other micro blog that has that volume of data so for them, uh, you know, I think I mean, I don't want to make this dumb sound too Machiavellian, but uh I think that's a huge part of it of making the you know The api open is that they have people talking about and thinking about and looking at their data all the time Uh from the inside, I think it also makes their developers a lot happier They like working on code that might be used somewhere else at some other point in time So, you know, they have happier developers. They have people that are writing academic papers about their service So using a copy-left license for them makes a lot of sense so And I don't know if I don't think anyone is really successfully forked twitter So the that I mean they could try but they'd have to find another Neese and user base and other kind of thing, you know, so Uh, so those are to come needs that use copy left licenses that you know, I would say are doing okay Uh Regardless of what your business model is, uh, you do not want to be sued for patent infringement, right? No one no one enjoys that And there are a few licenses that specifically mentioned, uh patents Gpl version three in a patchy version two both say like hey, you can use all this code But then don't come back and sue us for patent infringement for the functionality contained within the code that we're giving your license to which is um It's kind of it seems like a weird thing that you would have to say it'd be like Oh here you can have like half my sandwich But then don't be a jerk because I put onions in it or something, you know Maybe that I'm not sure if that metaphor follows But it's like I'm giving you a thing don't be a jerk That's you know to put it more plainly so the Apache license and um gpl v3 both say that like hey You can use all this code, but don't be a jerk on the on the stuff that is possibly patent encumbered in here So, uh, and that's tricky because certain companies are like We really like being a jerk with our patents. I mean not like when we're giving talks at conferences But like quietly with letters that we send in the middle of the night. So, uh, so, you know, that's Hopefully your company is not a jerk so So I would say it if you're going to use a license a patchy permissive gpl v3 is copy left They both have patent protection clause. So that's the beginning of how you kind of Start to grapple with the the landmine of patent infringement So we are going to talk more about patents Does anyone um Everyone's with me so far excellent. Okay. I just love this Slide with the insane con posse. They represented themselves in court Which must have been amazing anyway, uh, so patents how do those work? Uh, so the patent bargain is supposed to be you publicize an invention that you have made some some thought or idea or Improvement that you intend to add to the grand sum of human endeavor How nice of you except you would like 20 years monopoly on it and the government says that's okay as long as you Fully describe the invention so that the end of the 20 years like we can all make that whatever it is Which is pretty good when what you're inventing is like a spinning genny or like a a new type of railroad train or something like that because Wining up the means of production to actually build lots of trains or spinning gennies or whatever may take you a couple of years You have to go dig iron out of the ground and find people to make it into that thing and blah, blah, blah, blah So, uh, our patent system kind of comes from that time frame As you might have guessed I don't feel that that time frame is such a great fit for software Because we don't really have to dig anything out of the ground in order to share software that we wrote over the weekend so, um The reason we call it so the bargain is a little uneven and uh famously scholars like to refer to the current situation is having too many nooks and crannies to really Offer like fully like oh, hey now everyone knows where where the grand total sum of human knowledge is and it's like Oh, yeah, sure. They're just going to go to the us pto pto and look at like, you know 500 000 patents Uh, no, uh, their site's actually terrible. Anything you search for like Uh, you it it loads up as if from like a modem or something like 10 results at a time Uh, so the uh, the idea that patents are adding to the grand sum of human knowledge is a pretty weak one in this day and age But we you know, we're sort of stuck with it and I'll talk about that in a sec Um, so what patents do do is they protect your assets In so far as they give you an the right to exclude others from working on what you've worked on if you have a patent They don't really give you the right to do anything else like you can't um You can't make people like your version of the thing better or anything like that All you can do is say you've inferred my patent and I want you to stop And then also pay me so um which is Surprisingly its own business model actually so patent trolls, uh Or they like they like to be called non-practicing entities Everyone likes to be called something other than what you like to call them. I guess but uh, they end up costing firms that are actually producing products and writing code and software and making things Uh, approximately 29 billion dollars a year and out of pocket costs and this is from 2011 This is actually if you looked at it on a graph. It's one of those exponential growth rates Uh, a couple of those things that have changed as far as uh patent trolls. Um, the line between Practicing and non-practicing entities has gotten a lot fuzzier. It's sort of this thing as if like, um You're in the park and like some guy ran by with an old lady's purse and then you were like, hey, where are all the old ladies? That's sort of what's happened practicing entities have um figured out how to spin off Arms that troll Shell corporations that troll on their behalf they license their patents to other companies with different names so that they can Monetize their patents. So the line is very fuzzy You'll see companies that used to have a practicing business in a couple of different areas and then like, you know, they didn't quite You know get where the technology was going and so now they have a few defunct areas and one area left Where they're still practicing, but they have patents on those other areas and so they are still sending letters So the I just want to You know, make sure that you know, we're not only talking about trolls here. There are plenty of bad actors in this space uh, so For companies that means do you want to spend your money on r&d? Or do you want to spend your money on policing? Like do you want to? Monetize your patents or do you want to pay money to make more things and do more innovation? So and that's a real question. Um It's probably a little bit quicker money on the patent infringement suits But it doesn't really add much to the overall endeavor of our industry to have everybody working on policing each other all the time Because eventually you run out of money to hire developers and actually make things Which you know, I might I might be a little biased on the developers versus patent attorneys so, um So this is calling 10 if you if you like reading academic papers on patents She's the best place to start. Uh, I happen to I know most people don't but uh, she said You know, she's written a lot of uh papers on why to open up the patent system She's done a lot of looking at the numbers on who is getting sued for patent infringement It tends to be more often like smaller and medium-sized companies than it used to be Because the big companies have started fighting back. So, um This is these are some of the trends and things that are going on in patent infringement suit area So, uh, I work at the open invention network one way that you can Ameliorate your risk in the free and open source software and is to join a defensive patent pool They're also like some so we we run a defensive patent pool 10 years old with about 1900 companies and projects in there And we all agree to not sue each other and then you get a crash license and all your patents So if you get sued from outside the pool, you can counter sue They're also starting to be some other interesting ideas, uh license on transfer network. Uh, I think maybe 80 percent of the patents that trolls use are coming from practicing entities originally And so there's now a lot of energy to kind of figure out like well when we're done with patents like Well, at least let's make sure we don't sound the trolls because that doesn't help anyone So, uh, so the license on transfer network is an aggregate of companies that Try to help each other not sell their patents to trolls and they're done with them Uh, and then uh, we also coordinate defensive publications So this is uh keeping other entities from getting patents on stuff that you might use but you don't want to actually get more patents. So So there are some ways to ameliorate your risk and um I I don't want it to sound like it's all doom and gloom So there are some ways to ameliorate your risk So if you do go and you talk to the legal and they're like, oh, I heard that we would get sued for using all that open source stuff There's actually no numbers that show that open source companies get sued more I think we just tend to talk about it more because we Maybe makes us angrier. I don't know but um, and we also tend to know when each other are getting sued because It's a clear chain of upstream and downstream and things like that. So That's uh, so defensive publications during the offensive patent pool And um, if you are going to create patents like uh, be careful what happens to them when you're done with them Uh, so How else to go and do floss? governance is How you talk about the way that you interact with the free software communities or the open source projects that you're participating in And uh, there are a lot of different models for this. So, um If you are like, oh, but what if we we don't know how to have a relationship with an open source project Or we don't know how to have um, you know, what kinds of guidelines we should set up for our Our employees on participating and contributing to upstream Luckily, you do not have to reinvent the wheel. This has all been figured out before there are plenty of other companies probably your direct competitors or Doing very similar things. So there's a lot of there's a lot of uh, writing on governance out there again You do not have to reinvent the wheel. Um, you could probably tomorrow find like an example of Here's our policy for having employees Uh, contribute to open source projects that fits your business model so, um So then we're gonna do the 10 cent tour. So, uh It's a very interesting time to be in. So when I started this we were just starting to be like, oh, these patents are a problem I was like 10 years ago and then it like Now now it's like, oh, aren't we almost done? There's almost solved or whatever Yes and no um, so There's a couple of different things. So, uh We have had some federal patent reform already and we may yet still have more there's a couple different um, a couple different ideas going on. Uh, some of them are, um Fees shifting where the idea is that trolls there wouldn't be as much incentive to sue companies if When they lose the company gets paid back for their legal fees from the troll That that makes sense everyone. Okay. Uh, so that's one of the ideas some of the other ideas are, um Making it so that you have to do a better job of Disclosing the real party in interest. So remember I talked about practicing entities that have like a uh, You know, like if it was like the Depp Nicholson corporation and I'd be like, oh, no, no, no That's not me. That's the pebb Nicholson corporation. They just happen to have a license on all my patents But some of the reform that's being discussed is I would have to be like, yeah, okay I'm the sole owner of pebb Nicholson to be honest So, uh, so that's one thing and then also being clear about like what types of patents you're suing on Like so you can't just say like it seems like you have some technology that we have some patents on And there are like letters that don't really disclose anymore than that Like they might say just a little bit like, yeah, we have some scanning technology patents and you scan So I was like, what what is that? Um, but some letters don't say what activity the defendant is engaging in that is supposedly infringing on patents So someone those are some of the ideas at the federal level some of the stuff about the uh disclosing the real party in interest and Saying the patent that's actually being sued on and saying the activity the defendant is supposedly engaging in there that they're getting sued for Different states have been trying to pass legislation that would address different parts of that None of them have been able to do so much of the fee shifting unfortunately That's seen as something that would happen to have to happen at the federal level so another thing that's an interesting opportunity for uh, you know Dealing with some of the furthest and overly broad patents is interpartis review So this is when a patent that already exists and is crummy you go and you say like I noticed this patent It's crummy and I would like to tell the us pto to look at it again and please do their job this time and um This this is a great tool. It's not cheap to use. You still need to hire people and pay some fees to us pto to have them Look at things um We also did a huge uh case last year, uh, alice versus cls bank Which invalidated a lot of patents we think but you can still be sued by a patent a patent That would not make it through a court case Because the only way to know if it would make it through a court case is to have it do that actual process so Some of the estimates are that um as many as like 80 percent of the software patents that are currently on the books Would be bogus if they were taken to court tomorrow Of course, that's really expensive 80 of 200 000 Software patents on the books that we would have to go to court and look at so um So there's still a lot of patents out there and that's expensive to challenge them whether in court or by asking the us pto to Take a look at them So strategies and resources Uh communication is key. Uh, this is one of the You know, we we kind of touched on this at the beginning like leading with the uh business interest as opposed to leading uh with the Why freedom is the best when you're talking to your employer depending on your employer, of course But communication is you you have to tell them what you actually want to do as a business model So if what you intend to monetize like if you're you're you're envisioning like maybe you're going to run Like some kind of a service. It's really niche Um, it's going to be you know, whatever micro blogging for people who knit and you're like, no, no, no yarn ads That's all we need doesn't matter what the software is We're going to be like yarn sellers are going to be busting down our door. That's all we need So if that's the case then tell your lawyer that so you can't they can't give you advice on what type of licensing to choose or um, you know, how to think about How encumbered with patents your spaces unless you give them all the information So you want to hear pretty thorough like this is this is what I envision us doing This is where I envision the money coming from and this is the way I envision us interacting with the community So you want to be really clear about that uh, and timing is everything of You cannot go in on friday night and say we're going to release this code on monday sometime If you just take a look at it over the weekend to make sure licenses are cool No, that is that is the you will almost certainly get no they're not That is that is the only answer that you will get in that time frame. So, uh, I would say that earlier you start involving The person you want to have review your licensing plans the better So you want to let them know like here's our new project Here are some of the technologies that we're hoping to incorporate Here's the code base that we hope people are going to be linking our stuff up with Here's the way that we want to be interacting with the community and you know at every step of the way if it changes It's like, oh, it turns out we're not going to do a new sql. It looks like it was going to be easy But it's kind of annoying or whatever it is So you want to keep them apprised at every single step of the process And then again the great listening skills are going to help you seal the deal so If you use some of those arguments that I talked about at the beginning like oh It's free and then you like to skid her off into transparency and your boss says she's like did you say free Just like like no money So then you might want to just keep pressing on that point, you know But the idea is to be responsive and and how to be a dialogue You don't want to come in and read from a set of cards or or this slide presentation even For your boss you can use this if you think monkeys will help at your workplace So um So I hope that you'll uh jump in. I hope that you'll Go out and talk to your employer about using more free software at work Whether that's in tooling or whether that's in your actual products and stuff and uh that you will go ahead and You know get them more invested contribute to upstream. Maybe get them to host a meet up at your office It's great. Uh, it will help you get more happy employees And um, you know and we'll build bigger greater things, right? So these are a couple of organizations that want to help you do free software stuff Uh the fsf they have a faq about their licenses the open invention network where I work Which we would love to have you sign on for the less suing if you're into less suing. We're into you um software freedom concerns you which is working to um Police the licenses and make sure that people don't abuse them It's uh, if you think of the copy left code as a commons making sure that everyone uses that responsibly osi has a another slice of list of free and open source licenses that you can take a look at and they have some They have their own ideas and thoughts about like how some of those interact with each other and So these are all great resources. Uh patchy of course has a lot of information about their own license. So uh Find a buddy and uh, and I hope that you will get involved in free software Uh, oh, I meant to put that the other way around but if you do like reading these kinds of things, uh, this was um Uh that first one uh ces just like two weeks ago there was an amazing panel on patents and um fixing our broken innovation system with The new head of us tto did an intro on it and then they had um someone from qualcomm and uh a politician um Darryl Eiffa who has submitted some legislation on patent reform It kind of starts out really polite and then it like It gets no hold barred as you go along. So if you want to know like why the policy uh landscape is kind of Mind field that panel will give you uh like that one hour later. You'll know you'll know exactly why it's a total mess Um, if you want to uh share how to make money from open source platforms There's a four part series by john mark walker who was at red hat at the time on linux.com Uh talking about like why the red hat model is reproducible and uh, you know So we talked about that a little bit and why uh, some of the other models when people think oh, we're just like red hat We have community and we have different versions of code. That's enough, right? It's A little more complicated than that than that and he goes really into it and then um, like I said, if you like reading academic papers about uh patent trolls and stuff Colleen Chen is great and then uh, julio samuels who she was at EFF and now she's um At her own organization and has been looking at the legislation that might impact the scope of patent ability Especially for software. So so these are all um, if you like this topic then you will enjoy looking at these Uh, of course, it wouldn't be a legal talk without all of the credits And then I would be more than happy to take your questions. Thanks. So obviously i'm deb my you could even my my colleague too She co-presented with me the last time we talked about this All right, what do you think is your brain full or I knew that Yeah in the back Yeah, so I'd actually take a look at the um, oh Uh, I'd take a look at John mark's piece on that too. I'm sorry. The question was is there a line on which parts you open and which Parts you don't um In general if you Uh, are trying to give something away and hope that it will make people want to buy something from you Uh, what you give away should not be crappy So if you're if you're uh, if your business model is we will make a crappy version of something And hopefully that will attract people to want to give us money for another version of it That's not that business model is not going to work out for you. Um If you People have had more success with we gave away something that people actually wanted and then there are a couple little add-ons that are proprietary Uh, although, you know, it depends on the space and what your users are like Uh as to how well that works. I'm not a super fan of proprietary software So I'm maybe not the best person asked for advice on how to mix and match but um But there there is some stuff about like business models that And mix and matches that don't work in that piece there Does that help a little bit? Yes Right, so it depends on what your business model is if you're using an old style kind of like, you know Microsoft have five years ago business model then you may kind of have already painted yourselves into a corner but if you're looking to kind of I mean it's You can only prop up a older business model with Technical constraints for so long eventually people will find a way Around it and or just find another thing to use. So, um If you're trying really hard to get people to pay for something that they really really don't want to pay for See like cord cutters They will figure out a way to not pay for it whether it means getting it from someone else or not doing it so, um, I guess the idea is to kind of talk with your CEO about Looking at newer business models and why some of the traditional companies that have never used open source before Are starting to use them and why they're opening up and doing that So that's you know without knowing too much more about your specific situation That's what I would say is that if you're if your business model relies on trying to get people to do something that don't want to do then you might want to start moving to other business models So yeah Yeah, you're still going to know your code the best and so if you're selling code plus something And you think that people mostly want the other something whether that's hardware or service or you know access to the platform then Yeah, you're the expertise that you have because you wrote it is always going to override someone who forks it and starts to try Use it tomorrow Other questions The green lantern has no questions Okay, good to know um, if you Have questions that you didn't want to ask in front of the oh, sorry. I'm on the wrong direction here Uh, if you have questions that you didn't want to ask in front of the group I'm uh debit open invention network. You can always ask me stuff In that regard as I said, I'm not a lawyer. So, um, You know, I won't give you free legal advice This is not a backdoor to free legal advice, but I can point you at some other resources Uh, if there's something else, yeah Okay, so for the recording if there is one, um Der Hunts has said that the uh, the freeze and freedom obcast is another place for pedantic but accessible conversation on free software legal topics and And you and you can use it. It's usually tagged pretty well like every five minutes like what the topic was it's covered So if you are looking to get another perspective on something specific, it's pretty searchable and easy to find The uh episode where they discussed that topic Anything else to add or ask or Okay, awesome. Thanks so much for coming Dustin hey, that actually works. All right. Welcome everybody We're about to start off the next session and like to tell you a little bit about our next speaker We have John Hawley who's uh led the system administration team on colonel.org for nearly a decade um, his other work includes, uh, working on syslinux open ssi A caching git web and patches to bind to enable godmf He is the author of a pxc knife I'm about pronouncing it correctly and and a set of interfaces around common utilities And diagnostic tools that are needed by your average systems engineer now, um, he's also worked on sync diff and Which is a faithful full file synchronizer and a file transfer mechanism He's currently working for intel on their open hardware and the minnow board Please help me to welcome john Well, I thank you all for coming to literally the last session of the last day of scale. You must all be Yes, this this may actually be the best session if for no other reason while the title says Failure is half the fun of success. The real answer is this is the john hawley story hour It's where where i'm basically going to tell you everything i've ever seen done wrong And how it all ended up probably for the best in the long run So I gave this talk um, a few months ago down in guadalajara actually And I really had the intention of going down there and talking to them about Uh, that it was actually okay to fail, you know, we you know even up here and you know everywhere we have this perception that um Failure is like the ultimate thing that we can do wrong and that you know once we've failed It's just the end of the world, you know, everyone hates me, you know, you might as well throw everything in the trash and um The reality is is that's not actually true, you know failure is You know failure is a step in a process. It is um the only reason that you would actually genuinely fail is if you stop the process um You know when we when we fail What we need to do what we really need to learn and what we need to do is Learn to keep going learn to figure out why you failed what went wrong What you can do better next time or just quite literally how you can fix what's already broken And in some cases, you know, and most of this talk is just going to be about You know things that have gone wrong that or that i've seen gone wrong before And that no it's okay for you to fail as well Because i'm not perfect. You're not perfect It's all good to a case in point Uh up there on the screen. I've got a little robot That uh, I uh I ended up building a few years years ago. Um, you can't quite see it because uh the Room's a little washed out and I did not think to actually lighten my slide Which is hilarious because you know, I uh Mentioned that to someone this morning as one of the failures of projectors but um You know one of the things that uh that I did with this robot was I had never built a robot before You know, I was a low-level systems programmer by trade And uh, I'd never built a robot and I said it. I'm just going to build k9 from dr. Who And uh, you know, I I I didn't think about this too much and I just started plowing straight in To say that I bit off more than I could chew was a bit of an understatement and uh Started with a box of parts and God, you really can't see is there any way we can dim some of the lights Because I mean, I know for the folks who are you know, going to be on the video stream This is going to look fantastic. But um, those of you here in the room. You can't even tell how how ridiculously jumbled Yeah, I I don't think it's going to work You know what? The slides will be online if you want to come look at them afterwards fine. I'm just going to I'm just going to talk to them anyway, but I ordered a pile of parts You know, I had this vague idea of what I wanted to build And I went ahead and bought up hundreds of dollars worth of parts and just went great I'm going to start slapping things together and we'll see what sticks And then I opened up the box of parts and went holy shit. What the hell have I got myself into? And So slowly but surely I started putting those parts together testing what was going on I did not actually chop that in half with my chop saw thankfully um But everything was a trial and error process. You know, I knew that I um Wasn't quite sure what was going on. I didn't know exactly how all of these pieces were going to go together And in fact, you know for things like these tank treads that I was using there wasn't even good documentation Uh, because you know to do up tank treads. Well, you know You're they're going to go around a couple of wheels and you would think that they would give you ideas of what the spacing should be between uh each piece of the uh The cogs and whatnot. Well, they didn't So you had two choices one you could try and figure out the math Which is actually slightly more complex than you would think Or you could just start drilling holes and hope that you get it right I chose option two Unsurprisingly And I just started building things And I did not get it right on the first try. I did not get it right on the second try I did not get it right on the fifth try the sixth try You know after you you know built your castles one on top of each other in a swamp It stood it's it actually worked and um I actually got a mostly working robot And uh, this is uh chassis version number Six actually this is exactly chassis number version six And as you can see there's a whole pile of wires and there's this computer involved and then I burnt out the computer That was running the whole thing So even though I had made it, you know, oh so far built a robot It still failed And you know after you you know spend all this time building this chassis you burn out a robot you go Your options are you can either laugh you can cry and or and or you can laugh and uh move on or you can cry and give up and Unsurprisingly, I just laughed it off Accepted that uh that I had failed or that I had had a problem that I needed to fix it Change some stuff out And eventually got a fully working, you know after substantially more trial and error Got a fully working robot that has been around the world a couple of times now The interesting problem with that particular dog is uh, he was here at scale last year. He also caught fire at scale last year So I can't even say that he's been a resounding success after you know constantly failing he's currently in time out in fact because uh well, he didn't quite catch on that much fire, but uh, uh there was a a couple of wires that You can't quite see it there, but they had rubbed up against each other in shipping and Eventually after you know thousands of miles being handled by luggage monkeys at the airlines shorted and Caused a small electrical fire next to 17 amp hours of batteries So um But you know even still, you know, he actually caught on fire the day before scale started So I actually had one small saving grace and that I had time to fix him An emergency run to fries And 300 dollar well with cal chance. He's actually in the audience. He's pointing at himself Uh, uh, was it 140 dollars in parts? Or was it three? No, it was 140 dollars in parts give or take Um, including buying a new multimeter Uh Came back wired him back up and he was actually running around for the rest of the conference And it's hard to it's hard to claim that There hasn't been anything that hasn't failed on that particular robot I mean robots are one of those things where once you start building them There is nothing that will ever work right on a robot robots are constantly broken Be it's you know, if you look up robot in the definition definition It's you know autonomous vehicle and or always broken And actually I'm kind of curious. How many of you have actually built a robot? Anybody Okay, a couple of you. So, you know, I'm at least speaking to the crowd that the robot's oh No killing the robot doesn't count No kids don't count. They're a little too autonomous And when they break it's usually a little more dramatic than you know the dog catching on fire So Um Yeah, so I I mean I'm kind of you know walking through you through this story is just saying you know, look Step one in understanding what's going on in the world. Just start biting off more than you can chew Start looking at what's going on around the world and go look. I just want to build a robot And don't build, you know, a teeny tiny little robot build something or do something that's Dramatically outside of your reach Don't expect to actually succeed at it, but pick something that's dramatically outside of your reach And I'm suggesting this for one simple fact is the the more and faster you fail At what you're working on the more you're going to learn And the faster you're failing the more the faster you're learning And that's kind of what happened with this particular robot But this is not the only instance of uh People failing and having to learn from uh their mistakes The dog um in its first adventure Actually ended up going to edinborough in scotland Two years ago a little bit more than two years ago and it raced against the octoblimp And for those of you who are watching this May know that Beth Flanagan was the operator of that particular blimp It's actually a very nice blimp. Um at the end of the day Ways substantially less than the dog You know flies around And we decided to have a little bit of a bet That uh the dog could uh outrace the blimp or that the blimp could outrace the dog Pige's uh belief was that the miracle of flight would by far outstrip anything that uh some land-based silly dog thing could do Well, she learned a very very valuable lesson. Um the day that uh The race took place As the blimp as you can vaguely see in the the uh screenshot there immediately creamed out of control Everybody's got you know, I see a couple laptops and I'm sure you guys have cell phones in the audience. Um How are you getting networked to those devices? You know, you're connected to the internet. How are you getting internet? Scales wi-fi What usually not in the case of scale thankfully does not work at a conference Particularly a tech conference the wi-fi Most conventions that you go to that are technical people show up with Anywhere between three and eight devices that will uh be connected to the network Um, there are some very large conferences uh that happen in vegas Uh that I know of that they plan on eight devices per person and they're still underestimating in most cases Mostly because there's various people who then you know, they pull out a laptop two laptop three laptop five laptop You know two phones and half a dozen other things but taunting the demo gods with wi-fi or networks in the general sense is always a bad idea And so I I diverge on this little rant about wi-fi for one very specific reason What's the worst control protocol to use for your uh drone at a conference? wi-fi Uh Beth did not actually build um most of the blimp She uh passed all of this work off to an intern The intern thought that the most brilliant way to deal with the communications protocol between Uh for the control protocol for the the the blimp was to use a point-to-point wireless link And uh, you know the and throughout the week Uh or several days before this race happened, you know the blimp would be up and it'd be flying around and she can control it reasonably well without a whole lot of problems But that but you can kind of well as I nearly fall off the uh the stage That would have been a failure. I could have talked about Um as you can see there are several hundred people in this room now Who have all pulled out their cell phones and are taking picture and video When you wake a cell phone up the first thing it does it starts talking to the network So as soon as the race started everybody whips out their phone and anything in the 2.4 gigahertz spectrum became an unusable mess Thus the blimp careening off into control and nearly killing people so A an obvious lesson was learned or reminded of at that point, which is wi-fi is not a good control protocol when it's mission critical and you know and um What's interesting about that is that I have I made a similar decision with the dog In that I needed a remote control protocol for the robot And I had thought about this. Well, let's use wi-fi. No, wait a minute wi-fi is a bad idea wi-fi for control protocols On robots is always a bad idea, particularly if you actually intend to take it to a conference So I chose something that made a little more sense and that and that actually ended up being an xbox 360 controller Because someone actually went through and did a ridiculous amount of planning For these controllers to work in a really nasty rf or 2.4 gigahertz rf environment They do spread spectrum. They bounce it all over the channel map But strangely enough your xbox 360 controller almost always absolutely works. It doesn't matter how messy the the spectrum is That's my backup controller for the dog And it worked So a lesson was learned problems were seen lessons were learned The blimp did not quite do that Although we can't actually find the blimp now, so I can't entirely confirm that that may not have happened after the fact but um Yeah Things went wrong We learned from them That's what wi-fi looks like at a conference. I'm apparently I should apparently be like You know hitting my slides more often. I keep forgetting what's up here Um, but yeah, that you know, that's what wi-fi looks like at a conference. The bus is the blimp. I wasn't going to point that out but yes Most uh commercial drones that are available in the market use the 2.4 gigahertz spectrum Some of them actually use directly wi-fi Please don't fly them at a conference We'll just we'll just leave that there. Just don't don't fly them at a conference. We don't need uh We don't need that At conferences fire is bad Just don't do it And the helicopters do way substantially more. This is kind of you know another picture of the the blimp cleaning off and nearly killing people Although speaking of drones um After the uh the incident with the the blimp I was effectively challenged by pidge that Well, she participated in the miracle of flight. So technically she won she won the race I contest that particular uh statement And although our bet was that I got bragging rights for a year or two years later or two and a half years later I guess i'm still bragging about it So I guess I've also broken that portion of the bet But uh, so I said screw that let's put a minnow board in flight And I started actually just taking a commercial off-the-shelf drone And bolting a minnow board to it and doing in-flight computer vision processing So actually had the minnow board feeding information back into an uh an existing flight computer This actually worked pretty well And as you can see that drone, uh, that's down there. It's a pretty sizable drone. It's about about yay big Blade spin at several thousand rpm each one of them is about this long So when things go wrong they can go really really really really badly. Oh my god, you know duck and cover bad And despite the fact that uh, you know, I'm I've flown this for you know hundreds of hours And done quite a number of things. I had one particular incident. Um in the last year where it crashed And it crashed pretty spectacularly You know, um There are two times in an air in an airframe Uh existence that are the most dangerous uh times in its life Takeoff and landing In this case they were both the same thing During takeoff, uh one of the rotors flew off literally From the airframe At which point the the entire airframe became unbalanced and instead of you know throttling up and I'm letting everything stabilize. I did the incredibly smart thing of cutting all the power So it's about this far off the ground. I cut all the power It then falls immediately everything rolls and you know half the blades all break off So again More failures, but the drone's back up and running. I didn't stop or I I I didn't I I haven't stopped flying it But uh, you know, yes everything, you know drones are for One, you know, basically two purposes one you fly them to take video to post on youtube And two to break that you fix them Those are the only two real uses for drones as far as I can tell anybody drone player You don't count you pop the one I told you to buy though And sometimes there there are just things in the world that you just don't want to screw up How many people here have commit access to repositories or run large rate arrays? Do you have you ever thought about the miracle Of what happens to get from when you hit the save button on something or the commit button To that those bits actually ending up on a hard drive And then the ability to pull that all back out in the same order in which they were put in If you're a storage person This is what keeps this entire idea is what keeps you up at night because this shouldn't work There are so many moving pieces. There are so many Points of failure in this whole thing that it just honestly should not work And and and then you start thinking about you know multi disk arrays and you know petabytes of disk and you know Most people curl up under the chair and cry because it's just mind-boggling So how many of you have actually pushed a commit that deleted everything Apparently gen twos deleted everything at least once Or how many of you have accidentally hit rm minus rf on something that's like the most precious thing possible Backups are great. Aren't they and this is the point I'm sorry Oh rm. Oh alias to rf minus rf. Oh, yeah aliases are yeah bad. Yeah And they're done that undid that alias So um, yeah storage is one of those things where everybody You everybody depends on it everybody absolutely depends on it But but it is one of the most fragile things in the world and if you look around there are stories everywhere of somebody Accidentally screwing everything up. In fact, there's some stories from the google summer of code students There's actually a Fantastic story from one of the python students. I don't think it was I think it was last year of the year before was the last year Where, you know, they had been working on their code. They had been doing all of these really great things They had fun, you know, the the project had finally given them full commit access to the repository And the first thing they pushed completely trashed the entire repository history from end to end And the student Unsurprisingly was mortified. You know the first time anybody had ever trusted them and they had utterly failed And you know, they then went immediately fessed up to it and they are by the time they had even talked to the rest of their project They had a plan on how it how it was all going to get fixed. They actually went through the entire process fixed the repository back to a completely good state And they moved on with life And if you think about this most students, I mean google summer of code is all students in uh college These are, you know, a lot of, um You know folks who have never even had to deal with source control let alone been given enough Control, you know control or a responsibility to push something that is, you know used by hundreds of thousands of people And you know to be given all of this in one go and to screw it up You know most people would immediately curl up into a ball under a table and you know, they They'd go and become the hermit that you know You know sit in the that the watchtowers for forests and never touch a computer again Um by screwing up like this, but the student, you know, my hat goes off to them They not only, you know did all you know screwed up Stick, you know had a plan for fixing it by the time they um Let anybody know fixed it, but then they had the um brilliance to write a blog post about what happened So they they not only fessed up to it. They told everyone what they did and they shared their story And you know of all the things in the world. That's probably one of the most courageous things i've ever seen is just admitting I asked up on such an epic level and yet I want to tell everybody it's okay They didn't get in trouble, you know, they had a plan they fixed it screwing up is okay Screwing up and not fixing it or completely walking away. That's when failure actually happens It is not the act of You know the the screw up itself running rm-rf crashing A drone it's the act, you know failure is when you screw up And you don't try to fix it. You can't figure out how to fix it. You give up and you know To some uh, to some extent that this is what this talk is about is screwing up is okay We all do it. We've all done it some of us substantially more than others If you know my background at all, I had a very exciting later half of uh, uh, 2011 Um where I had probably the yeah, may I have a software notification? There we go Um later half of the I had a very exciting later half of 2011 If you ever want to look that up or want to talk to me offline about that I will Fly very deeply and tell you my sorted story um But uh, yeah, I mean the people screw up Sometimes it's really epic I'm going to get back. I'm going to go back in my history a little bit Back to my second job out of university Worked for a company called Orion Multisystems They made These very pretty compute clusters This is 12 full computers hacked onto a single pcb. The pcb is this big I'm not I'm not actually joking for those of you who make hardware The pcb is 48 layers thick to put this into perspective most high-end motherboards Are 8 to 12 layers thick This is the equivalent of taking 12 You know just stacks of motherboards and smushing them into a single pcb At the time And I believe to date this is the largest and most complex pcb That flex tronics has ever done I believe they also swore at the time when they were manufacturing this that they would never do that again so When you start looking at something that's literally this big and complex There's a lot that can go in fact there's a lot that can go wrong even in the manufacturing process In one case we had we were doing bring up on a board. We were doing some testing it everything was going great Unbeknownst to us at the time there was an air pocket in one of the power planes Guess what happens when you put Air near a very um high temperature power source Not only did it expand It exploded and well Effectively you create an oxygen rich environment in which all you need Is a spark which oh hey look there's a lot of power right there Go figure Flames literally shot out the side of the board Needless to say that one was dead But in that failure, you know, we had spent, you know thousands, you know, you know thousands of dollars for that single board And that one failure I mean there are failures of hardware in small companies and we were a small company that could have ruined us Well, thankfully that one didn't We died for other reasons but um You know, we didn't you know, it didn't stop anybody But you know sometimes you end up with a failure or a screw up That is, you know in retrospect kind of funny, you know You don't usually see fire coming out of the edge of a board. It's kind of pretty Really hard to recreate But we did a lot of really fantastic things with this particular compute system. It was not perfect by any stretch of the imagination You know, we had 12 computers on on this board and literally 12 computers North bridge south bridge Ethernet memory each one of these I mean it's You know, if you look down here This is a full computer It's about that big there are 12 sets of this on the motherboard along with two ethernet Or two full ethernet fabrics and power for everything, etc, etc But they don't have serial ports or hdmi ports on any of them Except for the head node on that particular board Which you can see is way up there kind of in the left hand side Which makes it really problematic when you try to Bring things up like windows Windows particularly at this time. This is about 10 15 years ago Really wants a console or something that it can at least talk to the world with and if it can't find it It doesn't work at all So we had a trade show up in seattle at one While we were there that we the microsoft really wanted to show off their the latest clustering software And we we said yes, we can make it work And well we could get it working on the head node really really well Except trying to get it on to all the other compute nodes was a problem because we couldn't get a console out So one weekend we literally went and created serial consoles For every node on a cluster And when I say a cluster this is a single board We had machines that would take Six of these as I do the math in my head And put them into a single fabric 72 or 96 nodes ultimately So maybe that is eight of them eight of them it is eight So we we had eight of these so we had 96 computers that we then had to go and hand solder and hand create Serial console systems for For the boards To bring this all up We did it we took it up And we were the only company at this particular trade show That had a working cluster with microsoft Unfortunately, they were apparently still angry at us because we were a 32 bit processor and not 64 bit But we at least had the but we released up and running So that is the claim to fame on that And you know sometimes just shit happens and You know Again more things that just go wrong a few years ago a couple of years ago actually was uh Working on building some starship bridges. Why because well, you know, everybody wants a starship bridge, right? And I was doing some cnc milling and I Happened to miscalculate something and you know Chunks of Plastic were flying all over the place from the cnc machine because well, I did something wrong I learned I fixed it. I mostly slowed my progression down and instead of this supposing Supposedly supposed to take me about Eight hours to do everything took me 40 hours of standing over a cnc machine Because I had to throttle everything down so so much slower than I had expected Still turned out pretty well I think I've Ta-da not that you can really see that because it's a really dark picture, but it turned out pretty nicely So I'm running a little bit fast. So maybe uh, so hopefully You folks in the audience will have some more exciting stories of things going wrong but um Thomas Edison was once criticized rather brutally for uh Trying to invent the light bulb because he had tried 10,000 different ways and none of them had worked He stared at the the reporter who was uh interviewing him at the time and he said I haven't found you know, I haven't failed 10,000 times. I've just found 10,000 ways that don't work And in reality, that's what we're all doing whether we're working on software working on hardware Particularly when we start going outside of our immediate comfort zones, we're doing nothing but learning and uh When things screw up when things fail when we make mistakes That's not us failing. That's us. Uh, just You know screwing up effectively failure is when you screw up And you don't fix it. You don't learn from it and you don't do anything about it So I You know, I actually actively encourage you all go do something Crazy go do something weird, you know fail fail faster The faster you fail the better you'll be the better your projects will be the better your software will be the better your hardware will be Because we you don't get to be You know Anywhere in the world if you're not screwing up So it's has a story. I've told a bunch I'm sure, you know, I'm sure there's somebody in here who's screwed up and willing to admit to it Oh, hey Oh You can see the screen. Whoo. We've learned something Oh here, I'll just back up and show you some of the pictures you can oh, well that one's still dark See if I can Oh, you can see the crowd better. You can understand why the wi-fi was in the house That one's still funny Oh, you can actually see the the break in the wire a little bit better now So you can see the molten plastic and they uh Thankfully, I used a thin enough wire that the wire itself acted as a fuse and burnt itself It's a little bit easier to see. Yeah, I should probably lighten these pictures up at some point. Anyway Come on. Who's got a story? I know there's not a whole lot of people in here, but somebody's gotta have a story You got a story Yeah, I mean Doing things. I mean you're you're talking about playing video games at levels that you know, you're gonna fail at or You know picking on or taking on fights that you know, you can't win But at least the process, you know, just because you're not going to succeed You know at least going in you know, you're not going to or probably not going to anyway Um, but you've learned something just from the process itself I was gonna say you should email Steven Rostad and I about uh our plans on that But uh for those of you in uh on the recording or whether you uh Could hear that he he's suggesting, you know, well, you know, there's system D out there and you know Everybody's kind of you know just uh accepted it as the foregone conclusion Although the gen two guys up here in the front would disagree with you on that Um as I believe you guys are still open rc, right? And they're very proud about being open rc Oh, they're whatever I want. So I want sys 5 I want sys 5 So I can I can still have it. Maybe I need to become a gen two guy You know, I've ragged on the gen two guys for god. How many years is it now 15? No, I'm not running up start Okay, nobody likes upstart. I'm sure there's somebody who's gonna like email me now and say that they love upstart Yeah, well I was gonna say if you really want to um Go back and look at things Take a look at one of my top or Linux.com did a write up on one of my talks years ago from the Ottawa Linux symposium There's a there's a choice quote in one of them Uh, so I'll let you go look that up and google that for yourself But uh, yeah, so, you know take on system d come up with a new in it system and you know Except that you know system d is not perfect. Maybe it needs to learn You know, it needs to have a a different in it system to actually go and compete against I mean competition is actually a very healthy thing and the fact that there Isn't in the in it world right now is actually kind of a problem I I will refrain from comment on that one on the uh advice of people who tell me I shouldn't You know quite flame and troll that much So come on it's also screwed up in the room Yes, I mean it Yeah, I mean it's there's always somebody who's got more expertise in an area than you well almost always somebody who's got more expertise in an area than you and Usually the people who have more expertise in your case the bike mechanics are going to be more than happy to I take your money To fix your bike You know at least in the open source world where at least nice enough to give advice Instead of immediately demanding money um, yeah, I mean and you know sitting at the feet of somebody else and going You know, I've screwed up, you know, how do I deal with this? I mean, I you know, I have I there are kernel developers who still ping me about well, how do I deal with this? How do I deal with that? Mostly in systems administration kind of things because I used to admin kernel.org for about a decade You know, they all know, you know, mostly how to contact me Um, and they know that I know What they're looking for and I'm sometimes faster than trying to read the band page Which usually is a bad sign that the man page needs some updating but Yeah, yeah, is there a better approach, you know, what should I have actually learned from this failure as opposed to that? I you know screwed up So mostly that I invited the gen two guys to this talk and they're not heckling me nearly enough You're tired. No, you're not tired. You're gen two developers. You're you're compiling something I really do rag on you guys like in every talk I ever do, don't I? That's why you like me so who else has failed Besides, you know, the gen two guys for causing global warming Really, I'm gonna do that talk someday. I've been threatening that for years How gen two has directly caused global warming by causing everybody to recompile everything all the time Huh What is the polar bear ever done for me? Let's see. Uh, it has caused the creation of plushies that are very cute And they eat, uh seals and they help the inuits stay alive. How's that? Is that enough? Seals and shirts that need to be eaten Well, that'll teach the seals for you know screwing up and Not learning anything, right? I can always end early. I mean it is the last day And the last talk of scale. I'm happy to end early if you guys want So by all means thank you for attending. I hope you enjoyed scale. I hope you enjoyed the talks And if you did, please let uh The the the volunteers know that you appreciated scale because good lord. I've been watching them work their asses off all weekend And watching them work their asses off for months to pull this all off and honestly This is one of the best conferences. I've ever been to and I go to a lot of conferences. So Thank you