 And that's all I got done. All right, so my name is Jay Pipes. I'm from Arantis. I'm going to totally screw up her name. But Yilder Galvancha? All right, pretty good. From Ericsson. And we're going to talk a little bit about the OpenStack contribution process and community. So I'm going to talk a little bit about my experience and some tips that I have for folks that are interested in contributing code and specifications and just contributing to the general discussions around roadmap, product discussions, that kind of thing. And then Ildeco's going to tell you that I'm completely wrong in the real world. Everything doesn't work out as optimistically as I think. So anyway, I'm going to be going over some of the tools that we use in the community and some best practices for using those tools and understanding the product roadmaps inside of OpenStack and the code review and contribution process. And then Ildeco's going to tell you also that that's all a bunch of BS and that you should really use Microsoft Outlook still. But I'm just kidding. You'll see. Anybody seen the movie Soylent Green? It's people. OpenStack is made of people. Anyway, Soylent Green was made out of people. OK, not big movie people. OK, all right, so I put this slide in there because it really is true. I look around you. OpenStack is all the people in this room. It's all the people out in those rooms that weren't kind enough to come to our talk today. But you can't just say, oh, OpenStack is a set of products or projects. Sure, it's source code. But for the most part, it's the people working on that source code that are contributing to the documentation and writing blog posts and doing deployment modules, not just writing the code of NOVA or Neutron and that kind of thing. There's a million ways to contribute to the OpenStack community. And the first thing when you're sort of introducing yourself to the OpenStack contribution community is remembering the first rule, that it's a bunch of people out there, right? Even though you're maybe posting to a mailing list and IRC, you've got to remember, these are people on the other end of whatever you're sending out there. These are people that are contributing to the code bases that are writing documentation, that are doing translations. So treat them with respect. So what kind of people do we have in the community? We have all sorts, not just developers. We have deployers, marketing folks, business managers, operators, right? We have people that do development but also do deployment. We have deployers who are running operations. We have operations folks that are making business decisions about the productization of OpenStack or ecosystem type products. Don't assume that when you're talking to a person in the community, whether it's on a mailing list or in person here, that that person only plays a single role within their organization, right? That's a very big mistake that a lot of people say. Oh, Jay's just a developer. Jay just works on NOVA, right? Well, it may be that I have some operations experience or that I've worked on deployment modules, right? Or that every once in a while they bring me into business decisions, but I hate that. But anyway, don't assume that whoever you're talking to is just one-sided. So there's a lot of overlap. There's also a very diverse set of contributors. You've got people coming from government organizations, all sorts of development houses, product shops, systems integrators, tooling and library developers. How many people know that, for instance, the creator of the SQL Alchemy Python library works for Red Hat on OpenStack? One person, right? The reason I put this slide up there is there are a lot of people working at lots of different organizations in the OpenStack community and ecosystem that bridge a lot of different roles and that work at a variety of different employers that bring a different viewpoint to the table during discussions, right? Obviously, we have a lot of people currently interested in NFV, right? We have a lot of telco organizations that are pushing certain agendas. We've got government organizations that are very security focused. So every viewpoint is sort of expressed in the OpenStack community. So one of my favorite websites, you had one job. You probably can't see it, but the cat is not exactly performing the job that the cat was intended to do. My point here is, again, don't make the mistake of assuming that whoever you're talking to or interacting with in the OpenStack community has a single role or that they don't understand the viewpoint that you might be expressing or that someone else in the group is expressing. A lot of people in the OpenStack ecosystem especially in my experience have a lot of cross arena, cross industry experience. And it's really good to listen as much as you talk and understand those experiences. So tools we use. Mailing lists. How many people here actively use the OpenStack mailing lists, OpenStack Dev mailing lists, OpenStack infrastructure mailing lists, OpenStack operators mailing lists, any of the mailing lists? One, two, three, four, five, six. Six out of the room. OK. Well, this is really the preferred way of doing discussion-oriented conversations about OpenStack. We have a number of mailing lists. And if the topic that you want to discuss with some community members is oriented in a threaded or conversational format, the mailing lists are the way to do that. What is not the way to have those conversations? Private email threads between 10 or 12 people at different organizations. That is a closed loop. That is the opposite of the openness and transparency that the OpenStack community is all about. The mailing lists are all run on a list server that's managed by the OpenStack infrastructure team. There are many, many lists. But really the ones that are the main ones, the most popular lists are OpenStack, which is for usage questions, and then OpenStack Dev, which is the developer mailing list. It's by far the most trafficked list. We typically, any given month, will have 250 to 500 conversations on that mailing list, some of which reach hundreds of mailing list posts within that thread. If you ever see a conversation about what is OpenStack, or is OpenStack a big tent? Those conversations tend to go on for weeks and months. Anyway, so the mailing lists really are where a lot of that conversation takes place. The OpenStack operators mailing list is specifically for operators. So operators and employers will ask specific questions like, why doesn't this work? Is this supposed to work this way? And you'll get other operators responding to those posts, as well as developers responding, yeah, it's not really supposed to work like that. This is a little documentation that's not really documented. And that kind of stuff happens on the operators list. If you're going to post on the mailing list, don't use Microsoft Outlook. I'm not a Microsoftator. There's just very specific reasons why you don't want to use Microsoft Outlook. It does not handle threaded conversations or replies correctly. And by correctly, I mean the right way. Also, don't use HTML email. Example of why not to use HTML email. Ildigo said, did you just pick someone from Ericsson to put, no, actually I didn't. This happened to be in my inbox at the time that I was writing this slide. And it's a perfect example of why not to use HTML email. So on the left here is a conversation between Alan Kavanaugh from Ericsson, Keshava from HP, and a number of other folks from Cisco. And until they're discussing NFV related stuff, all fine and dandy, it's actually really, really good conversation and lots of fun. However, when this mailing list thread gets produced onto the archive of the mailing list, it turns from that on the left, which is, I think, actually kind of hard to distinguish. For instance, Keshava is actually both the green and the light blue on the left. But anyway, all of that stuff gets condensed into a non-indented, non-formatted text output on the right. So these are actually the exact same conversation. But the one on the right is the archive that is publicly indexed by Google. And so when you're going to read it, it makes absolutely no sense. So that's the reason why not to use HTML email. Some other things. Don't make a habit of top posting. The example in the last slide was a mixture of top posting and inner posting, all in the same thing. Try not to do that a lot. Sometimes it only makes sense to top post. But most of the time, if you're doing a developer-related type of conversation, try not to do it. Also, don't forget to include a topic marker in your subject line. And this is so people know what you're talking about. Kind of basic stuff, but so many people don't remember that. Anyway, a lot of people have filters on their email so that if there's not a match on a particular topic marker, it just goes into DevNol or the Wastebasket. And so just always remember to put a topic in there. Nova. You can have multiple topic markers, too. So you can bracket Nova, bracket Neutron. If it's related to both Neutron and Nova, that kind of thing. But always include one so people can filter in their head when they're looking at it. Also, don't send code review requests to any mailing list at all. Just find people on IRC. We don't buy it, generally. Oh, yeah. Forgot to say something about reading that Wiki page and then read it again at the very top of that Wiki page, so something about Outlook. Anyway. So IRC is our preferred method of communication for anything that's sort of real time has to sort of go back and forth between people. If you're not familiar with IRC, it's very similar to IMing. Only it doesn't suck. So we have lots of channels on FreeNode. There's probably about 50 OpenStack-related channels. Each of the main projects, like a Nova, has an OpenStack-Nova channel in FreeNode, so OpenStack-NewTron, OpenStack-Noven, and so forth. In fact, most of the Stackforge ecosystem projects also have IRC channels. So if you're interested in learning more about a particular project or volunteering to do some work or even providing some feedback on roadmaps and you just want to get in touch with people, that's generally where most of the developer contributors are hanging out. And all the meetings that we have, whether it's for cross-project release management type stuff, whether it's infrastructure related or whether it's within a sub-project of OpenStack or all on IRC. And there is a link there that has all of the meeting times for every single meeting. Launchpad. We hate it and we have to use it. Each project in Launchpad has its own project page, so you'll have Launchpad.net, Forge-Slash-Nova, Forge-Slash-NewTron, and so forth. If you want to see the general roadmap of a particular project, you can look at the blueprints and milestones pages within Launchpad for that particular project. We're gradually moving away from Launchpad for hopefully everything, but until we get to the point where Storyboard, which is pictured over here on the right, is ready for prime time, most of the stuff that we do with bug tracking and milestone targeting, which I cut out of the slides because it's so boring. We're actually done in Launchpad still. But until we get to the nice, cool Storyboard thing on the right, we're going to be using Launchpad. Garrett, how many people have issued a Garrett patch? Yay, lots of people. How many people love Garrett? Right, so Garrett is our code review tool of choice. It manages our Git source trees for all of the OpenStack and Stack Forge projects. And it has a web user interface, I guess we'd call it, which is interesting to learn and use. There's also a command line interface called GERDI, G-E-R-T-T-Y, which I happen to use and love. All projects that are in OpenStack and Stack Forge code namespaces are hosted on git.openstack.org. And the code review system that manages those trees is Garrett. So review.openstack.org is the front interface for Garrett. And each of the projects has what we call a core team. And they are the people that have approval rights. So it's very important to separate merge rights from approval rights. So Abel, if I have approve rights on NOVA, because I'm a NOVA core member, does that mean I can merge code into the NOVA tree? You know, you can play along. So, yeah, no, no, actually, no, it doesn't mean that. It means that I can approve a patch to go into a continuous integration system. And that continuous integration system, if all tests have passed for that particular patch, then the continuous integration system can merge into a source tree. So we have a non-human gatekeeper method for controlling the code that goes into our source trees. It's very important to understand that if, you know, you say, oh, he's in NOVA core, so he can merge code into NOVA. That doesn't make any sense in our community because only non-humans, the CI system, the Jenkins system, can actually merge code. It's actually Zool that merges it, but whatever. So no humans actually merge code in our system, which is a real check against spurious code and buggy code getting into our trees. It still happens, though. Code reviews, yep, that's what it looks like. That's awful. All right, other OpenStack development tools. Git review is a Python plug-in for Git that we use to interact with Garrett. Girdy, as I explained, is a command line tool for doing code reviews in Garrett. If you haven't checked it out, it really is very slick. I actually have pretty much switched from using the web interface of Garrett to using Girdy now. I think that's how you pronounce it. Girdy? Girdy? Gird TTY? I don't know. But it's actually really slick. You can go through all your reviews. You can do it offline, which means you don't need internet connection, which means when I was on the plane flight over here, I was doing a whole bunch of code reviews in Girdy, and as soon as I came online, it just published all my code reviews, which is bonus. JJB and NodePool and Jenkins are facets of our upstream continuous integration system, which I definitely recommend you check out. Go to ci.openstack.org and read through all of the documentation that the fabulous infrastructure team has put together. Jenkins is our continuous integration server. NodePool manages a set of hundreds and hundreds and hundreds of VMs that run in a continuous fashion all of the tests for all OpenStack projects. JJB is a system for constructing the horrible XML configuration mess for Jenkins jobs. It's really a time saver. And Zool is a pipeline, a dependent patch set pipeline manager for, well, it's not just for OpenStack, but it's primarily used in OpenStack. But it allows you to have multiple dependent projects, patches to multiple interdependent projects be managed in a pipeline. So that if a failure occurs in a gating test on the interdependent project, that it can trigger a failure out of the queues for an interdependent project. It's a very slick tool. I'll show you what kind of it looks like in a second. Elastic Recheck is something that is built out of Elastic Search, LogStash, and Kibana. And it's a tool that our infrastructure team has developed to allow us to query the failures that occur across all of the continuous integration system. I was talking about NodePool and Zool. As you can see here, it's on average per hour is between four and 900 VMs that are running at any given time across Rackspace cloud and HP clouds availability zones that are dedicated, well not dedicated, but are allowed our upstream continuous integration system to spawn VMs, run a whole bunch of tests, and then report back into our centralized continuous integration system. It runs all the time, it's incredibly complex and incredibly massive, and we have a fine group of people that work on all that kind of stuff. Zools managed pipelines, kind of explained it earlier, but this is kind of what it looks like if you go to zool.opensack.org, or status.opensack.org, Zool. It'll show you all of the patches for all projects in Garrett that are in a pipeline ready to be merged. And here you'll see the little dropover and a little red button. It's saying that, hey, one of the interdependent projects for this, for the project that this patches for has failed, and so we know we're gonna fail out that whole set of patches in our queue. Anyway, something to check into, ci.opensack.org, forward slash zool. I love pugs, Abel loves pugs, we're pug owners. This is not one of our pugs. This little guy is representing a pug on a gate. Our gate is the set of tests that we run for each of the integrated release projects. So there's a whole conversation about what is the integrated release? Well, right now it's Nova, Neutron, Heat, Glance, Cinder, all of those kind of projects, and they have interdependencies between them. And the gate is the system that prevents breaking changes in one of those projects from affecting patches going into the master branch, or stable branches of the other projects. If you want a really quick way to get involved in the community, by the way, help the infrastructure team write documentation. It's a great way to understand how their continuous integration systems for OpenStack work, and it's something that they definitely need help with. So it's a great way to get involved. How am I doing on time? Bad, terrible. Oh, I'm at 20 minutes. Oh yeah, I'm not talking quickly or anything. Okay, great way to get karma, do code reviews. You don't have to be an expert on everything. Yieldico is, but most people aren't. So, just remember, you are the Ericsson Einstein. There you go. Okay, so just remember the bottom point here, the bullet point is the most important thing. Ask questions on code reviews. Don't act like you're the expert on everything. Ask questions if you don't know the answer to them. I mean, it's something so simple, but most people are just like, oh well, I'm not gonna comment on it if I don't understand it. Ask a question, because I guarantee you there's 10 other people that are thinking the same thing. So ask a question and get it answered. It costs nothing to be courteous and polite during your code reviews. I do see a lot of people, for some reason, get very upset if they get a negative code review. But just remember that people are just supplying feedback to you, just take it at that. There's no reason to get upset and be a Walter Sobchak if you are a fan of Big Lebowski. Oh, by the way, sorry. Quantity, not as important as quality of reviews. I have to say that again. Final word on reviews, you get what you put in, get out what you put in. Kind of simple, but worth repeating. How to influence people in the community. Technique zero, don't go over on your time allotted in a talk. Technique one, break work into small pieces. This really does, it's very simple, but a lot of people forget it. They go ahead and they put blueprint specifications and they push it out to Novus specs or neutron specs and they forget that a 1500 line specification is not going to be read by the same number of people as a 500 line specification. If you break things out into digestible chunks or morsels that people can consume, you have a much better chance of influencing people and getting your viewpoints across. Participate, goes without saying, but there's lots of people that go and work behind a wall and don't participate in the ways that the OpenStack community participates on IRC and mailing list. If you're one of those people who have lots of meetings with project managers and product managers and never interact with anyone in the upstream community, please don't complain that your particular feature request is not on the roadmap. You have to come out of the box and talk to people. Identifying the problem. Last point here is kind of important. Bring data to support your point. If you're making a feature request via a blueprint specification, make sure you include some data on that blueprint says hey, these 10 users have been asking for this in these different variations. Or 50% of my users experience this. Just bring some raw data. You can just make it up. Just bring some raw data to the table. Numbers talk. Identify natural incentives. Lots of people here work in competing organizations, but what they don't realize is that a lot of those organizations have similar aims. Find people in competing organizations that want the same thing and use them. Use them to naturally move an objective forward. Not everyone's an enemy. And then code talks. Proof of concept code works really well to persuade someone that your idea is worth thinking about. Now I'm going to pass it over to Ildakow. It's gonna say that everything I said is wrong. Thanks. Thanks. It seems that Jay has some female genes because he talks way more than I do. But I will try to finish on time. Forgive me if I can't. So first of all, he didn't lie in the last 25 minutes. Sorry. But I would like to note first that we are not able to give you the perfect recipe for contributing. So we cannot tell you exactly how to do that. You all have to find your own way in the project you've chosen within those community members there. We just tried to give you some suggestions that worked for us and inspired by our own experiences. But first of all, just don't give up trying really. It will be sometimes hard, painful or annoying, but don't give up. And let's see then how sometimes things go on another way. My part will be more a story than this of items. Please forgive me for that. So once upon a time, on a hot sunny day during the last summer, end of summer, we had some discussions with the colleagues that, oh, how good that was if the salameter queries could be improved a bit. Nothing fancy, just some tiny modifications. Okay, challenge accepted. So my journey began there. And then where it ended after four months of hard work counted from the design summit where I first explained the idea, you can see that it ended up in a rich query functionality, seven parts sets and more than 2,500 lines of code. So well, it took a while to get all of that merged. What we used during that four months, you can see I think all the items that are presented here was mentioned by Jay already. So the first evidence that he didn't lie to you, we use all these things. The journey began with registering a blueprint at that time on Launchpad. A note again here that right now, I think in almost all, if not all, the projects we use, a blueprint template which is much better than Launchpad because it gives you a guideline that what information you would need to think it over and provide to the community about an idea. So it helps not just you, not just us, but you a lot to know what you really plan to add upstream. And also mailing list and IRC just like how Jay mentioned we use that a lot. And then the first step, as I mentioned already to held a session on the design summit one year ago about the idea that we had. So we already had some directions to go to, but well, things are usually going on another way than you originally imagined. So by the end of that design summit, we ended up with a completely or not completely, but a very new direction to implement a complete rich query functionality. So the lessons on this slide is to be adaptive. And if someone suggests you something, consider it and think it through what are the cons and what are the pros on the side of the suggestions because maybe you will end up with a most featureful and better way of implementation of an idea. So just don't say that, no, no, no, my idea is perfect because it usually never is, at least not at the first phase. And what's happening after your idea was kind of verified and you know what you would like to do. Again, reflecting back to what Jay has already mentioned, upload some patch to Gary to show others that what you really thought to do because documentation is sometimes means other things to others than to you. In that phase, we thought it was kind of middle of December that we would like to get some feedback also, not just push some code on Gary and then wait for Christmas to pass and these kind of things. So we sent a mail to the mailing list and well, yeah, you can see under again. So the mail was landed on the mailing list with the reference to the blueprint and also the working progress patches already uploaded into Gary. Then, and the question at the end that hey guys, could you please review and share your thoughts with us, give some suggestions and feedbacks and the suggestions and feedbacks just came. So it was like a Christmas gift. The day of Christmas Eve was kind of busy for me and my colleague with whom I was working on this feature and there you can see that Jay's eyes always watching you. And that particular question was the say hi by Jay. That was the first. That was the very first. Yes. It's like a start of a romantic storyline. Are you using Outlook? Really? So yes, this is how our journey with Jay started. So yes, but as you could see earlier, he has a really good reason for mentioning that try to avoid using Outlook and HTML mails. So yes, but let's get back to the slide a little bit. The lesson on this one is to be always prepared because you can receive questions even on Christmas Eve. And it is better if you can explain why you were thinking to do something and why you chose that way, et cetera. So be always prepared to answer questions. And here we are in the middle of thunderstorms on our journey in this story, the but why part. I guess the most annoying for almost anyone. Let me share just a really short story about my life. I usually visit my mom in every one month but nowadays in every two on the countryside. And while she's a critical one, I'm her only child so you can imagine how hard that situation is. So for example, I can go to the hairdresser to get some new style, to avoid the questions like, oh, why are you so thin or why you're working so much? So I arrive home, she looks at me with that analytic face, you know, question marks in her eyes. And then, hmm, your hair looks good, but why did you choose that style? It's just I cannot see your face and whatever stuff. OK, cool. So what I can do in my everyday life or just similar situations in the community, I can ignore her, for example, but well, I really love my mom and we have a really good relationship so I wouldn't do that. So other option is to take a really deep breath, keep calm, calm down and put the emotions on the shelf. That's the most important part. And then, let's talk it through because even if she's my mother and she's really worrying, she can have a point because, well, look at me, I cannot see all the aspects. So there can be some places around that where she can have a point even if I haven't seen before. So let's get back to here, the but why part. So we had the Coden Garrett, the man on the mailing list already. And well, we had some really tough questions. And even a minus one on our first patch, I can tell you that's really shocking. But as I mentioned, as a first takeaway for you, don't give up because there is way forward. You need to be able to describe professionally that why you chose your way. So here you can see that we added a new endpoint with a new query language that can be difficult. And the minus two was because of the query language that we chosen. And then on a kind of bloody meeting, we had some discussions about that. And we managed to convince the other folks that our query language is kind of really, really similar to the one that MongoDB is using, which is kind of pretty much familiar to a lot of guys around there in the world. So they kind of got convinced by the end. Another major API redesign was that you see in the last line. We just managed to get some validation in the code in Python-based, nothing fancy. And then we got the suggestions that why don't you use JSON schema? OK, sure, a funny thing. I don't know. So we had to spend some time with figuring out that what that is, how we can use, would it burst the effort? And then finally, we decided to implement this change. So what you should remember from this slide that you need the reasons that why you are doing something, but also, on the other hand, you need to consider what others suggest, because they know other technologies than you. They have other experiences than you have. And also, you should be open to use new technologies. They can be good or better than what you are already new. And here we go over the thunders, but still in the storm. So this is the part where the principles are not really working if you are trying to introduce a bigger change, which results in a huge code set. So our kind of almost first comment on the first patch was that it's really huge, so therefore, no one is willing to review it. But that's a hard point, because we chose functionality over small patches, which is kind of good from that point of view that, with the code, you try to kind of show that what your idea is in real life. So when you have functionality and the basics in the first patch, then it reflects to your idea what is written in several hundred lines of documentation. So it will, at the end, help the reviewers to understand what you really would like to do. So sometimes it works to have a bit bigger patch, but with enough functionality to show what you really would like to do. And I have to mention that this 1,000 lines is with tests and documentation, which is really important. So I think it is true for almost the whole code base that we had more tests than how the actual implementation, how big the actual implementation was. So quality is really important. Try to add as many test cases as you can, not just the happy path. Also the negative test cases as well. It's really important. And the other part is the dependent patches. Sometimes, again, in case of a large change, it's kind of you cannot avoid. You can try to choose a good way of prioritizing what is included in each patch. So when you have a chain of patches and those are one by one is getting merged, then you will have always the next highest priority feature or smaller functionality in. And the maintenance, well, I'm also not a liar. It will be hard and painful. I don't know how many of you likes rebasing a patch. Personally, I hate that. So what you can do in this case is kind of wait a little bit when you get the review comments. So first, when you're uploading your kind of first patch or first huge patch, you're praying for getting some reviews. And you're pretty happy that you get a few minus ones. Cool, someone has already reviewed it. Great. And of course, you're a new contributor, so you're eager to fix all the review comments. But if you have several patches uploaded in a chain, maybe you can wait a little bit. So not fixing the first one, but several in a row just to avoid the constant rebasing of your code base. So it's still acceptable. Of course, don't wait a few months. That's not that good. But you can wait for having a bunch of comments to fix. And last but not least, over all the storms, the code got merged, and you decided to stay. How to do that? So I think most of the items here was already mentioned by Jay. So it's really appreciated if you do the dirty work, like bug fixes. It is really helpful if you can do code reviews. It helps you to understand the code, and it helps us also to have feedbacks on every patches uploaded. And one thing that I would like to highlight here is the improved documentation part. So documentation is kind of like the lonely child with several stepmothers and really, really many stepfathers here. And no one really likes to take care of it. So I'm really feeling sorry for this little child here. So it is really helpful if you find something that is not correct or not up to date, then just upload a page then that fixes it. So if you visit the OpenStack manuals repository and documentations, you can see that it's a large set of documentations, which is as much hard to maintain as the code base that it describes. So it is really, again, very appreciable if you also checks and helps to improve the documentation as well. And yes, it is really good if you would like to be the member of this whole open organization here if you try to show that you're really involved in improving the quality with the tests, with the documentation, all these things, adding valuable review comments. And you're also involved in the design questions, discussions, for example, on the weekly meetings that every project has on IRC, and also be present on the IRC channel of the project that you've chosen to be a member of and answer questions and be part of the discussions there. And that was what I wanted to share with you. What is my experience in the last one year? I've been working with the community, Jay. So I don't know how time is, yeah, one minute over. So well, if you have any questions, you can found us around during the whole week. You saw the IRC nicks of both of us in that nice question of Jay's. So the slides will be uploaded. And thank you for your time and attention.