 Give me six reasons why software development doesn't suck. So you can't do it. Everybody agrees that software development's. And there's a lot of reasons for this. One is like technical. Getting computers to do what you want them to do can be very difficult. But mostly, it's humans. Humans are really strange people. You got to deal with managers. You got to deal with meetings, scheduling, deadlines, all the stuff. And that's really what makes software development tricky. It becomes sort of a problem to have to deal with all this sort of stuff. And people have come up with ways to deal with this as best as they can. These are things with agile, waterfall, pair programming, BDD, TDD, all that stuff, whatever is popular at that given moment. And I've tried all of them. And if they're work-free, that's great. Because software development is hard. But for those of us who don't buy into the whole agile stuff, it sucks. Because I want to be that guy who's really into their process and into their workflow. Because if you're going to identify with something that works really well for you, that's amazing. And for the longest time, I thought I was just weird. I still do. And I wanted something that really works well for me. And I think to the point the last year or two at GitHub, we've sort of come down and figured out a development process that is really interesting. It doesn't feel like process. It's really quick. It's really fast. It scales really well. And that's what I want to talk about today. So this is not the safari. I'll sort of this. This is how GitHub uses GitHub to build GitHub. The secret here is that this isn't really about GitHub. Online, I use GitHub three times in the title. So it's sort of about GitHub. But what I mean by that is that this is not about GitHub the tool. I'll talk about GitHub the tool, because it's easy to use it as an example for stuff. But I really want to talk about our workflow, our process, and how we use our tools, and how we choose our tools, and why that makes for a really fun development environment. So I don't think this is really about GitHub. If I worked for Twitter, I would give the same talk. If I worked for Facebook, I wouldn't give the same talk. I want this to be a talk for making you think about how your company can use the processes and workflows that we use to improve your own product. So I'm Zach Holman. I'm Holman on Twitter, Holman on GitHub. I do work for GitHub. It's a social coding site. Employee number nine. So I've been there for about a year and a half now. And we're now 46, 47 employees. So we've grown a lot. And I've been able to be there for a year and a half and see what hasn't worked out for us, how we've changed our workflow, and more importantly, what has worked for us. What assumptions have we made in the beginning that we've been able to keep with us the entire time? So first of all, I want to talk about a little bit about how we work. Otherwise, this talk will probably sound insane. The big part of what we do is we work asynchronously. So what does this actually mean? I looked this up in a dictionary and said, you can do shit without needing to pull me out of the zone. I read real edgy dictionaries. But what's so important for us is that getting people in the zone is really important. This is the difference between going to work and you're not in the zone and you're trying to constantly battle and trying to force yourself to write code. And it's one of those cases where throwing hours at the problem doesn't solve things. With software development, you want to be in the zone. Because when you're in the zone, you can do as much in an hour as you can 20 hours out of the zone. So we want to get more people in the zone for our employees as much as possible. And to do that, we have no meetings, no deadlines, no managers, which is kind of insane. You can do it, too. Fire all the managers, you can tell them I said that. It goes back to this really works for us. It's a very fun place to work because it gives people in the zone as much as possible. So what do I mean by that? One, chat rooms. We do everything in a chat room. This sort of stems from when GitHub was a three-person company. They did everything online. We didn't have an office for the first two years. And this really brings us through everything. If it's you and me at the office at 2 in the morning, no one else is around. We're at the same table. We're still going to talk in the chat room. The reason for that is that, firstly, let's everybody see what's going on. There's a lot more visibility in the company. I can go out to lunch. I can go to Kansas City. And when I come back, whenever I got internet, I can see exactly what happened to the company in the specific areas. And it's a much easier way of letting everybody get involved. And there's no more secret projects where only, oh, that guy knows that project. You'll have to talk to him in a range of time because that sucks. Do you actually do that? Do what? Do you actually do three days involved? Yes. It depends. So as an aside, we have a bunch of campfire rooms. We have like 20 or 30 now. I don't read the danger room because that's all the animated gifs in Cuba at Rain Supreme. That's insane. But specific ones where I'm really involved with, like get-to-have-enterprise, I'll go back and read the transcript from the last few days and stuff like that. We want to break out as much as possible in separate groups. And then it becomes much more manageable to see what's going on in your domain of responsibility. And then lastly, time flexibility. Work when you want to work. This is really important. We have a lot of people coming to the office at six in the morning. That's definitely not me because I'm not productive at that point. We want people to be productive as much as they can. And some people are super productive in the morning. They cannot do work at night. And that's cool. I'm totally the opposite. And mandating that you have to come in at a certain time, leave at a certain time, just damages that. It brings people out of the zone. So I wrote about this three parts and stuff. You want to check that out? You can. But I want to get on to actual other stuff. Let's talk about pull requests and branching. I think this is a huge part of white GitHub, worse as well as we do. I want to talk about issues. I want to talk about OAuth as identity, hooks and hubot, and secrets. So first, pull requests and branching. This is the most interesting part of GitHub, I think. You guys have really weird branches. And I see the support requests come in. And someone details their process. And your job just drops in each paragraph. Like, how do you work in this fashion? And by this, there was one company I saw who has a master branch, makes sense. Then they cut a deploy branch every time they want to do deploy, makes sense. But then they just cut a deploy branch off of each individual deploy branch. So by the end of this, you have no idea where the hell you are. This is just insane. It's overly complicated. You don't need to do this stuff. More likely, I see this a lot in large enterprise companies. You have a repository that you work on, and then you have a group of trusted developers. These are the guys with full rewrite access to the repo, and they can do whatever they want. And then you have a group that's, I call them shitty developers, because they obviously must be shitty if they put them on the separate side, and they only give them read access. And they're trying to set up all these arbitrary guidelines, and it just seems really insane to me. So what you have is they all have to forfeit their own project from the main repository. All those forks then have to go back to the trust developers for them to figure out, OK, is this good to go back to the real repository if it's not, then you've got to send back to the shitty developers, and then they have to pull down new changes to the repository. If you see a chart like this in your organization, something is fucked up. Keep your branch as simple. I think this is really important. This is fundamentally important. This is sort of the foundation where you build your product. All right, I'm trolling you now. It's just like there's so much in you as that. It's not a solution. You mean the stuff I was going to cover in the next minute? You know, you haven't talked yet. I'm going to troll your ass. So keep your branches simple. Branches should be simple. Let me talk about how we do branching, because I think obviously we have a perfect branching situation here, Dr. Nick. What we have is a master branch, and then we have a branch where you work on a bug fix or a feature or anything like that. When you're done with that, that goes in a master. This is it. That's the entire branching strategy. We don't really branch off of separate branches or do anything more complex than this. And it becomes a lot easier to bring on somebody new and just say, oh, you cut a branch. When that's ready to go in, push the master. Along with that, everybody can push everyone to deploy. Our designer on their first day can deploy to GitHub. We have a lot of automations surrounding deploying to the 50 plus servers we have now. And that's good. We want people to have responsibility over the code that they actually write. Too many times, you have something where like a release manager, where you push it to the release manager, and then that becomes their responsibility for the stuff. We want people who write the stuff to be responsible. This also frees yourself up from micromanaging other people's code, but the trust developers and the shade developers part, you want people to be responsible for their own code. For us, master is always deployable. This is like the one hard rule that is constant at GitHub. If you push the master, assuming it can be deployed as soon as it passes CI. The reason we do this is because we deploy 10 to 40 times a day. We like very short changes. And if something breaks, you know exactly what breaks really quickly. If you're nervous, you can deploy to staging. We have a separate staging environment. We can also deploy separate branches, and test that out, and then deploy to master afterwards. You can also deploy to a subset. So if you could say, let me just deploy to two boxes right away, and then see if anything blows up. If it doesn't, then merge into master and deploy from there. This is also sort of Twitter-driven development, because you guys are way more on top of things than we are sometimes. And people will tweet constantly if something breaks. Now, we have grass and stuff where a number of exceptions and then deploy is really marked on the graph. And then we mark that against ad replies to GitHub on Twitter, and it's actually very proportional to if you break something, people will talk about it. So keep your branches simple. That's really important. I know what you guys are thinking. What about code quality? If you have a fucking designer in there messing with your code, no one likes designers. They do weird stuff. So how do you keep up code quality? And our response to that is pull requests. This is central to everything we do. Pull requests are a code review. What are pull requests? Pull requests are a discussion. It's a simple discussion about a piece of code, a distinct piece of code. It's a discussion about codes, a discussion about that particular feature, a discussion about that particular strategy. All of this stuff comes together. You can do a bunch of different stuff to it. You can reference issues in it. You can reference other bits of code diffs in it. You can write code comments. It's all about figuring out how you make a discussion on a piece of code. And then from that point, you can figure out, do you want this in the actual code base? All this stuff happens on a separate branch. So none of this stuff ends up being dangerous, because it's separate from your production code until it's ready. This is what it looks like. I'm sure most of you guys actually know what pull requests are. But it's title, description, bunch of commits. Once those commits are done, you can discuss on those commits, and discuss them more, and then you commit, and then you get merged back into the master or whatever branch that you're trying to merge it into. The pull request itself is broken out into a discussion where you can actually talk about the changes, the commits, where you can show all the commits, and then the diff, where you can actually see what has actually changed in the code. And this is really cool, because we're really large GitHub at this point. And no one person can really dig into every line of code that's changed. But it's important that you have a lot of people who know what you're doing. So for a lot of pull requests, I won't look at the actual code. I'll just look at the discussion and see the title description. And that's a good way. It's like an RSS of what development is happening without having to sit down and do a stand-up meeting or anything like that. I can just see it in my inbox as it goes by, and it's very simple. So I lied. This isn't actually our development strategy. It's more similar to this, where you branch off, send a pull request, and then merge it into master. And that's the guard against getting really shitty code deployed accidentally. So big secret number one, you don't need to fork anything. This is sort of our common misconception. You can do pull requests between branches on the same repo, and that's what we do consistently at GitHub. We don't fork the main GitHub repo all the time. We just do pull requests between individual branches. And then you have to set up weird permission strategies to let people under your different forks and stuff. It's just a much simpler way to do things. So wire pull requests actually rad four reasons. One, they're asynchronous, no meetings. Everybody's had that time where you're in a code review in person, and you're projecting code on the wall, and you're trying to figure out, OK, you're running it through your MRI in your head, and you're trying to figure out, OK, this compiles to this and does this. It's a terrible way of reviewing code. With pull requests, you can easily pull down the diff and run it locally. And you can do that whenever you want. You don't have to have a separate meeting on that stuff. It's really helpful to have it broken away from some sort of synchronous, like, this has to get reviewed right now, and stuff like that. Secondly, email is a good interface for this too. When I wake up in the morning, I can see all the pull requests that have been generated from Europe, for example. And so again, it's just a quite easy way to figure out what's going on in the company without mandating some stand-up, which doesn't even work when half your team is across the world. They're also really accessible, and we use this a lot. Designers evolve entire mock-ups in a pull request. And that's really powerful too. It's version control for mock-ups. You can see, as you go along, what the mock-up has changed over time. You can discuss on that too. We also have a bunch of non-technical staff. We have a shipper who ships out all your beautiful shop stuff. And she can go in and check out a pull request. This is going to change to the shop.github.com site. Can you actually say, does this make sense? Yeah. She doesn't have to go see the code diff, but she can see what's actually happening and check it out. They're also historical. They can be experiments, and they should be experiments. We have a lot of pull requests that are designed not to ever get deployed because we want to see, OK, what do you guys think about this interface? What do you think about this API? Does this make sense? If it does, then I can just close the pull request and then actually build it for real. Don't be afraid to try something out, throw it away, because github retains the history. And they should be really fun to make. You should not have to fear about doing some insane experiment when you're drunk and seeing how it works out. So I want to show you just a couple of quick pull requests that we do at github. It's a little bit more context. Status.github.com, there was a problem a while ago, again, where you guys caught stuff before we did. Or more correctly, we would get the notifications that something is down, but it took us another couple of minutes to actually update the status site. So I added something just real time pulling of different services on github.com just to see if they're up. And this is a pull request. I started out just with a bare bones interface, and I just dumped this on a page. It would check this, and I didn't have to care about styling or anything like that. Because I knew once I posted this, somebody would come in and be like, I want to design that. And that's where we have a lot, where designers are just going to see what's happening to the company. And when it's ready for them to come in, they'll do a design. At that point, I have a design, and I can build on that, I can commit to that design and then just match it, and it's very easy for me. I never have to worry about that in the slightest. And there's no upfront meetings. It just all happens very asynchronously. By the end of it, we close it, merge it, ship it, and that's it. It's very simple, very easy. You should not be afraid to mix your designers and developers. They should work together. Just like pull requests are kind of scary from a designer's point of view, but they don't have to get into the nitty-gritty if they don't want to. It's helpful for them to get into the discussion section. Don't be afraid to post screenshots a lot. You can use this in the Markdown, which I'll show you in a couple of seconds, but it's very good to be very visual, even on back-end stuff. You can also post gifs, because they're funny. We do that a lot. We also have something called code editing, or it's a syntax highlighting in your code editor. And this was earlier this year, and this entire pull request. We did all this stuff, did this bunch of discussion, and then we're like, fuck it. It doesn't work. We can't do it. Close it, and that's it. That's important. That's still really good, because they're cheap as hell. You shouldn't be afraid to experiment. That seemed really kind of a bummer at first, because we really wanted this to work out. But the library we were using didn't work out. Don't be afraid to toss it away. This was great because five or six months later, we found a different library that did a lot better, and integrated that, and worked how we wanted it to work. And we could go back to the original pull request and say, what was my design decision here? What did I want to do here? And then you don't have to reinvent the wheel every time. So really play around with pull requests. This is a pull request that started in the beginning this year, hasn't shipped yet. It's got a bunch of stuff in it. And that's really cool. We can hire somebody tomorrow. And if they're asking what they want to work on, we can point them to this. There's no back alleyway deals about communicating between two people that then you'd have to chase down somebody else, and be like, can you please tell me what your thoughts are on this? It's all right here, and we can go back later and check it out. Get your done, don't waste your time. This is something that you have to constantly worry about. Can you improve your workflow? Can you make it a lot better? And it's something that you have to do constantly, because otherwise, it'll sneak up on you and really fuck you down the road. So recap, use pull request more. They're awesome, ship a lot more. I think they're really great. I think they're really important. And if you are able to do them in your company, I should investigate them. I want to talk about issues. Everybody has issues in different companies. I'm talking about tickets, bogs, stuff like that. So everybody's seeing a screen like this, like a new issue create screen. The problem with this is there's a lot of stuff going on here. And something makes sense, and you're like, OK, yeah, that's cool. And then they just layer more and more stuff on there. Before you know it, there's some fucked up shit going on there. And you try to figure out what is the point of all this stuff, duplicating priorities and stuff like that. This is the one on GitHub.com. And this is not, again, about GitHub. This is the tool we choose internally. All it is is title, description. And we do away with the whole bunch of priorities and craziness, because for the most part, you don't need them. Ask yourself, can you survive on simpler tools? Can you work better and faster on simpler tools? I think you can. Merlin Mann, I sort of have a man crush on him, Twitter crush. He's amazing. He wrote this awesome post on email priorities. And he said, priorities was not manufactured or signed. Otherwise, it's necessarily not a priority. Making something a big, red, top, top, big, highest number on priority changes in nothing but text styling. If it were really important, it would already be done, period. I think that's really important. He's talking about email, but the same is true for stuff across the board. Issues, prioritization, dragging, dropping, like all these issues to prioritize and stuff. Think about it if you actually need that. If you do, that's cool. Then use it. But if you don't need that, you're wasting a lot of time. Look at all the stuff that we have as developers. Ask them to get rid of all of it. Resist, adding meta work, it gets in the way of real work, the real work will get done. Quickly, we use issues for three things. Should something broken? Something's broken, that gets fixed in a few hours? That's it. We also use issues a lot for, wouldn't it be cool if, and then it's another place to have a discussion. This is prior to the floor request, because you don't want to throw it on code yet, but it's fun to talk about. We also use this to-do list sometimes. That's just kind of an easy way. You don't have to set up a weird issue tracking, to-do board or something like that. It's all just in one issue. Do you need more than that? Really think about that, because most of the time you don't. Sometimes you do, which is fine. But most of the time you don't. Simple tools means a better product, because you can build it simpler. OK, all off his identity. This is fun. I love Ruby. I really love Ruby. I'm not talking about the speed of Ruby, because it is slow as dicks. I'm talking about how fast it is to build something. Like, it is really fast to build something. So I looked through my development folder about what projects I have in there, and I have one called Secretary of Labor, All Caps. I was really drunk when I wrote this, and I saw in the next morning it's just a single Sinatra app, and it's got a stuff-referential sim link called What That Points to Itself. I have no idea what it does. I have one called Unmarked Van, which tracks people's physical locations. I have one called Trace, which is a real-time stats graphing in Redis. The point is, all this stuff was really fun to make. It was fun to experiment. None of this stuff is ever shipping, especially not that first one. But it's fun to try stuff out. And we do this stuff at GitHub, too. We have a bunch of internal apps. We probably have more than this at this point. But some of them are more fun. Some of them are very serious, CI, stuff like that. We have an internal Twitter, which is more fun, status updates, and stuff like that. The point is that we want people to experiment. It keeps things really fun. It keeps things interesting. The problem with all of this is that outsiders aren't welcome. Like, we're a really open company, but you don't want some people just trudging through all your stuff. So yeah, that's some authentication layer. For us, we use GitHub for that authentication layer. We have all of our teams. We have our orgs. We have everything in GitHub. So we just use OAuth to attack that and use that as our authentication in all of these separate mini apps. You guys probably all seen the screen. Allow them to check out your profile. Yes, no. You can use this in a bunch of different ways. There's a whole bunch of different Ruby OAuth libraries. This is one done by Atmos. All we have to do is just check whether or not they're a member of the GitHub org. If they are, let them in the app. It's very easy. If we hire somebody, we don't have to update the 30, 40 different apps we have. It's all in one place. It's just a nice, consistent way of doing things. Think about if you have this problem, can you pawn it off on somebody else? And that means GitHub. A lot of people have Twitter now. You can use Twitter's authentication for that. Or if you're big enough to have your own internal LDAP server or whatever you got, just don't reinvent the wheel. It's good to use a consolidated feature for all this stuff. Okay, let's talk about Hubot. Everybody know what Hubot? So three. So Hubot's our friendly campfire bot. He's cool. He can do some stuff. We have like 300 commands for us. You can deploy over GitHub app. This is important when you have 50-some servers because you have no idea how to deploy all that stuff. So we just say Hubot deploy X to production and he does it. It's great. You can run branch-level tests. So if you push a branch up, that will just run automatically. You can play music in the office. You can tell us who is in the office based on their Wi-Fi presence, on their MAC address, on their phone, or their laptop. You can build usage graphs like you can chart all those exceptions versus deploys versus Twitter mentions, send received text messages. You can mustache every image posted in chat, like that. You can track who swears the most every day. You know, we can do all this stuff. And it's more fun because it's a shared side project. Again, it's fun to have experimentation. Use cool technologies. Stupidity is not open source. I just thought it was it. He's not open source. As of last week, it's awesome. We have tons of adapters now. You can use them in IRC, Campfire, XMVP, and tons of stuff. So check them out if you haven't already. This is a really fun project you have inside your company. In terms of how we use them for GitHub, we use them in a bunch of different ways. Specifically, we use them for branch status a lot. You can ask him who about what hasn't been deployed, and he'll return it at compare view of what hasn't been deployed yet. So you know, if you're gonna deploy this, these are the concrete disks that haven't been deployed yet and be worried about these if you deploy them. We can do this on a different branch too. So if you're gonna deploy a separate branch to production or you want to know what on that branch has been merged into master yet, you can do all this. And it all stems from the shot, which we know what is the currently deployed code on GitHub because it's all in this one site. It's just an endpoint and you can use this all over the place like CI, stuff like that. It's just a nice helpful way of doing that and you can build it into compare views and it's fun. You can also use a lot of the API in GitHub too. I can ask one of the pull requests on my particular repository and I'll send it back all the pull requests. Very simple, very easy, and it becomes a nice way of setting up a to-do list. If you're ever like, what should I work on today? This is a nice way to start. We also have the same thing for issues. We have bug days every like second Thursday or so, just as a way to clear out old issues. And this is kind of fun. You can ask it how many issues do we close today and respond back how many issues you close today. It's another way of sort of, I hate the word, but like gamification of like your work. It's fun to be like, oh, can we do more issues knocked out this week than we did the last time? It's fun. Think about what you can automate at your company because the stuff that really sucks are the things that you have to repeatedly do over and over again. And we're developers, it's fun to over engineer stuff sometimes and like you'll spend way more time automating something than you think you'll get out of it, but you'll actually get a lot more out of it than you probably think. Plus it's really fun to do. So the moral of story, who by knows where you live? I don't know, he's freaky. Finally, I just want to talk about some secrets. I want to blow through these because GitHub's got some funny secrets. We talk stuff away sometimes, our bad. Here's some fun stuff you can have. Emoji, all those fun things. We do these on pull requests all the time. We just plus one stuff. We say heart. I don't know, it's fun. It's a fun way of getting people involved on stuff really cool. Go to that link to see a whole list of emoji and stuff like that. All my slides are up on my site later. You can check out these links and stuff later. Shortcuts, if you guys don't know all this stuff, W will bring up the branch selector on a Rebo page. T will do the finding projects sort of like the TextMate finding project really quick. And if you press question mark on any page, it'll bring up the help dialog. And there's a crap ton of shortcuts you can do to do a bunch of different stuff. Issues, for example, has a whole bunch of quick access to issues. Images are awesome, especially when they're animated. This is the markdown format for posting an image into issues. This will post the image. Unfortunately, this only works if your dorky dodgeball team is the image. I'm just kidding. That's our dodgeball team. Check out those shorts. Amazing. The other one is, what? Who won dodgeball? I can't remember. Is that some Heroku guy who asked that? Heroku won dodgeball. We raised 50 grand. It was great. For charity, not for Heroku. Also a different patch. I mean, Git was based on a mailing list point of view. So all of the stuff that we expose on GitHub is also available in text basically. So just add .git, .div, or .patch after most URLs and we'll spit out a nice thing that you can directly need to get later. The other thing we have is friends code. This is sort of a, you get the formatting of any code that you paste into issues, pull requests, stuff like that. All you have to do is triple tick and then say what language you want and then all syntax highlight for that language. A nice quick thing to do, especially do it in readme's because readme's with just blocks of code in them kind of sucks. It's nice to have that syntax really highlighted. Mentions, we use this a lot. If you're gonna open a issue, we'll usually add like cc, add technically or add defunct and then that will notify them and get them to put their eyes on your pull requests. It's a nice way of just having communication amongst your pull requests issues, stuff like that. White space, you can add question mark w equals one to reduce white space and diffs. Sometimes it's really helpful if some guys have pull requests with a whole bunch of white space changes and you're like, I don't even know what you're changing. Quick and easy way to do that. Rebo references, you can reference issues and pull requests across different repositories. So you just, in your issue body, you just say like user slash repo number sign issues and then that will bring in the link on the referenced issue and it'll just connect the two visually in the webpage, it's kind of nice. I have a pseudo announcement right now. We have a URL short that we've been using for a while now. I'm announcing this here because I can. So if you have a really long URL, you can shorten it to a git.io URL. You do that via this, it's a very simple service. The red URL down there is actually a gist that actually shows you how to set up a shell script to do all this for you. So if you ever want to tweet something or have some shortened URL or if you want to snag one for your vanity URL, github.com slash holman, stuff like that, that's a nice way to do it. Check out all these links in my talk later slides. So as the recap, really think through this stuff. This is the stuff that will bite you further down the line unless you really think it through each time. And the problem is that you don't notice it being a problem until it's too late. And at that point you can't go back and fix it. So it's gotta be a constant battle of like, what can I improve? How can I make my developers happier? Do I really need all this extra stuff? And then I think you can make cool products. So thanks.