 Thanks for coming right after lunch. I realize it's a very short time in between. I'll try and speak slowly because I realize that I speak Japanese, so I know how hard it is to understand someone speaking in a different language. So let me know if I go too fast. So, my name is Chad Furman. I'm the Ansible Senior Principal Product Manager at Red Hat. But before I came to Red Hat, I was actually a customer for a very, very long time. And what I'm going to talk to you guys about today is building a community of practice. Why you build a community of practice and getting people engaged in open source and all of the different components around that. So just a little agenda. Why you actually want to build a community of practice? How we did it? How we did enablement via hackathons, been billathons? How we did examples. So instead of just telling you about the things, I want to show you some of the things that we ran our customers through. So how open source works, and I like to talk about it via cookies, because who doesn't like cookies? And then talking about inner sourcing, because I think open source is fantastic, but a lot of large companies' inner sourcing is very hard to do. And then the last example is my favorite, Get 101, because a lot of people don't actually know how to use Get. So why did we do communities of practice? And it's funny to me being here talking to you as someone from Red Hat, because I actually got this slide from Red Hat when I was a customer. So I was the enterprise architect at Exxon Mobile, and that's why I'm here talking about how we built communities of practice at a Fortune 5, and Exxon is not a tech company. Although they'll argue with you they are a tech company, but their tech doesn't have anything to do with computers. Their tech is about how to take petroleum out of the ground and make it into products. So whenever I approached some friends of mine at Red Hat and asked them why did Red Hat build a community of practice? They gave me this slide, and I've kept this slide with me for almost seven years now. And it actually was exactly the same problems that we had. People were super disconnected, so disconnected from people on their teams, disconnected from other teams and silos, and they didn't know how to do a lot of things, and there was this tribal knowledge that you always had to go and find a specific person to do things. So for us it was a guy named Lance. Lance knew how to do everything. Lance knew where all the bodies were buried and how to get everything accomplished, but unfortunately Lance didn't scale. And then we had a lot of different problems with support, whether it be support from a managed service provider, support from, like on the Red Hat side, when someone would call support, and getting the experience from the customers to the developers and understanding what people are coding for is hard. And then of course, support and engineering teams have no idea what's going on until there's a crisis. And I could talk about this from last week when I had one of my engineers come to me and say, oh, I didn't know this was one of our customers. I shop there all the time, but unless people know the customers they're working on and what their product is for, it's really hard to actually properly engineer something. And then of course, lack of sharing, lack of communication, and getting things across regions and business units. And one of the reasons why I'm in Japan this week is to work with a lot of my counterparts here to help them understand a lot of these things as well. And there were so many silos. So many silos. Everybody is doing things in their own spaces, and these are the four big silos that we dealt with. But no one from network operations ever talked to SecOps. And no one from the development teams even knew what ports their applications were running on. When I deployed OpenShift there and started doing Kubernetes, I kept having developers coming to me and be like, well, this works on my laptop, but I can't get it to work on Kubernetes. And I'd be like, well, what port does it run on? I don't know. What language do you write it in? Node? And I was like, please don't tell me it's running in dev mode. Yes, it's running in dev mode. But getting the dev ops folks and the net ops folks and the IT ops folks actually all starting to work together and collaborate and talk to one another was huge. So here's how we did it. And each one of these bullets, I could probably spend an hour talking about, but building communities of practice was the very first thing we had to do because we realized that we had a lot of people writing Python. We had a lot of people writing Java. We had a lot of people writing Spring Boot and all of everything in between, even down to COBOL. But nobody had a centralized way or methodology to actually work together on these things. So getting people to start sharing their code, and that's the next one on here, having sharing sessions and demos, having weekly calls or bi-weekly monthly sessions where everybody just gets on a call and even if it's just a recorded demo and showing what they did before and getting people to ask questions. I can't tell you how many times people would say, well, I just can't get Python to work on my laptop. What do I need to do? And then somebody's like, well, have you ever set up a virtual environment? Have you ever tried using a container? Have you ever tried using a container on Kubernetes that has all of the Python dependencies that you need and getting those people to talk about that? And it was no longer Lance. Lance became 500 people because it was a community of 500 Python developers internally that were sharing code and showing each other that whole open-source ethos of people sharing and contributing and working together was so important to how we started being successful. But on top of that, we actually got, I think this is a big one, we started having centralized standards. So instead of everybody just going and creating their own virtual environments and Python containers, we started creating repositories in Git that had very standardized ways to build the containers. So instead of everybody going and choosing Python version this and library version Y, why don't we just build a container that everybody uses? So going forward, one container for Python. Here's your Python 3.11. And then that moved on to other things like Terraform and Ansible. And we started building pipelines on how to build VMs on VMware, which my brain kind of heard at that at first, but building VMs is something that's not trivial, especially in a large company that has very large security environments. So having best practices and wikis, but then getting people to contribute back was huge. So the biggest question I always get is, well, I don't code. I think everybody in open-source gets this. I don't code. How do I contribute to open-source? Put it in an issue. Let us know that there's a problem. You don't have to be able to code to contribute to open-source. And that's when we... We opened the floodgates a little bit because everyone started putting in issues and we had so many issues that we actually started having to build up our DevOps teams to be able to support all the issues, which wasn't a bad thing because it actually became that we were solving so many problems and things were happening faster and faster. So after that, we started getting into code reviews and paired programming because a lot of those people that were developers, or even not developers, started being like, well, I can start writing docs. I can start looking at the code and figuring out what the code is doing. So they would actually sit with the more experienced people and they would become the next generation of experienced people. And then this also got us to the point of having a reusable code, which was unheard of in an oil and gas company because there were people there that would send me code and emails. Has anybody ever gotten code in an email before? Yeah, it happens. It's not very sustainable, though. The best part was, is the guy who wrote that code said that he had written it in 1995. And then now we actually get to the point where you're actually becoming a community and you're interacting with the community, creating pull requests, starting to actually communicate with one another, not just putting in an issue and letting it die, but actually going and contributing to the code base. And then we really evolved into this thing that I didn't know was a thing at the time, but we kind of got what we started calling citizen developers and citizen data scientists. So it wasn't people that were by trade engineers or developers or data scientists, but they were able to go and look at the code and make small tweaks to it to make it better to do the things they needed to do. And the fact that they were able to do that in Git is because we ran hackathons. And the hackathons were basically us just getting people from all walks of life in a room and teaching them very simple open source concepts, like Git. Source control is kind of the way in the door to open source, and that was huge for us. And it even evolved down into chat ops where we had people that were just sitting in Slack or Zoom or Teams and name your chat operation and people just asking each other questions all the time. And so I like to always break down the way we did hackathons because it sounds easy when I say it out loud, but there's a lot that goes into building out a hackathon. It's not just like, hey, let's get everybody in a room. We had to work, everybody ran an agile, so we had to work in agile sprints, talk to the product managers, talk to the scrum masters and make sure that we actually had time inside of the pie increments to do the things. And a lot of what we would do is arrange with them to make sure that whatever we're going to work on in the hackathon, that was something in their sprint, so they're actually delivering something that had value to the company. But first, we kind of weasled our way in a little bit by starting with basic, super easy skills. So teaching people CICD, teaching people Git, teaching people applications like Terraform and Ansible, and actually getting into some of the more security aspects that you need to think of, because a lot of times, especially in big companies, security is the most important thing to get anything into production. So how do we actually create a CICD process to get the security done? And so instead of them having to understand all the security bits, we created a pave path code base where they could be like, here's the security CI you run it through, now you can get into production. And then the next part was, so how do we apply those skills to business problems, like I mentioned, but at the end, we actually, the way we judged, which was kind of fun, was who could get this into production faster. And really, if you could say you're getting your, whatever you built into production in the next week or two, you win. And we actually had some pretty cool prizes. I think somebody got a MacBook at one point. And if you download this, there's actually a link to way more logistical things you need to think of down to food preferences, logistics for size of venue, all of the fun things that you have to think about when you get into that. So let me just give you, walk you through kind of an example, and this was my favorite one because who doesn't love cookies? So the first thing about open source that I had to teach people, because again, this is an oil and gas company, what open source was, was what open source means. So closed source, obviously you bake the cookies, package them up, it puts Oreo or name your brand on the outside of it, and you sell them, and nobody has no idea how you built that. Whereas open source, you bake them, you package them, put a brand on them, but then you put the entire ingredients on there. And this seems kind of simple for all of us that are at open source summit and probably have been in the open source world for a long time, but getting this methodology into these people's thought process and how they're going to be actually getting to contribute to open source at some point took about five years. So the other part of this was the rules for the recipe. So free to use, modify, and share the recipe as long as you agree to bake it in compliance with all of the things. You share your modifications with the community, and of course you agree that you will not hold the baking community or recipe maintainers accountable for the consequences of cookie consumption. So if you make bad cookies, nobody can sue you. This is kind of the same slide of the previous one, but it's very much something we had to force and push quite a bit. But this was the fun part. How does this benefit people? And I don't think a lot of people understand this from open source in general. I work for Red Hat. If anybody ever asked somebody from Red Hat the first thing we're told at Red Hat is don't tell people you sell free software. But at the end of the day, we sell free software. But the benefits of open source and why we do this and why we contribute to the community is because it makes things better. That's the whole point of open source. And I remember the days of very, very hard coded things where you could never go and see them. You can see what's in the code. You can make the code better. And you can learn things. And this goes back to the discussion of how do I go from doing an issue in GitHub to a pull request to becoming an engineer or a developer because I've spent that time to look at the code and you cannot do that with code that is not open source. And so how do we open source our recipes? This is where the source control piece comes in but I didn't ever tell them about source control until later because that was scary. As soon as you started talking about Git, they were like, oh no, no, no, we're not developers. We're not developers. But showing them where they can go and see the instructions or the recipes of how to get where they want to go, how they can contribute via issues, pull requests, et cetera. And then how people can reuse that same recipe. So back to the security thing I mentioned earlier. Nobody ever wanted to do a security pipeline for any application that they deliver. They just want to deliver code as fast as possible. However, if you have that as a reusable cookie cutter recipe that everybody can just put into their CI pipeline, everybody's getting their security right from the start. And then that's where I went into inner sourcing because I had to build an open source office while I was there as well. That took about six years to get to. But first, I was really afraid with all of these new engineers learning about open source. Let's figure out how to do this internally before we try and do this externally. So getting all of those centralized workflows, all of the CI CD, all of the APIs and security artifacts, pushing it into Git, running it through pipelines, getting it to the point where people can contribute and start feeling ownership of the things, and then getting all of the squads to reuse code. Again, about three to four years to get everybody there. But we got to the point where people were comfortable enough that they felt that they could put it out on the open internet, which is kind of scary. I don't know how many people have contributed to open source in their lives. Was your first time a little bit scary? Depending on the community, some of them could be more harsher than others, but it was very similar for them, even internally, getting to the point where they didn't want Lance looking at their code and being like, why did you do it this way? But getting comfortable with being uncomfortable, I think as part of open source, is getting people comfortable with the fact that someone may ask that hard question of why you did it this way, and that's just going to make you get better at delivering code. And then my last, my fun one, which those have contributed before will laugh a little bit, Git 101 was the most important part of every hackathon and every enablement session that we did. And it makes sense if you think about it from, a lot of people know how to clone, people may know how to fork, they probably know how to push and maybe do a merge request, but what happens when it goes beyond that? Has anybody ever had to cherry pick a branch? How was the first time that you did that? Yeah, how many people have deleted and then just re-pulled down and then copied and pasted things? Yes, exactly. So a lot of it was showing people the proper ways of doing that and understanding how to look in the logs and how to go and pull the right things and look at the hashes and all of the things that when you're more seasoned at this, it just becomes something you do every day. But it's not something that these people have ever even thought of. And all of the information of how we ran that hackathon is actually available here. I've actually created a Git repo of it. It's about an hour and a half, and it includes the presentation as well as all of the source code to do it, but I really enjoyed it because it's not something that I got to do every day anymore because I'm a product manager now, I don't write codes like I used to, but there's so much intricacy to Git that unless you do it every single day, I mean, I'll have to go Google how to do a cherry pick. It's been a while. Of course, I work in the Ansible BU, and it's funny because I ended up coming to Red Hat to work on Ansible because there were things about Ansible that I wanted to make better. So that's why I'm here. But this is kind of an example agenda that we ran with, it was actually one that we ran when I was a customer and then I just recently ran it with another customer about six months ago, but just to kind of show you what it looked like for a full setup of this. The logistics are always the fun part, but just starting off with the Git 101, going into like 101s of how to use the tool, replace tool here with whichever tool you want people to learn how to use, but then taking the things that they learned and applying that, and that's the biggest thing when you're building these communities and teaching people how to do new things is you can go and learn things on the internet all day long, but unless you actually apply it to something that's meaningful to you and meaningful to your job, you're not going to remember it. I would love to say that I would, but unless it's something that I'm going to do again and again and again and make it part of my daily workflow, and that's why we call the second part hack and run, because you want to learn something and then apply it. I go pretty fast, but if anybody has any questions and wants to dig into any of that, please feel free. I also work in large company. One challenge I can see within our sourcing is that as soon as you open to other teams the software you did for your team, basically you have to support the other teams, and that's not at all what your team is made for. How do you handle that? That is a very good question. I would love to say you add more people to your team, but we all know that's not an option. So the way that we handled it is a lot of it was, I mean, volunteer work just like open source. It was people from other teams wanting to contribute. Sometimes there were issues that sat out there and sat for a very long time. We had to prioritize and go through and figure out which ones made the most sense to work on, and sometimes we had to say, unfortunately we're never going to work on this one. Yeah, but what if some feature of your software may be misused is causing a production issue? Now you're in the line of resolving that. That's, again, a priority. So I can say that as somebody who works with an engineering group very often, prioritization is on a weekly basis. Sometimes if you may be working on a new feature, and that new feature has now been deprioritized because you have an operational issue that you need to fix for a customer. So absolutely, if you have X amount of engineers and Y amount of work, there's an equation there of... There's no good answer to this, unfortunately. But yes, I work with that problem often working with an engineering team. Good question. Thank you for your presentation. Thank you. And then I have a question about the answer of a navigator because before we are using the answer of play command for playing the playbook. But nowadays it seems like a shift to the answer of a navigator is correct? You can do it both ways. Yeah, but it's a series of my customers, they wonder like, so read it. So it's going to be like, don't use the answer of play of command or document command anymore in the future. We all still use the answer of a playbook. It's fine. Because my many customers, they worry about that. Yeah, it's not going to be deprecated. Okay. So, I mean, you kind of spoke about the hackathons and the cookies and that was interesting. I'm kind of curious, when you started this engagement, which was quite long, you said five years for this, six years for that, four years for that. How did you engage with kind of middle and upper management to create, because I would assume that you would need to change the incentive structure at the company. Otherwise, like people are like, oh, why are you teaching this other team Git? I mean, kind of the same problems that open source struggles with, with maintainers, unpaid labor, et cetera. So I'm kind of curious if you have anything to say about that part of creating this at this company. I talked mostly about the bottom up. I probably should have talked a little bit more of the top down. So you absolutely have to work with management. As much as none of us being engineers don't want to work with managers, we have to sometimes. And making sure that they're on board. So a good part of the whole, we were in the middle of a digital transformation because I think most companies were in the 2016, 2017 time frame. Lots of companies still are, don't get me wrong. They did not change incentives there. I have worked with people that I know of since then that have very much changed the way their incentives worked. But what it came down to for them was SLAs and KPIs. So of course everybody wants five nines. People wants to deliver things faster. All of those, all of those buzz words we talk about. But the reality of it was that first hackathon that we ran, I was able to show them in one week how we could save the company five million dollars. And it was basically because they had 25 people restarting services on servers. So three lines of code later, we fixed that. So there is so much that needs to happen at the management layer as well. It can't just be grassroots as much as I would love to say that it can be. There has to be people, at least manager, director levels that are backing you on a lot of these things. But I have oodles of slide decks that I could show you of having to explain. That's why I take it to cookies. Because everybody understands cookies. Hopefully everybody understands how to make cookies. But putting it to a level of simplicity to help them kind of see where that goes and to explain that into a management level. Because managers don't know about Python or Python versions or Java or any of those things. They know about business value and the deliverables they have to their business. So applying things to a business level and showing them the KPIs and SLA's that you can hit or exceed. So what that came from, so back to the kind of the service explanation, those services were for one of the security tools. So now security incidents were much better taken care of because we weren't killing services. So that SLA was shown. And I think you can apply that to many things because once we started automating lots of things it was no longer, oh, it takes six weeks to get a VM. It was, oh, it takes six minutes to get a VM. Because automated network, automated storage, automated virtualization, Kubernetes namespaces, all of those things that used to be a requirement or a request to six different teams became a simple ticket request that you got something as a self-service. Very good question. Thank you for the presentation, Chad. As you build the communities of practice over the years, what sort of metrics did you check out or did you keep track of to measure success from the programs? And that question always comes. Or the other part of it is you followed the tyranny of metrics and no metrics at all. I've seen all of those. For myself it really depended team by team and what that team's particular metrics were. And those changed because the way that they delivered things changed. So again, the biggest hurdle for everything was getting resources, whether that be people to work on things or VMs or storage or networking workspace. There were so many different silos and pieces that they needed to get things that we had to change the metrics once people got everything to self-service and it used to be I want to deliver it in two weeks and then it became one week and then it became three days and then it became a day. So it was almost like a sliding schedule of how fast to deliver. So really at the end of the day, what a manager cares about is how fast you can get out of business process. So in your case, as inner source, is there something that you're targeting, something I'm targeting in my company as well? For you it was problems to solve rather than continuous programs to run. An example for this, like a metric to solve is let's say teams in support are unfamiliar with Git and we want to see how we can help them with the hackathons and everything to teach them Git. The metric to track for them is our issues coming, like you said, right? So for a continuous program that you might be running, what metrics did you track for that? Because for problems to solve, it is how we can go faster, better, stronger. We did track Git. We tracked contributions and issues, and we tracked how long the issues lasted basically because that's one of the hardest things, like you mentioned earlier. And issues will never get solved. But that's also, I think, being comfortable as a maintainer to say, we'll never do this because as someone who's been a contributor that has done PRs even to the project I work on and had nobody look at them, I would rather someone tell me that they're never going to do it than to just let it rot. And that was another big learning that we had, like sometimes you have to be okay being a little uncomfortable. Yeah, no, thank you. Great questions. So you just shared that with one hackathon you were able to save the company $5 million something? Are you able to share more details on how you were able to do that? 25 people restarting services all day. So 25 people, all they did was log on to VMs and restart a service. And how does that save money? I'm sorry. Because those 25 people can now work on higher level things. So instead of having to hire more people to do, like he said earlier, resource constraints. So instead of the company having to go hire 25 more people, I freed up 25 people to go do something higher level. Like testing, basically. So you're getting the community to do the testing, basically, rather than hiring testers. Yes. Yeah, we didn't have to hire anybody new. We found better things for those people to do. Oh, okay. So basically the head count is like, uh-huh. The head count was how we changed it, but they were about to hire more people. Oh, right, right. And they didn't have to hire any more people. Okay, yeah, yeah. Thank you. No, no, no problem. Thank you for the clarifying questions. I appreciate it. They use Ansible platform. So you recommended to use Ansible Terraform for measurement, right? They use Ansible, they use Terraform, they use all the things. It's a big company. Oh, okay. There are probably things that I don't even know about anymore because it's been a couple of years. I'd like to get back to the community part of it. Yes. When you try to build a community and there is a lance, or a few lances around, the problem I see is that many people may have a sort of imposter syndrome, like there's trouble with installing Python, how do I get Python on my Mac? And they will not ask it because they feel they should know, they feel stupid for not knowing. Would you have tips to help that emerge and be more open and inclusive and easy for these discussions to happen? So we built ambassadors within the communities, and those ambassadors were specifically to kind of help with those type of things. And after the second or third hackathon, we had more people showing up because of exactly this. They were like, well, we want to contribute, but I don't think my code is good enough. And that's a real thing. The first time I contributed to open source, I was terrified. I was waiting for a flame war on whatever I contributed. And I think it's something that everyone does. And I think that goes part back to the communities themselves and the people who maintain the communities need to be more compassionate about contributions. There is a person behind that computer that is contributing and whether or not their code is good or bad, helping them make better code is part of our job as maintainers of open source, my opinion. But I hope that everyone... I have seen really bad flame wars in communities, and that is what kills a community. And when it happens at work, these are people you still have to work with after. So it's even more toxic. Yes. So what kind of incentive programs you had for the ambassador? And like you mentioned right now, it's a job of a maintainer to help improve the code of the committers, right? So that involves a lot of time from them. It does. Where I worked, we did not have them. I've worked with a lot of people since I've come to Red Hat at different companies that have fantastic incentive programs where they give points. So internally at Red Hat, we have the ability to give anyone in Red Hat points, which actually just becomes an Amazon card. So like 50 points is $50 Amazon card. And I've worked with... I'm trying to think of it as a multiple large companies, let's say credit card companies, banks, et cetera, that they run very similar kind of point systems internally to give people things. We also, from a Red Hat perspective, we always try and give back to the community whenever we can as well. So sometimes it's something very simple, like swag, t-shirt, a doll. Like it doesn't have to be monetary a lot of times, but making people feel like they're part of it and they have a t-shirt that has the logo of the thing that they worked on. That's big and a t-shirt's not very expensive. Yeah, it's more of worth recognizing that. So you're not giving that t-shirt to anybody. Correct. The other is having that t-shirt and they've earned it. There's only 12 of these hats in existence. And I've given the other 11 to contributors or people that have been big in the community. Okay. Thank you. Thank you guys for being so interactive. This is a great, great session. Any more questions? Thank you.