 Hello, and welcome to DevConf 2022. My name is Moes Shabee, and I'm the moderator of this session. Next up, we have Timothy Upnell speaking about what's new and ahead in the Ansible community. Timothy Upnell is a senior product manager in the Ansible team at Red Hat. Now I'm going to give the floor to Timothy. Thanks, Moes. Hello, everyone. Glad to have you here to talk about what we've been doing with Ansible, particularly in the community area. I'm going to do a quick overview of the different projects that we have going on that have happened in the last, particularly in the last year or two, and some of the things that we're working on in the community level going forward from there. One thing I want to impress upon everyone is that Ansible isn't just the command line tool that it's known for and that the project began as. It's grown to be a lot more. There's a lot more in the ecosystem that's available to the community and as opportunities to contribute and get involved in the project. So I hope I can show you those things and get you interested in being a contributor or even just beginning to use all this work that the community's been doing. So with that, moving right along. Oops, sorry. Okay, what I always like to begin with when I talk about Ansible or the principles that Ansible was based on from the very beginning, just as a reminder and to set some context to what we're talking about, why things happen the way that they do. And those principles that have been there from the very beginning, I was a contributor to version 0.5, I think it was, and have been using it ever since then and these have remained the same is that Ansible's about being simple but allowing you to do powerful things in a simple way and remaining agentless so that you don't have to have all that preset up and things of that nature that you have to manage so that it adds to the simplicity of how it works. So one of the things that's happened though over the many years that Ansible's been around is that it's seen tremendous amount of growth, growth in terms of its user base, some of its code base but also in the use cases of how people are using Ansible and what they are automating with it. And with that growth has come, modules, I think when I first started getting involved with Ansible, like I said, it was like 0.5. I think there's only like 50 or 60 modules that was less than a hundred. At one point, we reached almost 4,000. It was growing at a huge pace because of all the different things that people were using Ansible for in all the different ways in all the different use cases. So that was causing some side effects. That type of growth and that type of success caused us to have to look at how Ansible was being managed, architected, distributed, the whole thing. So things worked great in the beginning but with that growth, it started to break down. It started to cause problems. There were side effects to that growth. And the ways that we saw it, and I'll just touch on this a little bit because this is a community event, but we were getting from the customers, the people that were using Ansible through Red Hat were getting very confused to what to do about when they had issues. They wanted this stability of longer life cycles because of the nature and the scope of the work they were doing, but at the same time, they needed the latest features and enhancements that they needed that quickly. On the developer side and more on the community side, we had like in the Ansible project itself, we reached over 4,300 open issues. We had pull requests or 2,000. You know, an interesting thing that went on then is we were measuring how quickly our team were closing those things and they were closing them as fast as they ever had been before, but the amount of issues and the amount of PRs that were rolling in were rolling in even faster than the ones that they could go through and process. So that was becoming an issue. The other thing that we were seeing is that with all these different use cases, people using it for networking, for security, for Windows, for Linux, for cloud, one size didn't fit all. You know, the single batteries included distribution just was having trouble meeting the needs of all these different use cases. And then another thing we were seeing and this goes to the open issues and PR thing, but even the work that our team was doing to enhance the application to the project was that some things went in quickly and other things didn't. So there was a fair bit of frustration to what was getting done, when it was getting done in certain parts of the community and the user base. So these were all things that we looked at and said, okay, we need to make some changes here. We need to adapt to this growth and the success and the evolution of Ansible. So what that led us to is to say, all right, in order to do this, Ansible has always been known for shipping a batteries included distro. You installed Ansible, you got everything, you got the modules, the plugins, there was no package manager, you just installed it, you had it, you were ready to go. But that had now become a liability. So the first thing, the first step we had to do was separate the Ansible content from the core engine, the runtime itself. So in version 2.9, that was the last one where we had that all inclusive batteries included type of distro. And then we separated out the content from the core engine itself. So this is a very important thing that had a lot of effects that rolled out of that, that came out of that decision that changed to do that thing. And you're gonna see this in the next slides and also in how we version different things. And so I'll get through all of this. So in order, when we, in separating the content, we needed a way to bundle up and provide the content that was now living separately from the core engine itself. And in doing that, we came up with what is called Ansible content collections. And these are a new standard way of distributing, maintaining and consuming Ansible automation. So they're standardized way that you can organize and package things like roles, modules, utility libraries, plugins and documentation into one portable bundle that you can distribute separately from the Ansible engine itself. Each of these collections are independently versioned. They can work on their own timeline within their own sub-communities, within the Ansible community. It just gave a lot more flexibility and freedom in order to ship this new innovation when it was ready instead of tying it down to the engine itself. So what happens now today, when we create a distribution, I'll explain these version numbers here on the screen in a bit. I know this is a source of some confusion for people that haven't been watching the Ansible community closely for the last couple months. But we have, when we now do a community package build, we take a, we pull in all the different content collections that have been, that are being included and take the latest stable version of the Ansible core engine and they're bundled all up into one package and then put out into Pi Pi for your use to do a PIP install Ansible. So, just to review, what was happening before we had collections in older versions of Ansible, like for example here, Ansible 2.5, is that we would release 2.5 and then four to six months would pass, in which case there would be bug improvements to modules, new people, people in the community would introduce new modules, new functionality enhancements to those modules they would get merged in, but you couldn't use them until we would get around to version 2.6 after that four to six months would go by. So during that time, as an end user, you did not have access to that functionality. So what happened then after collections, we can ship a distribution, but then bug improvements for modules be introduced, but then using the collections mechanism, end users could install new versions of the collections, updated version of the collections, collections with bug fixes with new enhancements as soon as they were available, that by the time in 5.0 rolled around, let's say the next release of the Ansible package, they were already making use of these enhancements that were done on the content level before we shipped the next major release. So this is what we saw as an advantage and the best compromise, if you will, to dealing with this growth issue of both use cases and just the user base and the needs of end users. So talk a little bit about the community package in explaining this big change that happened in the Ansible space about two years ago, roughly. We were working on it for longer than that, but that's when things started to be released for use. What came out of this was the Ansible community package. And what this is, is an open source batteries included distribution of Ansible, similar to the one that existed before collections were introduced. So it tries to simulate that same, you just do a pip install Ansible and you get a ton of content that you're ready to go with and use just like you did before, despite all the changes that we made and how we developed Ansible. So inside of the each of Ansible community package release is the latest Ansible core engine and then the modules and plugins which are considered core that are so essential to using Ansible, you just can't use Ansible without those. So that's part of the core collection. But then on top of that is we've bundled thousands of, well, it's actually hundreds of community supported Ansible collections which contain thousands of modules and plugins. And these are all selected based on the Ansible community inclusion policy. There's a whole system and policy that I'll talk about a little bit later to how we pick the ones that go into the community package itself. And like I said, this is just to give end users the ability to use these, to have a distribution that they can install very easily like they did before, but it doesn't stop them from going out and getting other collections that we may not have included in the community package or creating their own for their own use and bringing them into using with the Ansible engine. So one thing I wanna point about the Ansible community package is that it has a different versioning scheme than the Ansible core engine. I know this came up yesterday, we had an Ansible community meetup and this was one of the questions that was asked of us was that there was confusion on, oh, I heard there's Ansible 210, 211, 212, but then I'm seeing, 4.0, 5.0, 5.2. That's because there are now, in this case, two distinct different projects going on with their own independent versioning. So the core engine, the runtime of Ansible that things started off as, they began, those continue to have releases like 2.11, 2.12 right now is the latest stable there. The community package is using a semantic versioning system so that it leapt to a 3.00 about six months later, there was a 4.00 and another about six months later, there was a 5.00 and they've since done a 5.1 and 5.2 and I understand a 5.3 is due out shortly. So something to keep in mind and hopefully that explains some of the version numbers you saw in my earlier slides when I was talking about the separation of content inside of Ansible. So moving right along, one other thing to note is that these releases as community packages goes into PiPi, they are picked up by other distribution repositories and system, but PiPi is where this goes first and is sort of the official place that it gets published to. If you have more questions about how this whole system works, it was a great blog post written, it's about 18 months ago that I've linked to here that covers frequently asked questions and goes into a lot more detail to how the Ansible community package comes together and works and some of the differences with the core engine. One of the other things that we've added and if you take, one of the few things you take away from this is to sign up for the Bullhorn. The Bullhorn is a, currently it's a bi-weekly newsletter that goes out with all the activity and news about the community distribution and the Ansible community in general of all the different collections that are being published, things like contributor conferences or any type of activity. I really love reading the Bullhorn when it hits my inbox. I always like looking at it because there's so much going on, even myself as working inside of Red Hat as an Ansible product manager have trouble keeping up with all the activity and the Bullhorn does a really great job of summarizing all that and letting you know what you need to know. If you wanna look back there at all the archives, I think they're on their 44th release, there's a link to the archive. You can look back on what's been coming up. I really, really encourage you to subscribe to the Bullhorn to keep up with the Ansible community. Okay, so that covers the main things that happened inside of the engine and with the Ansible distribution that everyone is used to being able to work with out there. We've begun introducing a lot of other projects around the Ansible ecosystem. While this one isn't necessarily a project, it's something that it's getting a lot of attention that we've gone out and done, mostly in response to organizations that were using Ansible at huge scale with a great deal of sophistication and both scale in the number of nodes that they were managing, the number of people that were using it, the geographic disbursement of those people using Ansible Automation and it's something called execution environments that you might hear of out there. It doesn't directly impact the community package or the core engine, but it is getting a lot of interest in the community and what it does is like I said, it addresses the Ansible dependencies. This is really tied to our AWX project where you had to have virtual environments to run different Ansible automation jobs and that wasn't very portable. It was difficult to maintain if you were running in a cluster, you had to keep those in sync. So this is what the execution environments were designed to address and what they do is that they package up all the components needed for automation in a portable cloud native way. So it's bundling together all the collections that your automation will need, any type of additional binary RPMs, PIP dependencies, but it also includes a minimal Ansible core release and puts it in one container that then can be moved around and utilized wherever it's needed. So this diagram here shows you what goes into an execution environment where it's the collection, the libraries and the Ansible core getting combined up and it's based on a rel UBI image. So you have that baseline really solid Linux services in the container itself that forms the execution environment. Now out of that with creating these execution environments, like I said, the execution environment itself is not a project, a standalone project itself was more of a concept and that's being built into other Ansible tools. In creating these execution environments, we've had to create some new tooling. So we have here, I'm gonna go over these in a little more detail, Ansible Builder which lets you create these execution environments very easily, Ansible Navigator which lets you do development and it's a dev tool that for working with these execution environments and testing and developing automation against them and then a whole host of other Ansible content tools that aid in the creation and testing of your Ansible content. So Ansible Builder, briefly stated is like I was saying is a tool for creating your execution environment. It allows you to, we as Red Hat are creating some that are ready to use, ready to go of different flavors but if you have your own need, a very specific need that you wanna do, what this is a Python application that will let you, it'll produce a build context and then you create a YAML file saying what needs to go into the execution environment itself and then you run the tool against it and it builds the container for you, it builds the execution environment in a way that other Ansible systems can then utilize it in order to do portable automation, executions wherever it goes. Another tool, Ansible Navigator that I was mentioning is the idea here was to create a more cohesive, consistent top level developer experience for the content that was going to be run in an execution environment. So this is when you're creating a playbook that you can test it against the actual execution environment that it's gonna run on top of. This is really a CLI and it is based on a lot of CLI conventions of people that are used to VIM will be right at home inside of Ansible Navigator. It's a text-based user interface. So it's not a command-line tool, but it's also not a GUI tool. So it gives you, I believe it's based on end curses, it gives you different windows to show what's happening as you're running your playbooks against the execution environments and the lookup documentation right in the window itself and a whole lot of other VIM-like commands that it recognizes and just provides a better developer experience in particular for working with these execution environments. So what this all looks like is a general workflow is that you use Ansible Builder to create an execution environment and then to develop your automation user, Ansible Navigator, which then executes against a playbook along with that execution environment. As I mentioned, there's also more Ansible Content Tools. I have Ansible Navigator here on the list because that is part of this family, but that one deserved a special call-out. But other projects that we have that we're developing out there in the community is an Ansible Language Server just enhancing the development, development tools. Ansible Lint was something that was created in the community and adopted by Red Hat for giving you best practice tips and pointing out things that might be an issue within your playbook. Ansible Test, which just came out of the core project for testing, doing the type of unit testing against modules and plugins of that nature. Molecule is another project that came started off in the community we brought in. This is for doing like integration testing of how will my module, how will my roles work together and doing that level of testing there. And then another thing that's out there is if you're a VS Code user, there's now a VS Code plugin for doing your development inside of VS Code and getting all the tool tips and formatting, coloring right inside of VS Code of your Ansible Content. We also have a lot of work going on in Ansible Galaxy. Right now there's working through some things. There's been the community version at Galaxy which has been around for many, many years. They added collection support to that a couple of years ago and it also handled standalone roles, excuse me. We needed also to create something else for Galaxy. Well, for what we were doing on the product level, that code base we've now referred to as Galaxy NG. We couldn't, with the way the community version code was written was not going to allow us to do what we needed to there so that we had to create our own a new code base which was Galaxy NG. And that's the upstream to a component in the Ansible platform called Automation Hub. The one thing that didn't happen during NG development was the support of standalone roles. So it only supports collections right now but there's plans to add support for standalone roles in this code base. And the reason that we're doing that is that we're looking to move to a unified code base meaning that what's being used on Galaxy, sorry, Ansible, sorry, I have the URL wrong here. It should be galaxy.ansible.com. That is using the same code base as what we're using for the upstream of Automation Hub. And the reason of that is not only just consistency but also the ability right now we're managing two code bases and this way everything will get more timely updates and more attention and things like that. So look for that to come or come join in on that work. A couple other quick things. I know I'm running out of time here happening out in the community. This is one that I work on directly and often it's the Ansible operator SDK. If you're working in Kubernetes there's this concept of operators which run on your cluster itself to help you manage the lifecycle of Kubernetes services and applications. The operator SDK has an Ansible plug-in to it or that it's actually part of the operator SDK that lets you write Ansible content to manage things right on the cluster itself. So I won't be able to go into a lot of depth. I usually give this one slide as a whole separate 30, 50 minute presentation but definitely be aware of that and check that out if you're doing Kubernetes work. And then the last thing I wanted to mention being that this is Dev Comp is a Python library that's being used. We've developed and use extensively in a few different projects and that's called Ansible Runner. And what it is is it's a stable and consistent interface to Ansible itself. We've seen people try to use and wrap around what is happening in Ansible Playbook. And then that's prone to breaks. Ansible Runner is a solution to that you could write code against and that it will remain stable so that your code doesn't break over time. That was never the intent of Ansible Playbook and some of the things in Ansible Core. So you can use that to run Ansible Core directly or as part of another system. And it just enables the potential for embedding Ansible services, Ansible functionality into other systems rather than try to manage the complexities of the Ansible Core moving around. We use this ourselves. Ansible Runner is used with execution environments inside of AWX. And it's also a part of the Ansible operator SDK is using Ansible Runner itself. And that's just a few of the ways that we've already seen this particular library use. So if you're a developer, you might wanna check this out. It's out there on PyPy. Okay, a couple of quick- Sorry, Timothy. For the interrupt, your time is over. Could you please conclude or... Yeah, so I was just gonna do. This is, these are the links to a whole bunch of community resources. I'll be posting the slides right after this so you can get the links from there. Lot of great resources out there. Bullhorn Matrix is like an IRC, a better version of IRC. And then there's a lot of ways you can contribute and learn Ansible, self-paced training guides. Sorry, I didn't get to talk about inclusion, policy and steering committee. And with that, I will end. Thank you very much, everyone for listening. I'm gonna pop over to that work adventure. Look for me in one of Tim A inside of one of the session rooms. And we can talk if you want. All right, thank you.