 Alright, welcome everyone for those of you who are new. Thank you very much for coming. Our next talk is Austin McDonald and I'm thinking he's talking about fall three. I'm going to randomly guess. Enjoy. Hello everyone, thank you for coming and especially thank you for coming this afternoon in this weather. Maybe this is not so terrible for you guys but I am from the southern United States and this is not for me. So I appreciate braving the weather. So today I'm going to be talking about the advancements that we've made with fall three, especially since last year in case any of you came to Dennis Cleveland's talk. We've made a lot of improvements, most of them have been under the hood except for we've written a lot of new plugins for fall three and we're very pleased to be approaching our release candidate coming up. So just in case some of you are not familiar with fall three, what is it? I'm just going to go through some of the basics. If your production environments are created by installing software from repositories that you do not control, then you do not really have full control over your production environments. The entire thing could go down, somebody could update something or remove left pad or whatever and everything could just completely fall apart and so if you absolutely need to have control over your production environments you need to be managing your own repositories. And if you want to be extra careful you should be life-cycling those repositories, promoting them from dev, testing, production, just standard best practices and you can do this with fall three. So what we have here are a plugin architecture and so pulp can manage any content type, can pretty much do anything. All it needs is a plugin that does that content type. We're open source and community driven. This has especially been paying off recently. We've been putting more and more effort into our mailing list and helping out people to help us and we've been getting more and more contributions from the community including some plugins. So we're very pleased about this. Of course Python is awesome but I think the main reason that it's fantastic that pulp is written in Python is because it allows us to leverage libraries like Django and Django Rust framework. Okay, so pulp will clearly solve all of your dependency problems, all of your nervous things that keep you up at night. So now I'd like to talk about some of the new things that you can expect from pulp three. So remember this slide, this is the plugin slide. Every single plugin here has a plugin for pulp three. So you can manage Docker, Python, Chef, Debian, Ruby, RPM and Ansible roles. So if you do any of those things you can do it with pulp. So new features, the whiz bang new feature is version repositories. As you can see here you can test it before you send it to production. And if I were a sys admin I would be so nervous if I didn't. And of course version repositories also gives us the opposite. With pulp three you can roll back to a previous version and you do not have to republish. So you can do this instantly and fix something if something goes wrong. And of course this will happen because all software is terrible, right? Okay, this is a bit more of a subtle feature, but plugins can now implement dynamic web APIs. So if you have a fancy client that needs to do some kind of negotiation like the Docker client, you can now talk with pulp directly and implement a full web API. If you're up on the new hotness with Python then you know that async.io is awesome and performant and everyone should be using it. If you're not you should check that out, but all you really need to know is that pulp three is faster. Deferred downloading is a pretty fun feature. We also, we lovingly call it lazy sync. This was around for pulp two but only for the RPM plugin. But basically if you've got a sync down of, I don't know, 50,000 RPMs, I have no idea how many RPMs are in the Apple repository, but I know it takes like all day and it will use up your entire disk. So if you don't want to do that what you can do instead is sync down only the metadata, do it quickly and then fetch the bits as you need it. It's very convenient and in pulp three lazy sync is going to be available to all the plugins. It's built into the plugin API. And lastly our REST API docs are auto generated because we never like to update our REST API docs. So now we don't have to. Another pain point for pulp two is that it was fairly difficult to install. There's a lot of components. You've got a database, a queue, pulp itself. And we've taken care of this by providing ansible roles. And all you have to do is provide your own playbook and customize it and add variables, season to taste. This is how simple it is to choose which plugins you install. Just add a couple of lines of YAML. This is how simple it is to do a source install. Just give a pointer to the source directory. And that's pretty much how the ansible installer does for a whole pile of things. And a note for the users, you will know that the ansible installer will work because it's also the same way that the developers are installing our stuff. We use it every day. And if it breaks, it's more annoying to us than it is to you, I bet. There's been a few improvements under the hood in pulp three. Most of this was covered last year in Dennis Cleveland's talk. But Postgres instead of Mongo, RQ instead of Celery, it's a semantically versioned plugin API, which is, that's actually really fantastic, even for the users, because now you know exactly which plugins can work with exactly which releases of pulp core, which was somewhat of a pain point before. No sim links, and we have a lot less code. Just for out of boredom, I think, one of the engineers on my team went through our repository last night and discovered that we're currently using 10% of the code that we were using for pulp two. And we're approaching feature parity, and this is removing docs, removing comments. So 10% is pretty fantastic. We're going to be conservative and say, after we get to parity, it'll be 20%, but we're still very proud of that. Okay, so this isn't a real demo, this is just a slide demo because I don't trust internet. Okay, so all your plugins in pulp three are going to follow a very similar workflow and you're going to use the same APIs. So a Docker demo will pretty much show you how you do it with Ansible, Gem, Chef, Debian, whatever. So first you create a repository. This is our fun little symbol for a repository. It's just a simple post to the endpoint and you pass like one argument. There's more options, but I'm showing the minimal versions. Second step, you create a remote. In this case, the remote is the Docker registry and we're specifically pointing to Docker Hub Busybox. So what we're going to be doing eventually is taking Busybox from Docker Hub and then owning it ourselves so that Docker Hub could blow up and we'd be fine. Okay, and then we kick off a sync task using our remote. We sync from the remote to the repository. Pulp goes and gets your stuff. Next, we create a publication. A publication is simply just the metadata that a client needs to use later. And with Docker, this is actually very simple. And lastly, we create a distribution. And now this is the very crucial step for CIS admins who want to make sure that they are in complete control of their environment. Notice in the Docker distribution that we've set the base path here to testing. What that means is that when you Docker pull, you Docker pull from your host port and you're in testing. If I had set this to production, this would be flash production. This means that you are absolutely in control all the time. You can keep updating your repositories without breaking the other people behind you. And it creates a linear path of versions which you can have production roll up to the next version if it works, which makes all of this much simpler. And just to prove that it works, I did all of those things. And then Docker pulled from my sandbox environment and it did impact work. Okay. That about wraps up the high level overview. There's some relevant links here. And if you would like to join us in Pulp IRC, Pound Pulp, Pound Pulp Dev. And our mailing lists are especially active. So before I go to questions, one thing that I would like to do is to direct you to these mailing lists, specifically Pulp lists, if you have any content types that you're looking for. During the next period of Pulp's engineering, we're going to be in a release candidate for kind of a long time. And during that time, we're going to be focused on plugin development. And so we need to know which plugins you would need to make your lives better. So do we have any questions? Yeah. That's a great question. Not yet, but there will be before we release the GA. Yes. Oh, syncing from one pulp to another pulp? Yeah. That is actually, that's pretty normal. It's going to have to be done at the plugin level. So if we go back here, when you create a publication, you're creating the metadata that the client would use for most of the time. But in some cases, the Python plugin does this, for instance. We have to do a different kind of publish in order to publish metadata that could be read by another pulp. But most plugins, it just kind of works under the hood. No worries. But Python plugin, it was not a big hassle to make it work. Oh. Of course. So pulp is split into many components. Well, not that many, but we've got our web app. We've got our content app. They're separate and scalable horizontally. We've also got a task work, a worker system that performed tasks from a web app. Those can be scaled horizontally. So if you're syncing a large repo and you want a lot of machines doing I.O. or you've got heavy calculations, then that's how you would go about that. You'd make a lot of workers. So for HA, what we're trying to do is we've got, since we've got the content app split from the web API, that the content app can be controlled separately and scaled horizontally. Does that answer your question? Yeah. Yeah. Yeah. So the default, the default system is just your local file system. But part of the reason why we were able to drop so much of the code is because we're leveraging great tools that were built on top of. And in this case, we get a lot for free from Django and Django's file system stuff. We don't work yet on S3, but we will. I'm told we will be in the next two, three weeks. Say again? Oh, instead of symlinks, we're now just doing the relative path that's stored in the database. And we're just acting like a normal Django web app. But we were having problems with the symlinks with very large installations we were running out of inodes and things like that. That seems like a ridiculously large one to me, but, you know, just developers thinking the same thing every day. Yeah. Oh, okay. Well, we are existing in the same space as Artifactory, but we're free. We're open source. And in my opinion, our plugins are more strong. They are kept up to date by Red Hat engineers. They have Red Hat QE behind it, as well as a growing community. So it depends on what you need, to be fully honest. But Artifactory at the moment has more plugins than PULP, but in the long term, we'll see. Our plugin API has made plugin writing super easy. By the way, if any of you are going to config management camp, we'll be doing a plugin writer workshop, and you'll see that it's really not that hard just to write a Django app. Any other questions? Yeah. Yeah. So the question was, is there a filtered sync? And the answer is that depends on the plugin. Syncs are completely written by the plugin writer, and that's just by necessity. We tried to do some things under the hood to help them, and we just really ended up getting in the way. So the plugin writer writes the sync task, and so, for instance, the Python plugin does have some kind of a filtered sync. You might specify that you want to sync only Django 1.11 Plus, and it will pull down all the versions of Django 1.11 Plus. So that's something that we do in Python. I am not familiar with that in any of the other plugins, except for it will happen in Docker pretty soon. Any other questions? Yeah. Are you asking about authentication? Yes. So for the RC, we are keeping our auth very simple. We have one user, it's the admin user, but if you're running a pulp and you are managing your repositories, chances are you're the admin. If that's not your use case and you do need more stuff, one of the reasons that we have not done it yet is that we have not had a very good understanding of how that would be used. So if what we have right now does not work for you, now is absolutely the time to let us know, because we can still slip in those backwards and compatible changes before the RC. After that, we will be stuck with it for kind of a long time. Any other questions? Yeah. Sooner than I want to. Yeah. So we're planning on putting out our RC sometime very soon. And then we will be having a very long RC, possibly in the months to half a year range, maybe more, I don't know. We're not going to release it until it's right. That's really what it comes down to. And satellite will not include it until it's right. But we don't have to wait for all of the plugins to be perfect. I think the plan is that maybe we'll be running Pulp 2 and Pulp 3 side by side. And some plugins will get ported one at a time. So sorry, I can't answer the time frame. Any other? Yeah. So the question is there going to be a GUI? And I've actually heard this question more often today than I've heard in the past. And so I have the suspicion that we probably will be doing a GUI. And I talked with one of our UX designers about that today. But it's very, very tentative. It's not the immediate priority. Right now, the primary interface is the REST API. But we will be having a CLI. And we also probably will be having a GUI eventually. If anyone likes writing GUIs, our REST API is very easy to use. Okay. Any other questions? All right. Well, thank you very much.