 Thank you everyone for being here today. I know there are a lot of great talks out there, so I'm really glad that you chose to attend this one. My name is Dennis Kleben, and I'm here to tell you why Pulp 3 is going to be simpler, better, and more awesome. So a little bit about myself. I'm primarily a Python developer, and I've been doing that since about 2009. I'm employed by Red Hat, and my primary responsibilities are contributing to Pulp. In my free time, I like to juggle. So if any of you want to juggle later, you can find me. I have six balls in my bag. All right. So I want to start by telling you a little bit about what Pulp can do. Pulp lets you manage all your packages. You can mirror external repositories, you can upload your custom packages to Pulp, and then you can mix all of that content and make it available to different environments in your infrastructure. So what's new in Pulp 3? The greatest feature that we're adding with Pulp 3 is versioned repositories, which means that every time you change content in a repository, a repository version is created and it is immutable, and you can publish that version or roll back to that version and it just makes things much smoother than Pulp 2. We have also provided a very concrete plugin API and it is semantically versioned so that there is now a Pulp core plugin package on PyPy that our plugins can depend on, and they can know that as long as they're using a specific version of the plugin API, their plugin is going to work. We were able to also reduce the size of our code base by relying on frameworks such as Django, Django REST framework, and Swagger. With this, we were able to create a Browseable REST API which I will demonstrate later in my demo. We also include REST API documentation with every install. We can take a look at what that looks like here. Let me zoom out a little, maybe a lot. All right. For every resource and endpoint, on your Pulp 3 installation, you can just go to API v3 docs and look up the documentation. We also dropped Mongo and I'm so glad that we did. We fine. Yeah. We're now using a relational database which can now do joins instead of having to use Python for that. We decided that we want to make Python our primary packaging method. So all of our builds are uploaded to PyPy and you can use PIP to install Pulp 3. We're also including an Ansible installer that will install using some Ansible roles that we published to Galaxy and it also installs some config files that you need. Pulp 3 also removes some features. We decided that the Pulp agent that we were providing with Pulp 2 was not something we wanted to keep carrying. There are other tools out there that can do a really good job of managing your machines such as Ansible. So we recommend to our users to do that. We also got rid of scheduled tasks. Once again, there are tools that can do this much better than Pulp can. I believe there's Rundeck and if you want to use Cron you could do that but there are tools out there. We also got rid of Celery Beat. So there is now one less daemon that you have to run to run Pulp. The responsibilities of Celery Beat have now been distributed to other workers. So all the workers are looking out for each other and doing the work of Celery Beat. We also got rid of SimLinks. So now when you publish, you don't have to wait on your slow spinny disks to write out all those SimLinks. All of the publishing happens at the database layer. So where are we with Pulp 3 now? We are in the Alpha phase. We're building Alpha builds about weekly. We're pushing them all to PyPy and we're hoping to enter beta pretty soon in the next few months. We have one plugin that already works and that's the plugin I'll be demonstrating today and that this plugin manages files. We are also already writing a couple of plugins. The Python plugin is under development and a plugin to manage Ansible roles is also being developed already. And we're going through the planning phase for RPM and Docker. And we're going to be publishing roadmaps for all of these plugins in the near future. And now to my favorite part of the presentation, the demo. So I'm gonna be rotating between the terminal and the slide deck here. And here we have a repository that's out on the internet. It contains three files. We also have a user that's going to upload a file and we also have a pulp which doesn't have any content in it right now. So let's begin by creating a repository. We're going to make a REST API call that looks like this with HTTP Py. We're gonna do that in the terminal. And so the repository got created. So now we have a repository. Next, we're going to sync that remote repository that we have. In order to do that, we have to first create an importer. The important part to notice here is the feed URL. This is the URL that the importer will use to discover content to download. After we create the importer, we will tell that importer to sync. And the sync will be an asynchronous task. So let's go do that. All right, so this asynchronous task can be followed through the REST API. And this is the browsable API right here. This is thanks to Django REST framework. We can see that the task completed and that three content units were added. And as a result, a repository version was created. The repository version has three files in it. And we can also look at the actual content. These are the three files in here. Test two ISO, test ISO, and test three ISO. So now we have a repository version for a repository foo. And we also have those three files stored in pulp so they can be reused for other repositories. Next, we want to publish this repository version. We're gonna make some REST API calls here also. We're gonna create a publisher that's specific for the file plugin. And then we're gonna call the publish method on that publisher. Publish is also an asynchronous task, which we can follow here. It has completed and it created a publication. So here I wanna point out a difference between a repository version and a publication. The repository version is an immutable set of content in pulp and the publication is the same set of content plus any metadata that needs to be present for a client to consume that content. So for the file plugin, it's the pulp manifest file that just lists all the files that are available in that repository. So we already created the publication and that's what we are showing here that the publication A is that of repository version one. Now, whenever we want to make this publication available via HTTP, we need to create a distribution. Distributions basically serve your publications. And we're gonna make this REST API call and we're gonna specify which publication we want to make available. So we made a distribution, which means we can now look at the generated pulp manifest file in our for that distribution. So as you can see, we performed a curl here on the distribution and the distribution contains the three files that we synced from our remote repository. So we now have a distribution that's serving our publication. Next, we're going to upload some content into pulp. And then after we upload the content, we're going to add it to our repository. And whenever you add content to a repository, a new repository version gets created. You can add one piece of content or you can add a bunch of content at once to create a new repository version. So we're gonna do the upload and now we're gonna do the, create a new repository version. And let's look at the output of this task. The task completed and it produced a new repository version, which has now four files in it. And if we look at the content, we will see that there is now a new file in there. foo.tar.gz. So we have a new repository version. So let's publish it and create another publication. Now, now that we have a new publication, we want to update our distribution so that new set of content is available at the same path that we published our first publication. So we're going to update the distribution and we're just gonna specify the new publication that we want to be available at that relative path. Yeah, so right now, before the update of the distribution, I curled the pulp manifest and it still has only three files. We're gonna update the distribution and then we're gonna go check the pulp manifest and we see that now there is a fourth file available. So let's talk about content promotion now. In pulp two, if you wanted to make the same set of content available at a different URL, you had to create a new repository, then you had to copy all of those packages into that repository, publish it again, and then you would have it available at a new URL. In pulp three, all you have to do is create another distribution, point it at the same publication that you're interested in serving at a different URL and you've got it available at a different URL. So we're gonna create another distribution and we're now gonna check that it's available at a different URL now. The other URL was foo slash dev and this one is foo slash test and it's showing us that it has the same content. This is the link to our YouTube channel. We do demos about once a month. So if you wanna follow the progress of pulp three or you wanna contribute, join us on YouTube and I'll take some questions now. Now UI interface. The question is did we start working on a UI interface? We have not, but all the plugins for pulp are going to be, they are Django apps. So it would not be hard for someone to create another Django app that is basically a UI for pulp three. So I think this will be a great contribution to make the pulp three. Yes? When will the GA be available? The question is when will the GA be available? I don't have an answer for that. We're hoping to go into beta soon and get the plugin writing going. So can't commit to a date. Go ahead. When you upload a whole bunch of files, is there an atomic guarantee? The question is when you upload a bunch of files, is there an atomic guarantee? For the version. For the version. The way the upload API works is that you first upload artifacts into pulp and we have an artifact API and then you convert, then you create content units out of the artifacts. So you do that before adding them to any repositories and once you've created those content units, you can do a bulk add to a repository and create a new repository version. If there is a problem with adding any of the content units for whatever reason, that version will not be created and you can try to do it again. Yes? Have you considered uploads via native tools like pip or whatever other tooling users? Could you rephrase that? If you publish a Python package for example using pip and it also publishes to pulp, is there anything planned around that? I believe, so the question is, does pulp plan to support publishing? Uploading content. Uploading content using API similars to PyPy? Yeah, same one, like directly from pip, like pip uploading into pulp. Yes, I believe actually on the Python plugins roadmap there is a use case that covers that. Yum repositories or DNF repositories, RPM packages, what's the state of the plugin? Oh yes, so the question is, what is the state of the plugin, of the RPM plugin? The RPM plugin is being planned and a roadmap for it will be published soon. It is a very important plugin, so I'm sure development will begin soon. Will there be a Debian support? The question is, will there be Debian support? Absolutely. With pulp two, we added Debian support last year and it has been a very popular plugin. We have made sure that the plugin API is very straightforward and easy to use, so porting that plugin from pulp two to pulp three should not be difficult. And how does replication between data centers let's say work, it's the same as pulp two? The question is how will replication between different instances of pulp work? We will not have nodes, nodes were deprecated in pulp two and we plan to support being able to sync from one pulp into another pulp, as if it was just another repository out there and you're just syncing it to your other pulp instance. The question is, is there going to be an OSG plugin? Yes, there will be. I forgot to put that on the slide. I wanted a year from now to make an HA pulp end point. How might I do that? An HA pulp end point? Yeah, how do you do that? Yeah, I believe that that was already built into pulp. And you can have redundant services running so that if there is a failure, it will fail over. That's already built in. Do you wanna? Sorry, I'm just clobbering on pulp. Yeah, pulp supports full HA both in pulp two and pulp three, each piece. There's no single point of failure at any level. It's been one of our goals. And pulp three right from the work, I was gonna say, can you repeat that a little bit? Yes, I'm gonna repeat for the camera. Pulp two and pulp three both are fully HA. The question is, can pulp be a container registry? Pulp itself cannot be a container registry. There's another project that we work on also. It's called Crane. And it works together with a pulp docker plugin to be a registry. So the content is stored in pulp but it's actually served by this other project called Crane. Yes? When you import the data, if you import it from a pulp manifest, is it possible to import data from a data set that isn't managed by pulp or is it flexible enough to have it possibly imported? Yeah, so pulp is designed to have plugins. Oh, sorry, I'm gonna repeat the question. Is it possible to import content into pulp that does not have a pulp manifest? The pulp manifest is only required by the file plugin. Each plugin is dedicated to particular content types. So the RPM plugin is going to support importing YUM repositories and every plugin does its own thing, yeah. And you probably can write a custom answer. Yeah, you can create whatever kind of plugin you want. I like to say that you can write a plugin for managing your cat photos. Though the file plugin kind of does that. Thank you, everyone.