 I'm Valentin. I work at Redhead in the container engines team and I help build and maintain a couple of core libraries and also the the container tools like Popman, Builda, Cryo and Scopeo and pretty much most of my talks with exactly this one and I don't want this talk to be just to explain it on our philosophy. So our philosophy I don't want to provide all solutions. On the other side, you can see this analogy of Army Knives and the one you see is one you can actually buy. It's awesome. It's all features that you can only ask of us with Army Knives but it comes at a price and here I'm not pointing necessarily to but non-final properties, right? If you want to add a feature to it, well, be careful because undecided side effects, if you want to remove a feature, well tricky. So we're really based on the right and right use case dedicated to us Army Knives which are easier to deploy so we have a quicker, well, time pocket or open source that makes sense. You know, you can get feed faster for users in the community and to customers of of Redhead and I guess most of you will know the song Sonic, you know, all all you need is a spoon and you got 10,000 knives or all you need is a knife and you got 10 spoons, something like this. You know, we want to provide pretty much this so our Swiss Army Knives tackle the entire life of a container which starts with building a container right there. We have Builda which is our specific Knives with Army Knives specialized in container building it's compatible with Docker files, but it provides way more powerful way of building container images. There's a bunch of really awesome talks by Malin and also by Dan. So if you Google, you will find some talks about Builda and how to use it and we can provide functionality because it's not baked into another tool, you know, we can innovate in the specific use cases. Same applies to distributing an image. So once, you know, we want to ship these this image somewhere and then we have Scopio. This is the first tool in our family of Swiss Army Knives if you want to see it like this. It's super powerful. You can change all kinds of compressions. Dan wrote a great blog that was published a couple of weeks ago about all these different so-called supports in the containers image library that Scopio really supports super well. It's not much a smart wrapper on the containers image library. There were a couple of talks and it's been mentioned in many other presentations here at DEF CON. There's Potman. It's a drop-in replacement for Docker. It's without a doubt the heaviest of our tools also because we want to drop in with Potman, but we have all the other tool to really specialize and innovate in cases. One orchestrated in the contrast mode where Potman is meant to be used with Cryo and Cryo's only purpose on this planet is to serve the Kubernetes. Nothing more than this. This means we have this super rock solid hardened runtime engine for Kubernetes and this and all the common building blocks. So blocks of these tools are two libraries. First is the container storage. The storage is almost the main desk. It's about the storage. So there we call for managing containers and containers images on the very low levels. We have file system drivers such as for overlay, butter, and this talk I'm dedicating to one specific conflict file in the containers image library which is responsible for managing container images on a high level to name these transports and block that Dan weeks ago already. And it's super powerful. It's a standalone and I'm simply impressed and surprised where this line is used. So it's used in way more projects than Potman and it's simple links really out at the wild, but also for segments. And there's a couple of conflict files which super powerful. This talk is on the breakthrough. So while the talk is much just easier to digest version of the main pages, following also the chat a little bit. No, if the isn't well, it can change the audio source, the microphone source. All right, back to the talk. The main is and I hope that this talk due to digest it a little bit easier. So one thing that we've been presenting in a couple of talks here at Defconn as well is pulling by short names. A short name is what we see on the left hand side. It's a name, an image reference without registry and without potential or an optional repository. So in this case, you know, you have Fedora, if you do a potman pull Fedora, solve to registry Fedora project.org slash Fedora colon and then latest, which is the tag. And the registry's con offers a lot of flexibility. So when you do a Docker pull Fedora will always resolve to the Docker hub. Understandably, when Docker was created, came up with a nice registry and like that is all used and used. So it made sense since there was one registry to always use this one. But after some time, you know, more and more public registry, it was quite pretty much every company has their own free Red Hat, Microsoft, Google, Amazon, you name it, and also all pretty much all Linux distribution. So in Susu, Debbie, and they all have their registries. So potman and its sibling projects really didn't want to lock in users into only resolving to one registry. They're use cases, maybe we're a company who wants to do a potman pull of an image and want to use to their own premise registry. So using the registry's you can resolve pretty much any kind of registry. Just as we mentioned before, we really the Fedora project, you know, they host their main Fedora images on registry fedoraproject.org. They're also pushed to Docker hub to remain compatible. So you can configure this in the registries.conf file in the so-called unqualified search registries. This is pretty much a list of strengths. And when you do a potman pull historically, and was going through the list in the specified all one by one, it contacted the first registry, tried to pull from it. If it doesn't, if the image couldn't be found, we'll go to the next registry to the third one, the fourth one, and so on and so forth. Oh, but there was last year we've been notified there is security risk by doing this. So the red hat security team reached out to us and pretty much showed us that an attacker would take over or take ownership over a repository on one of these registries. They might be able to force users into unintentionally pulling an image, then can be used as an as an attack. So in any case, pulling by short name is not the most secure one. When it comes to security, I always have users fully qualified in a true sense and use a digest, because this is then potman as dance and all the other tools and to really match if digests and the checksums are matching. We improved that with 3DO. We had an improved way of resolving the short name. So instead of more or less silently trying to pull these images in the background from the unqualified search registry, although we'll now prompt if potman and the lab access to a TTY. So the running in the terminal in this case, the user has to make an intentional choice, right? We make it explicit by where the image is going to pull from. And once you make this choice and the pull has been successful, potman will record a pair of short name and the fully image reference as an alias. Aliases are in the registry's call. So then you can configure your own aliases. You can store them or they will be stored in your record or prompt. And also we're shipping them in Fedora and in REL and other distributions to it as well. So the link on get out the contain short names. So this is a community wide project where the other Linux distributions companies create a full set of these aliases where the individual projects and distributions and companies, the software vendors, can make the choice and choose where they want to provide their default images. So send us, you know, Opus, Bad Hat, Suze, Oracle, Debian, and is there. If you have a project and you want a short name, please reach out, open a pull record or check an issue and we'll do one. So now in the talk, I go through a couple of config knobs that I think you should know and things that I use every day. And actually when preparing a demo for a talk that Dan and I gave earlier today, exactly this, because for the demo, we had a local container registry running for testing. And since I'm a container developer, you know, I need to check stuff all the time. And as you might have noticed, my bandwidth here is very limited. So I appreciate having a cache of images by means of a local registry. However, I am lazy, right? Humans by nature are lazy. I'm no exception to that. So I don't want to, you know, make the registry secure, you know, create the certificates. So that TLS verification works. So by default, TLS verification happens all the time for each connection to a registry. So we have to opt out from that. And this is a very, very nice way to just opt out from that without redundantly writing it on the command line. You just configure the registry, mark it as insecure and you're done. So for the syntax chunkies, they may have already noticed that registry of conf here is in the tumult format. And in the double brackets, they indicate that this is a table. So you can have a table of multiple registry object. And so you can define pretty much any registry you want by location. This is pretty much the address of the registry. And then there's a couple of more attributes that we will go through step by step. So if you want to block a registry, there's another field for it, which is called blocked. However, you can also, or to meet more use cases, maybe you just want to block just a specific namespace or also called or referred to as a repository, or maybe even a specific becomes a new field into play, which is the prefix. When selecting a registry, the image library goes or looks up the registry's con and tries to find if there is a registry configured that matches the registry that we're about to contact. And if there's no prefix specified, the location will as opposed. But you can overwrite this as shown here. Is this specific registry object will only be contacted if you pull from registry.example.org slash repository image below. But it will not be selected if you pull or contact another repository on it. So this gives a lot of flexibility to what you want to do in this case, a namespace or repository. Or as shown here, if you want to block a specific image, you can do this. Here comes, I think, mirroring registries is one of the oldest feature requests in the containers domain. So a mirror, like any other mirror, it's a server that will be contacted prior to the main source. So you can do this in the registry. Well, as shown here as an example, you can specify multiple mirrors. You can set them as insecure. And Potman and sibling projects will go through the list of mirrors in the specified order and only use the main source, the main registry location as the last resort. Full wins. So this offers a lot of flexibility in a super powerful, especially in air depth environments, which is used a lot about the remapping of references. So this is similar to a mirror, but more powerful. So prefix to dot.io and the location is mirror.gcr.io. This is a specific example that we deployed in Potman and the other projects back in November last year. What happened there is that the error came up or established rate limit. So I think you pull up to images per six hours. If you lock in, I think it's 600. If you want more, you got to pay for it. This is absolutely understandable why Docker is doing it, but it also caused a lot of issues for the entire world because Docker used extensively, especially in CI systems. And so many CI systems went south. They went red. People were somehow faced with a choice. Either you did to another registry, maybe Quay.io or use a mirror or pay. In any case, you got to make a choice. But we did was simply to use prefix remapping in reconfigure and the registry's con. So it was a super fast version for it. And this also was a core motivation to give this because the registry's con, while many others know what's going on there, these specific attributes and feature of it are not that widely known. So what happened is any reference from .io will be referred to .gcr.io, not hitting or you're not subject to rate limit. We didn't have to substitute image references in our CI. This worked by doing things that I want to highlight. Tom Sweeney and the team, they also wrote a blog post on it that is referenced below. I do not want to leave unmentioned here is sync. Scopio sync is a cool way to mess my great images for free to another. Or if you sit in an environment where con access, the network, the internet. Scopio sync, copy all the images you want from the outside world. You go into your data center, plug it in, and you sync also to your internal registry. So there, if you click on the link, this is a link on the upstream to the upstream. You can, as a simple see face, also YAML, it can eat YAML files if you want to relate it, if you have, you know, depending on the workflow that suits your needs. Try it. Fantastic. And especially for such things, we're from one street to another. This is really excellent. Also, one thing that is used in OpenShift a lot is the registry's conf directory. So if you're familiar with Linux, if you're a sysad main, you know, these .d directories are conventionally used for drop-in config files. And we support that as well for the registry's conf, both for root, for, you know, system-wide configuration files, but you can also use it for your rootless user. So if you're on a big grid or, you know, in an environment you want to come up with your own configs, no problem, just place them in your home directory and .config containers, and you're done. So what happens here is that every .conf suffixed file in the registry will be loaded in alpha numerical order, and on here you have 0, 0, 0.-shortnamesconf. This is where we store and ship the default list of aliases. So if you want to override an alias, just drop another config file. This is awesome if you're using Ansible or Salt or other config management systems, or you can go out-school in just SCP if that's what you want to do. So they will be, this is a very, very powerful way to add new registry settings to override aliases, to override the unqualified search registry's list. So it gives you a lot of flexibility. The details are written down on the man page, so this is meant as a pointer to it, and I find it incredibly powerful. So the takeaway messages from this talk is the registry's confidence. In my opinion, the holy grail of managing container registries is feature it. We've been talking about flexible short name. Now we can mark registries as insecure. We can block registries, repositories, images. We have a smart way of remaking references, and also the huge flexibility of using mirrors without locking in the users. And if you want to get into the details, somehow I was lucky because I wrote a blog post that was just published for four days earlier on Red Hat-enabled sysadmin. So you can go there and read about everything I just told you in detail. If you have questions, I saw a lot of traffic in the chat, so I guess Dan has answered everything already. Thank you, Antin. We have a few minutes for the, there are three of them. So if you want, I can read them because they're in the Q&A section. So question from Paulina Kubiak. Have you ever had any experience with using Docker with EL key stack and using metric beats to obtain data about containers? No, I don't even know what EL key is, to be honest. I'm sorry, I have no idea what EL key is. Thank you. So maybe Paulina can write it in the chat in a while. All right. No, I do not have any experience with that. Okay, thanks. Thanks, Moez, for helping. I had a question. As you said, the prompt for the long name is only shown when using TTY. What happens if Podman is used in a non-interactive context? This is an excellent question. Thanks for asking. So there's a couple of modes that we have. For now, when you upgrade to Portman in Fedora 33, nothing will change. So if you're in a non-interactive mode, if you do automated builds, if you run Portman in your automated environment, Portman will fall back to the previous mode where it goes through the list of unqualified search registers, registries, first successful poll will win. For Fedora 34, we're playing to the enforcing mode where in this case, Portman will just use to poll. If there's no recorded short name or if there's no matched areas for the short name you want to poll, and you're not in an interactive setting, then Portman will throw an error exactly with its reason and say, sorry, I'm in a non-interactive mode. I reject it. So I want to play nice and credibly make it more secure. Thank you, Valentin. So Paulina has added, I meant elastic search plus Loctesh plus Kibana for LLQ. Okay. Thanks a lot, Paulina. But no, I do not have any experience with that. So we have a last question from Honza Horak. Any tips how to edit the registry config files via command line? My set commands are a disaster and needed to be adopted to a new format a few times already. That's a funny one. Well, I edited it with VI. It's a Tomo config. So I would guess that there's some tool for JSON for Tomo as well. But I usually just use my editor of choice and go with that. Since it's Tomo, you know, you have bindings for all kinds of on principle go languages. So you can also do it programmatically, but from command line, I usually just use my editor of choice. But as Dan mentioned here, you can use drop-in config files to write existing keys. And this is also something that actually should be the default recommendation. If you want to edit something, just create a new config and throw it in the files. Because if the default one from the distribution may change, then you will benefit from updates there as well. Thank you, Valentin. So let's get to the last question from Antonio. Any thoughts of Docker Hub or Kauai.io support for OCI? So we could push Helm charts there. A great question. I think both registry support OCI and Helm charts are OCI artifacts already. Actually, just yesterday I was pushing an OCI artifact to Kauai.io and the Kauai team is already supporting Helm charts. I assume Docker Hub, well, there were some issues of Docker Hub and OCI images last year where images were not displayed, but they were supported. So although when you were on the website, although the image displayed, you could still pull it and also override it. So I think both registries support that already. I can be for sure, or I know for sure the Kauai.io does for Docker Hub. I would be surprised if they don't.