 Thanks for numerous attendance, given the beautiful weather outside. I'm done, and I'm gonna tell you a bit about testing container images with Python and PyTest. First, the boring part, so that's me. I work at SUSE, I kind of became the release engineer of our base container images. Besides that, I also do other stuff, but since we only have 15 minutes left, let's skip the boring part and go to the interesting part, and that's why on Earth would you use Python and PyTest if you have shell scripts. And we all know shell scripts are portable, they run literally everywhere, even on your 20-year-old AIX machine. They are super fast, exceptions and terms and conditions apply, and everyone understands them. And I'd like to use this small opportunity for a quick poll. Who of you likes shell scripts? Please raise your hand. Oh. Okay, that eliminates the next question. Never mind. My point has been served. So for the recording, there was like five people and there's, I guess, 50 in here, so I think I don't need to convince anyone. So let me still give you the short sales pitch why you should use Python, PyTest, and especially the plugin that I wrote for testing container images because well, shell scripts, in my opinion, they're pretty brittle. They usually tend to break, especially when you don't like it because I mean, I don't understand shell quoting. And when I write shell scripts, I have to start writing tests for my shell scripts. And if I start writing tests for my tests, I'm doing it wrong. So let's not do that. So what can this plugin do for you? You can define all your container images. It will do all the pulling, all the building, launching, cleanup for you. So you don't have to do that. You can use the testing from module, which gives you just a very nice abstraction layer all around all the common file system operations, checking whether there's packages installed. Essentially all the stuff that you also can do in a shell, but just in Python. We've been relatively meticulous about support of parallel test execution. So you can run all your tests in parallel. Also, if you expose ports, also if you do volume mounts and so on, because that's also, as I said, one of the features, it will find you three ports and actually tell you what the port is. So I mean, you can just bind some random port by a portman and docker and then have fun finding the JQ expression to find the correct port in the output of portman inspect. PyTest container will do that for you. If you use portman as the backend and not docker, then you can also use ports to create them to have a shared network. You can create and clean up container volumes and bind mounts if you need that for some reason. Also, if your container somehow declares a volume and creates anonymous volumes, they will also get removed afterwards. So you don't suddenly realize my container storage suddenly consumes 10 gigabytes. Where did that come from? And you can also use some fancy PyTest magic to just execute the same test on multiple container images. That's relatively useful for, for instance, for testing the same piece of software on multiple operating systems. And yeah, and it supports docker and portman. You can switch between the two by just setting an environment variable. So now with the sales hat off, sorry, next one, it actually runs on rel seven. I don't think that's a great feature because it also comes with all the catches, but it runs. And it also works on all the architecture. And let me tell you PowerPC and S390X weren't fun, but it works. So let me give you just a very short example, how this would look like in practice. You import all the stuff that you need. You define yourself a container. In this case, it's just the open source a tumbleweed container. And then you write yourself a test which does very, really nothing very impressive. It just checks whether the file at COS release exists. So this, what you see here is just testing from module that handles that. And this parameterization is just included for PyTest container. Now, if you would write yourself a put this into a Python file, install the required dependencies and execute PyTest, then it would, then the plugin will pull down the container image, launch it, launch the container before the test, connect to the container, check whether at COS release exists. And after the test has passed, it will just, it will remove it and spin it down again. So what can you use this for? You can use system tests of your container images if you create operating system container images. You can test applications that you deliver via containers because just because it's in a container doesn't mean it actually works. Or what I also think is a relatively nice thing and where we use it in a few places is that you run a few tests on multiple different operating systems. If you want to run it on Fedora, on OpenSUSE, on CentOS, whatever you like, you can do that. So for the record, the whole plugin uses a few features of PyTest, but I guess since there's so many of you, you probably know that this is a Python test framework on what it executes. It relies heavily on Python fixtures, which are just magic functions. These handle all the spinning down and the destruction of containers. And what also is relatively useful is that you can parameterize tests. In this case, your parameters are the containers. So and now let me just breathe through various examples what you can actually do in practice. So common thing is you have a base container and now you want to actually build stuff in there. So in this case, you would define yourself a container file. You put that in this derived container class. You don't have to take pictures, the slides are on GitHub. So there'll be a link at the end. Then you just pass it as a parameter. And in this case, it's just doing a very dumb check whether Python 3 is installed because we just did that up here. Somewhere if the font rendering wasn't broken, sorry about that. Another feature is getting a binding free ports. That is especially nasty if you want to run tests in parallel because if you try to bind the same port from two containers, oops, won't work. So in this case, you define yourself this, you pass just a parameter to this derived container class and then inside your test function, you will get access to the port which this port gets bound on the host. So then you can for instance, just curl it and find out doesn't my container respond to this or if it's running some other service you can use whatever means necessary to contact this host. You can create ports. This has, so as you see, I'm a very creative person when it comes to naming, hence this thing is also called PyTestContainer and the port class is called port. Makes it kind of obvious, but as I said, not a creative name and the port just expects a set of containers. The main use case for creating ports at least so far as I have used it to just expose some service to the outside and so hence you can also forward ports from this port and you use it kind of in the same way that you just pass it as a parameter to your test and then you can do whatever you want in your actual test. One thing that I have so far omitted is that all of these tests reuse the same containers and they assume that you, so as an optimization they assume that your containers actually don't mutate things. So if you suddenly start rmming things all your subsequent tests will be influenced by that and you might not wanna do that but there's a workaround for that. Just use a different fixture which is creatively called container per test and then you can also do things like testing whether rm minus rf will actually work. I haven't run this one because I'm too scared but maybe you're brave and in theory things like this should still work. You can also, if you have a set of tests which should all run on the same container image and you don't want to write this test mark parameterize for everyone you can just define a global variable called container images and use a fixture called auto container and then everything will just be automatically parameterized by some very nasty pie test magic that I'm not proud of. If you try to build containers then sometimes it also happens that containers depend on each other and you can do that one as well so you can define yourself a container then build another one on top build a one on top please don't add circular dependencies because that will most certainly break but then you've been warned. And once you execute the actual test the plugin will first look for the container that's been requested, find out okay I need the other one I need the base one, the tumbleweed one at the top pull that one, build the subsequent one and so on launch your test and after the test destroy everything again. If you want to have some abstraction over podman inspect or docker inspect there's also one for that so you can just grab the inspect property from the fixture and you'll get a bit of the output that podman inspect would give you. So for instance, config cnd but not everything because there's a lot of info and it's definitely not stable over releases and I don't want to support all of that. You can create bind mounts again this is very creatively called just bind mount which can be occasionally useful if you want to bind a specific directory. What you can also do is to bind really specific container volumes but you can also achieve that by just putting a volume instruction in a file but in case you need something like that you can make the plugin aware of that and then you'll also get all the automatic destruction that it supports. One thing that I think is very useful is health check please use health checks, they are great because often your services actually need a bit of time to launch and if you test your service yourself you'll launch the container and it will take you a bit to type the test command Python is faster and usually you wonder why does the test break then you check the running container and it's still working and it's actually working but your test is failing because your test executed far faster so if your container contains a health check then the plugin will automatically wait for your container to become healthy and after it's healthy it will start checking. If you don't want to do that for instance because you want to check whether your health check fails then you can also configure that you just tell it don't wait for it and then it will not bother you. As I said you can use Docker or Portman it will use Portman by default if it's installed otherwise it will use Docker so you can just switch that by an environment variable you can run tests in parallel by just installing PyTestExTest and by whatever means you want and then just pass the minus auto flag and really that's also how we use it so if you run heavy tests you can murder your machine as much as you like but it will run the tests in parallel and way faster and also the plugin really tries to clean up everything so all launched containers get destroyed all volumes get removed all ports get removed afterwards all temporary directories what is retained are images and intermediate layers just because then the test would take ages to execute. There's a few things using it for instance the BCI test suite a bunch of other services as you see all kind of tied to open sooths because I made them use it but maybe your project or you tell me about your project and I'll make you use it. I would like to thank a few people and mostly Jean-Philippe who made the initial prototype with myself and I'd like to ask you to give it a try if you find this useful as I said the plugin is on GitHub the documentation is there as usual it's there's room for improvement but I don't know where currently the slides are linked at the bottom and with that I would thank you for your time and be happy to answer all your questions so what do you mean exactly by... No, so if you use in the default mode you will get the containers will be shared to be frankly honest it's I know this is embarrassing because I wrote it but I'm not sure if it will it might be that the containers that if they actually if two workers in parallel launch the same job they will actually get different containers but the sharing is an implementation detail you shouldn't rely on that it's meant to speed things up so how that actually behaves will depend on the behavior of PyTest of X-Dist how it actually handles that so yeah so the thing is I'm not sure how X-Dist will handle this kind of fixture so it's just a fixture that's global for the whole test suit It should be a session scope fixture yeah so then they will be different