 Thanks, everyone, for coming to this presentation entitled From Pipe to Poetry, Python, Many Ways of Packaging and Publishing. Hope you guys enjoy it. For those who never heard about me, my name is Vinicius. Vinicius Gubiani Ferreira. I work at Asim Technologies as a senior software engineer. For most of my career at Asim, I've been a Python back-end developer. And since last year, I started working in the newly created quality assurance area of the company. I'm also a no-pour-source contributor. I work on translating the Python documentation for Brazilian Portuguese. And I also love craft beer and riding a shared biker on the park. So our agenda for today in this presentation, we're going to quickly go into the motivation for doing it. We're going to talk, of course, about package managers, BPPM, Poetry, everything related and between them. I'm going to show you a dummy application that we're going to package and deploy with each of the package managers. And we're going to wrap it up by showing the differences the weak and strong points side by side on the three package managers. And I hope you guys like it, so let's do it. First things first, the motivation for this presentation is to help everybody who's watching to reach for the next level. Even when we are experiencing programmers, there's always something new to learn or something that can be improved. I know I learned a lot during the research for this presentation because last time I tried to package a Python application many, many months ago, it was a total shame, disgrace to be honest. And this time I felt, wait, that's it. There's something wrong, can't be that easy. And it is. And I hope you guys learned something new, even if it's a tiny thing by the end of this presentation. If you did, then I already be glad. So package managers, they are a small but very important part of software engineering, which is called configuration management. And the configuration management for those who never heard about it is a system engineering process for establishing consistency of a product's attribute through its lifetime. So let's focus on consistency over here. And that means in simple terms or representation that if we add the same ingredients and follow the same process, we will have the same results. Because after all, everybody has or will have that works on my machine problem. So we have package managers to help us keep track of the ingredients, a.k.a. the packets that are used on our application and in which version they are actually running. And the quick disclaimer before we are getting to our first package manager, there's more in Python than just pip, ppm, and poetry. There's also easy install, which is now deprecated, but served the Python community very well until 2008, when it was replaced by pip, officially. There's also called, which is an amazing package manager for the data community around Python and R. There are also the package managers of the operating system itself that can allow you to install Python packets, but they are not usually the best choice because sometimes the version that you have available is a bit outdated. But we're not going to go into details of all of this. So speaking about pip, to get it started, what exactly does pip mean? It can actually mean several things like pip install packets, pip install Python, and even preferred install programmer. And the pip is, despite its age, is very widely used in the Python community. And sometimes it already comes pre-installed on some Python systems. And in just in case it's not already installed, you can just CRL these address and run it against the Python interpreter. And that's about it. It will be installed. If you want to install a specific Python package, we just run pip install in the package name. If we want a specific package version, then we just use the double equals and the number of the package. And finally, if we decide to create a manifest of all the package that we are actually using on our application, then we're just going to create a simple txt file, store that information in there, and use the minus r option to install it. So that's about it with pip. And some people might be thinking, so simple as that. And it's perfect. And turns out it's not, because I'm going to tell you a story that happened with me. It is a true story. And the following Europe Python's guidelines, since this is a family conference, this is rated E for everyone. But it's rated only by me, not any other group of people in so ever. So that story starts with a much less experienced admin issues about nine, 10 years ago. And he was saying, thank god this research project is finally being deliberate. I had enough of it. And it just started to break from the previous nights to the next morning. And we didn't understand what was going on. It was working fine on the previous nights. And at least I really felt like banging my head against the wall and trying to find a dark corner so I could cry alone. And after a couple of minutes of being sad, we started to look it up and to understanding what was going on. And we noticed that a Python package was breaking. Because one of the dependencies, a child dependence of a package, just flipped from beta version to release version, release candidate, sorry. And that started to break the project build, which took about hours. And after a few attempts, we had to do two minor fixes, which is deciding to manually install the previous Python version of that child dependency that was actually working. But unfortunately, it wasn't enough. We had to install it before the package that we are actually installing. So we were kind of tricking PIP into using the version that worked. And from that day on, we learned some interesting lessons with Python with PIP. And the first of all is package order does matter, at least in requirements.txt it does. Dependency sometimes is not so much like we think it does. But maybe that is because the computer do what we tell it to do and what we're actually expecting it would do. Works on my machine is a real thing. And that is very possible to happen with PIP. And there are only two approaches for using PIP. You can either just keep track of the top layer level package that you are actually interested, that you know they have a good use on your project. Or you can just keep track of everything like the image on the right. But then your requirements.txt becomes a big mess. Both of their approaches are not quite perfect. So after going through PIP, some people tend to look to other tools when they are seeing through package managers. And then they eventually find PIP. And how exactly does it differ from PIP? It takes ideas from many other tools, other package managers like Composer, NVM, Cargo, YAR, and others. And it already creates the virtual environment automatically for you. And some people might be thinking, what exactly is a virtual environment? Because I didn't mention it before. It's like a tiny sandbox where you install your project dependencies so they don't get mixed among different projects. And you're operating system packages, which is good because then you don't rely on Zulu or HOOT or anything else. Don't expose yourself to security vulnerabilities. It also introduces the concept of PIPfile and PIPfile.lock. And basically, PIPfile is the equivalent of requirements.txt, but with a slightly different syntax. It's not just plain text. It keeps tracks of your top level packages. And PIPfile.lock keeps track of the lower dependencies, the ones that you're not actually interested in doing so, but just taking note of this package, the relies on that package, link between them, and using hashes everywhere. And is that it? Turns out, though, PIPf also have some interesting features that we're going to discuss in a couple of more minutes. And beyond PPM, some people eventually stumble onto Poetry. And how does it different? It goes beyond. It also uses the same concept of two files for managing the dependencies, but this time they are calling the pyproject.to and the poetry.lock. But with the same ideas, pyproject.to.mail is a human readable file, while poetry.lock, not so much on the other hand, you should definitely try to avoid editing this file by hand. And with Poetry, it really feels like you're actually managing a project and not just a piece of code, like when you do with PIP or PPM, mostly because Poetry has some interesting comments that help you out, like maybe Poetry.need and Poetry.new project name. And it really feels like you can control everything with a single tool, including publishing and packaging, which you're going to see in a couple of more minutes. But I should probably warn that Poetry is dropping Python.to support soon. I'm not exactly sure when, but I saw it on their website. But if you're thinking of starting a new project with Python.to, then you're probably in the wrong presentation. You should be watching a presentation, maybe about migrating from Python.to to Python.to and not a presentation about package managers. All right, so after looking up into our package managers, I'm going to present a simple application that we're going to package and publish with each one of them. And this application is not going to make some billion dollars or maybe save some lives or anything like that, but it's just for fun. We're going to create the ShouldWeDeployToday clear application with the package name included, like DeployTodayP, DeployTodayPoetry, and that's it. And I'm not the owner of this website myself. I just find it really interesting, it's good for laughing because every now and then we always need a joke or laugh to make us happier. And it has a simple API, very simple, doesn't require any authentication or authorization. So you just get a response from the web and get a message and that's it. Simple as that. You get different messages according to the day of the week that you are and the time of the day. So getting started with PIP to package and distribute, we're going to start by creating a simple folder structure like this one on the right. We got our license, a read me, by project attempt, different configurations, a source folder, which we're going to create a module and set our code and a test folder. We're not going to write tests for this simple since this is a dummy application, but just in case you decide on an interesting project, that's nice to have right to think about it. And here's the code that we're actually going to use. We're just going to set it up in the source folder on the Dunder main file. It's just getting the GET method from the request library and get the API converted to JSON and just get a specific key and set it up printed on the screen on the standard output. We're going to of course need to get our request library into the requirements.txt. So over here we installed a fixed version and if it was successfully, we just take it to our manifest file, the requirements.txt. We're going to configure pyproject.toml to say that we're going to use setup to build the package, setup tools. And we're going to go through a lot of configuration. This is probably the worst slide this time presentation, so bear with me over here. We're going to set the package name, what's our name or email, some descriptions, the version. The configuration that I'm mostly interested in are the one on the right part, which is going to say where the package should actually be package, which is the source folder. Under installing pyres, we're going to say that we need the request library. And under the console scripts, we're going to say how should we actually call or client-online application. We're going to say deploy to the API and it's going to be loaded from the module, example package, the file name, and the thunder name method. So to generate the distribution, we're just going to use the build module that is available with Python. And after we tell it to build our application, we're going to notice that a new folder will come up, which is called the dist folder distribution. And inside it, there will be two files, the will and the target Z. They might come up in the internet such as be dist and sdist stands for binary distribution and source distribution. The binary distribution is actually your package compilates to your operating system and your architecture. And preferably, if the wheels people try to install if it is available for your operating system and your architecture because it's quicker, if it's not, then it will install from the source directly, your target Z, which is just your sort code, zip it. And after we build it, we're going to need to create a token on pipi.org, which is the Python package repository. And I'm actually using test pipi, which is like a staging area where you can upload your packets because if you try to do it on production, you're going to have pipi.org directly. And there are any people using our package and if you insert the bug, break your package, then you're going to make users kind of mad at you. So, preferably, always try to use test pipi.org. And after we grab the token, we just configure it with this specific file, set the username to the under token and place our token. And we're going to upload it to pipi with a package called wine. Just say that we actually want to use the test pipi repository and not the official pipi. Takes a couple of seconds to sign in and after that, we'll notice that both the wheels and the target Z will be uploaded. And we're going to install it from pipi, just use a different source than the official pipi, use the minus i option. And now we have our command line application ready, deploy today pip. If we call it, we get a suggestions that we should not deploy today. Since I was doing these lights on a Saturday night, he mentioned that I should not actually try to deploy it. All right, so, going for ppm, how does it different? We have the same folder structure and we just install ppm in the creative environment and install the request library. If we install the request library, ppm has already added for pip file itself for us. You can see in the middle that the request library was already added and the pip file.loc I'm not going to show it over here because I couldn't fit properly into this light, but trust me, you open over there, it'll be a big mess of a lot of hash. So don't change pip file.loc. To see the penises with ppm, an interesting feature that it has, you can actually see the graph the penises, which package, depend of which package and this is very helpful in case you have any issues with packets and works both ways either from top to bottom or bottom up. Some other interesting features interesting features that are available with PPM. We can actually check for security vulnerabilities and compliance with PEPs, install packets just as developer packets. And once we're actually satisfied, we just run ppm.lock and either update ppmfile.lock if necessary. And from now on, you actually achieve a true consistency. And if you just run ppm install dash dash in our PEP file, then it will use the .lock file and it'll work for sure. And as I mentioned before, we can actually already install any developer packets by just using the dash dash dash option. So what about packaging and publishing with PPM? On my research, at least I found that it was pretty much the same process on PEP, even a bit more complicated than PEP itself. So that's why I'm going to skip it and go through Poetry and also make it fitting to a 30-minute presentation where I was up. And with Poetry, it already creates the project folder structure for us with a simple comment as we can see on the right. It's slightly different, but it still works fine. And we add the request package and it takes a couple of more seconds. And after that, it is already added to pyproject.toml and Poetry.lock. We don't have to worry about anything else. Our source code is pretty much the same as before, just under a different folder. Under pyproject.toml, we have to actually say that the client's line application that we actually want, we change the name and the source where it will load a specific Python code. This is where it starts to get interesting. To distribute, we don't actually need any other tools. We just run Poetry build and takes a couple of seconds and it will build the source distribution and the binary distribution just as before. We have to also set the token and the test repository for pyp.org. And we can do it with this simple comments. Uploading to pyp is just a matter of saying Poetry publish. Also doesn't need any specific tools. A couple of seconds later, it is already uploaded. And if we install it from pyp test, just run this specific comment with p and we got a new application or a common line which will be deployed today Poetry. If we run it, we got another suggestion for if we should deploy it today. And since this was in business hours, I believe it was all start of the evening on Tuesday or Wednesday. Then we got a message, this is the way which is a quote from the modern lawyer. It's very yodel-like, very positive. And with Poetry, that's about it. And finally to wrap it up. As I mentioned before, a side-by-side comparison of each of the tools. My suggestion is that if you're just starting with Python, then preferably just try it with because it's probably the simplest of all the tools that you can get. But it's not self-sufficient. You will rely eventually on other tools and tools for virtual environments or packaging and publishing. As you can see, Poetry is probably the most advanced tool of the three because you'll not rely even for packaging and publishing. And once again, you should probably stay away from Python too. Just a reminder, this is not the purpose of this presentation, but it's worth remembering. And yeah, that's probably it. Here are the sources that I used for my presentation. In case you want to check it out later for more details that they were very helpful for me. I would like you to thank you so much for the European Python community and for everybody who has joined me for this presentation. I hope you guys enjoyed it. Thank you. Obrigado. Muchas gracias. Vielen Danking. Arigato. Xichue. And I'm still working on the Russian. I can't pronounce it. If you have any questions at all, feel free to contact me into any of these things or venues or Discord, any of the ones that are available on the European Python during the conference. You can also ask questions or doubts. I'd love to hear your feedback. Thank you so much. Thank you very much for your talk. And let's have your applause. We do have time for some questions. Do you mind some questions? No, not at all. Okay, we have a microphone here in front. Could you please get up to the microphone and ask your question? Thank you. Just don't touch the microphone. Just go down and ask into it. Thank you. Thank you for the presentation. So you mentioned a number of package tooling. So I was wondering between advanced ones like poetry, Konda, Mamba. Do you have a view on the trajectory? Like what do people use more? Poetry is Konda being faced out, is Mamba still? Konda, I'm not so sure about Konda. I'm pretty much sure that Konda is very used for anybody that uses, for example, Anna Konda, which is a way of distributing Python, which is more focused on the data community, like data analysis, data science and other. I myself am a web developer, so usually I don't deal with Konda or Anna Konda, but I know it exists. For those that use with data on a daily basis, I believe they are more focusing on using Konda themselves. And I noticed during my research for getting the examples that PIP, it's maybe at least in my opinion, losing a bit of momentum and more modern tools are taking places, like even PPM is a bit of a decline. My opinion, I believe that beginners tend to go by using PIP and intermediate to advanced and higher-piped users are more going with poetry, I believe, in my opinion. Okay, thanks for your answer. Are there any more questions? Please get to the microphone. Thank you. You've talked about poetry and, but there was an issue I encountered when I was developing with poetry recently. Basically I had a dependency, but when I installed it, I had an issue, and so searching the issue online, I fell on a GitHub issue, which mentioned that the package required poetry to be like in a preview version. So I had to build poetry from Git sources to actually add the dependency, which was quite weird. So I don't know if you ever encountered issues like this with poetry or... Yeah, I'm actually new to poetry. I'm just starting to use it. And I never saw something like that, to be honest, like to install a specific version of a package, you actually have to update your package manager. That sounds a bit weird, but maybe... I can, to be honest, I'm not sure what could be causing that. Maybe the version of poetry that you're installing was some weeks or months ago, that in that case it caused the issue. And that might be it. If you remember when you installed it, that might be the specific reason. Poetry is under constant development. Since it was new, something might have been broken during the development. That might be the reason, I believe. But usually package managers don't... That don't have to ever happen, I believe. I'm so sorry that you had that awful experience. Okay, thank you very much. That's about all the time we have for questions at this point. So let's have another round of applause for Vinicius for this great talk.