 Well, I actually thought it was a rehearsal, but I see it's a rehearsal with a live audience, okay? So I'm not quite ready, but let's let's let's just do this But what can you do better on a you know lovely Friday afternoon other than talk about packaging, right? So what was I thinking about back in February? And hi, my name is Tiago I come from the whereabouts of Lisbon and beautiful and sunny Portugal which you should visit it's an amazing place and by the way We're having our first PyCon. I learned that yesterday And in the end of September or August check it out. Go visit beautiful weather cheap beer a beautiful place friendly folks other than me, so come visit now Back in 2015 and Bilbao to your Python. I met this gentleman Nicholas and Nicholas is not an orangutan Nicholas. Are you there? You're there, okay? So that's But after having checked with him. He liked this image It's the picture he uses for his get-up avatar and back then Nicholas was talking about the BBC micro bit and the new editor if you don't know about the new editor it's a very Beginner friendly Python editor and it's much more than that Actually editor to code runs you code debug to code works with all sorts of tiny and fun micro controllers like the micro bit back then And of course it's written in Python and it bundles Python now being beginner friendly Means it meant back then and it still means today. That is a it is a single file download So beginners don't need to go which Python version do I download? What about an editor? How do I make them go together? It's a mess and we all know about that So it's a single file download you install you go At least on Windows and Mac which is which are obviously the most common platforms that beginners and most people use around the world And if you don't know it, you should use it. It's it's very fun It looks like this you type your code you hit the play button. It just goes. Okay. It's amazing. So back then The new editor and Nicholas made beginners all over the world like this They were happy because it was easy to learn about Python and how computers work great But time passes and time plus is more and more and more and at some point users want more of course, and that's perfectly natural and They wanted to be able to use third-party power packages from pipeline with me Okay, which is perfectly reasonable and leveraging the huge ecosystem of things It's is one of the amazing things that draw people to Python now The development team gave it a few shots. So how do we go about this? And after a few attempts, we came up to the conclusion that well you should actually create a virtual environment Well behind the scenes beginners don't care about virtual environments and then grab packages from pipeline and install them there Okay, this is the technically correct solution It's actually working a few years later But that made us kind of okay stop What are we doing here? We we want to do this but packaging wise this is not going to be easy And now if our previous talk was about Packaging within the Python ecosystem. So you package something that you will later They've installed so we are within the Python ecosystem now shipping me you or a generic Graphical user interface application goes out of the Python ecosystem and out into the world to users that don't know We don't care about Python so Common thoughts and questions. Do we have the VM module on the Python we bundled so that we can create the virtual environment? more importantly Not as trivial Does the Python we bundle on Windows or Mac support TLS because we need TLS or aka SSL To speak to by PI. Otherwise, we can't really reliably download packages and the clicker clicks And we were back then already using some customized Python run times to ship with me In particular for memory. We had to add or hack or bake in tk So as to support the turtle module. That's beginner friendly So there was this thing of a consistent Python run time that we are as developers We were using building me and that we were shipping to users Beginners don't really care which version of Python. They're running as long as it's kind of three ish Okay, but the consistency of behaviors across platforms of these hidden behaviors is important so this kind of what was development focus how do we do these things and It's time progressed not only users and developers had these thoughts but platforms evolves and new Python releases came out And they're coming as out every year now. So it's crazy fast and on Mac in particular Apple required kind of a secondary additional stamp for software to run say natively and without kind of frightening warnings on users platforms So there's this notarization process that was new for us and the tools that we were using back then Didn't know how to do it and we didn't to know what how do we go about this? So move forward to 2019 at Euro Python again in basil and there's Nicholas And there's the new editor development team and there's beginners all over the world and it was girls What are we going to do about this and the development team goes? This is not going to be easy. What do we do? We don't want beginners all over the world to look like this Okay, we want to give them the new they want and the tools they can use to explore and learn Okay, and one of the culprits here was packaging me and particular on Windows and Mac OS These are the topics I mean Different tools per platform with different capabilities with different Python version support and everything does a mess, okay? It worked but it felt like daunting Move forward one other year and that's Tiago back home. Okay in a sunny Portuguese beaches amazing and I dare say the Famous first words of many many many open source and volunteer projects and that's a heart in this be Okay, who hasn't said it before this this get me done and so this August something at once something in the morning. What was I thinking? I said let there be pop, okay? So I created pop And pop is about native packaging But I I I rather prefer the word native feeling okay, and what do I mean about native feeling now? I want to bridge the world between developers and common and users, okay? So for a developer what's native is I can use any Python version any GUI toolkit any dependencies Okay, and I want to leverage them the knowledge I have on Within Python ecosystem packaging. Okay. I don't want to learn new things Now for end users Okay, they're not Pythonistas. They don't care. They just want to use an application They want a single file download or an app store download we'll get to that. That's the easy part They want the native Installation feeling on Windows it's like at the next next next and you never read the things you just go next on Mac you drag things on Linux. There's no native, but we'll talk about Linux And you want the native feeling for launching you want the start menu and with a funny icon or the Mac launch thingy And blah blah blah you want it to be just another application and This you want a lot you don't want to distribute Applications out to the world with a huge guide saying well if you right-click and open the CLI Type this funky commands. Hi there And type this funky command and then you can run the application. That's not gonna work Okay, users need to click a thing and there you go. There's your application So signing a notary notarization and everything that's very important and boring and hard So that's what pop Wants to do and it's it's been doing that at two points because we've been distributing you But package and they don't have lots and lots and lots of complaints. It seems to work So how's it go? You set up your project and make it pip installable Your project your gooey application should be launchable with python-m blah your project This these are the requirements nothing else Now optionally you give it an icon and the text license and the signing certificate for the full-blown experience, of course and Pop gives you a ready to use MSI file that it can install either as a user or as a system level So it can be available to all users. It gives you a nice user interface in the start menu thingy On that it gives you a disk image file That's native that may require agreeing to a license if you want and it's a native app bundle what? Users common use CS an icon and you can have it anywhere and just double-click it and it goes and Even though all of this is still very young and very exploratory even more young even more brittle and even more exploratory I've been exploring the app image thingy on Linux Well, and which is very very very very very very rough and it seems to work So this is essentially pop and you go from one to the other with a command a single command So let me give you a demo on the first act. Okay, and the first act of the demo is this This is macOS This is my project. I won't bore you with the code, but you see there's a setup p y Nearly empty in the setup CFG. It's mostly declarative. Okay, and I could pip install this thing locally But I instead I will pop package This okay Go let it do its thing and for fun. Okay. Let's do the same one windows. It's the same project There you go. Same project now all the time pop package Dot bam go do your thing and for additional fun. Let's make it well package Dot it's the same project go and we'll leave it at that now Now on on the Mac platform pop who'll talk to Apple servers and Tim Cook is there waiting for the request or it might fail We don't know so what is what is pop doing now? Click go what is how does it actually work? Well, it starts by laying out a full Python installation in your builds directory and By full I mean it includes everything like you need in the vm vm module. It's there. You need tk and turtle It's there. You need you are a little blah blah blah. It's there. Okay. It's the full standard library So it's native for the developer. Okay, it's cool and then it uses that Python installations pip To pip install your project into that Python installation Along the way it collects metadata and blah blah blah, of course And then it goes crazy and shuffles files around in a way that makes sense in The target operating system so in windows it needs to be laid out like this on Mac It will be in the application bumble which is actually a directory that has some metadata and files go like that and on Linux app image It goes a different way so it kind of shuffles files around and cleans up a few things along the way and lastly It uses native tools to produce a native package Be it an MSI on windows or just came in John Mac or at the images blah blah blah and Optionally throughout this process. It will sign and notarize the artifact So the final artifact will always be that single file that we want to say to distribute to our end users It's a simple well it is okay. Is there any magic is magic pink is magic blue? Well, I think magic is blue, but there's no magic here There's essentially this full Python installation that I can't escape. It's a full Python installation You don't worry about that. We all know that right But this one is reluctable and it does have minimal system dependencies and this is the magic It's not my work. Okay. I just I'm just putting pieces together This is the amazing work of Gregory sork and this project. That's Python standalone builds. I can only say good things about Gregory from both Technically and in terms of response time while exploring this I had to ask Gregory for help in a few on a few occasions and His feedback in terms of speed like two days later He's based in San Francisco and the accuracy of technical detail and analysis is just amazing. So Gregory you're the savior. Okay, so let's get back to our demo and see how it goes. Okay act to let's see if The team at Apple has done their work. Okay. The team at Apple has done. Okay. There's this process here Denotarizing apparently it has been completed. Yeah, it was really fast. Okay, so you see at some point it requested and waited for a minute So there's a distribution directory here, which I will explore Go and there's the disk image for puppy. That's a project. Okay, and I'll open the disk image And there's the license. I'll have to agree. Okay, this is my two license I agree and For those of you that are somehow familiar with the Mac This is a pretty pretty much native experience and there's your icon. Yes, it's mu inspires But it's kind of in a box because that's puppy And I'm dragging it not to the applications folder, but I'm dragging it anywhere. I don't care Things just work here or so Mr. Tim Cook says I'll double-click it. There's puppy. Okay, puppy is a GUI app it's actually an app that I created to Kind of explore pop and it shows me internals of the Python runtime So I see okay. How's the working directory? What's the Syspath and things like that. This is puppy on Mac. It runs It's apparently it's been packaged with Python 3 9 13 blah blah blah. It's okay. I'm not too much concerned with that I'll quit it. Let's go and see how Our other builds are going so this is Linux. Okay, lots of errors. Okay, don't don't don't read that the snids improvement, but I'll go to the test folder again And if I look at this, there's the puppy app image Okay, and I can run it and I can ship it to any haha Often any Linux system in the world might work. It won't but it there are the odds that it runs are Reasonable and this one is Python 3 8 the same puppy. Okay, it's an icon same application And I'll quit it. How do you do this on Linux? So you go there, and then there's Windows, which should be right about here. It's also packaged. It did sign the binaries and things, with a fake code-signing certificate. So this certificate I'm using on the Mac is good, but on Windows it isn't. But for the purposes of this, it's I'm running on my own system. Again, I'll explore this directory, go. And this puppy, it's an MS-Hack while I open it. And there's your installer with the logo and funny things and the licensees, and you accept it, and you go and you install, and this takes time, okay? But we have time, we have time. So let it do its thing. And I have ideas about making this process even faster, because I think this is slow. A puppy is a very tiny application. The Mew editor is bigger, and this process is kind of slow. And we want beginners to experience things immediately. But that's a different thing. So let it copy its things. Today it's done, and offers you to launch Puppy. I won't launch it here to show you that. We have a nice start menu with the same icon, blah, blah, blah, and I click it. And this is the native experience for Windows users. And there's Puppy, and in this case, it's running with Python 3.10. We can ask Puppy to use different Python versions. By default it uses the current whichever it grabs, okay? So it's running, it's okay. It's like a mini demo for you to see, I typed one command, and I get three artifacts for the three platforms. And that's nice. I like that idea, okay? So what's the status? It's dangerous, okay? So pup has no test at all. Testing this is very difficult, okay? Really testing that. Your software is picking up some other software and packaging as it should, and installs as it should, and it runs as it should. How do you go about that? I know that Russell Keith McGee of Beware is putting some effort and some thoughts into the equivalent and a great inspiration that's Briefcase, a project that we used before to package me, and it's putting some thoughts into how to test all of this in the next month. So I'll certainly keep an eye on that and have discussions with him. So there are no tests. And pup stands for Plugable Micro-Packager. While it's currently technically pluggable, the architecture API is kind of over-engineered there, completely badly designed here, so it's very exploratory. It will be pluggable in a useful sense so that anyone can kind of customize the build process or the packaging process is different, kind of different artifact or whatever. The docs exist and they need some love. There's need for platform polish, okay? I don't like the font that's used on the Windows installer. Maybe that's just me. We'll be sprinting, I'll be sprinting on Saturday along with the amazing Mew Development team. We're kind of one and we're focusing on that. So join us. We will certainly have an amazing time. So I would say the status is exploratory, but not YOLO, okay? Maybe in the future it will be, the lessons from pup will be integrated into other tools or maybe just thrown away, I don't know, okay? There's some commitment but it's clearly exploratory. Maybe some day, maybe Saturday, I don't know, tomorrow, pup can package single file scripts, fun for beginners. Maybe we can package Linux flat pack once you go, do a Linux app image, you got half the world of Linux. Oh, that's the worst thing in the world. You come to that because things in Linux and people just kind of like to fight, right? So I think I'm going to focus on Linux flat pack just to bring out the anger in the defenders of Linux snapd's or something. Yeah, let's wake up everybody. I don't know. The publish to stores is an important and easy step now, okay? Pushing these artifacts to the Apple thingy store or to the Windows blah, blah, blah store is easy now. Whether we want it to do, to do it like automated or not, I'm not sure, but that step is easy. Supporting other Python runtimes is a thing that deserves thought, okay? And I'll get to that. So this is the status, maybe the vision for pup. Let's get back to our demo in the third act because we clean up after ourselves. I don't want to step back yet. We'll step back in a minute. But here, I ran and installed, on Windows I actually installed pup. So I will uninstall it because I mean, I clean up after myself. So I run the app with thingy. And there's pup, you see it's installed natively. So we're here, where's that? Let me refresh it with this puppy, okay? Installed natively pretty much alongside every other thing that's installed here with metadata, okay? Diversion, blah, blah, blah, uninstall it. Anyway, it goes. So this is perfectly native for Windows users on the old school you download the file and you install it. On Mac, you just delete it, it's simple. And an app image on Linux, again, you just delete it and you're set. I'll leave it at that, I don't care. It's just kind of showing you the native process of it. It's very native, it is. It's simple, but that's it, that's what it is. Now, I have three minutes plus five, because I timed it into 35 minutes, to step back for a minute and think a bit what kinds of thoughts and questions does this bring? The first is, well, for beginners or let's say for volunteers. I mean, most of us do lots of things as volunteers. So I'm not going to be a professional in a given domain. Sharing applications is something important that we all like to do and that beginners certainly like to do. And okay, if you're working on the web, you just share a URL, let's say. If you're working server side, you don't care because you don't do that for beginners or for fun, but for desktop applications are important in many aspects, but they need to be signed. They need to be stamped. And getting one such certificate is not only expensive, but it's difficult, okay? I had lots of trouble with Apple, supposedly, you just pay and you go next, next, next and you go, right? But if you, my certificate for the Apple thingy, yeah, it cost me a hundred thingies, dollars, euro, I don't know, but there's like 38 different options. Which certificate do I want to create this? It's difficult. And for Windows, it's an even bigger mess. Who do I buy it from? Who do I trust? Will it work? Do you need even more seals and stamps to do that? So I wonder, and I have no solution, but where's Let's Encrypt for code? How could it ever work? Is it email-based? Because Let's Encrypt for, let's say, service certificates works based on domain name validation. How could it work for people? Based on an email? Based on something? I think this is important. It would allow us on Mew to have a button to say, package this and ship it to the world so that beginners and learners and people doing things could easily share their desktop applications. How do we do this? This is not a Python problem. This is kind of an industry-wide problem. I think it's important. And now, about Python. And see Python in particular. Now, we have been in the success of Python throughout the past 30 years or so as the, it's because it's very easy to integrate with other libraries, mostly see-written. So you can easily bring in other libraries or embed Python into other code. But that's, you need some level of skill that most people don't have, OK? The way POP works is much simpler. It's very dumb, OK? Grasp Python, it goes from there and shuffles files, and there you go. And it works. And POP works the way it works, because someone like Gregory did the work in building Python standalone, which is pretty much self-contained. Well, this distribution includes metadata that tools like POP can use to figure out what's where, what can I count on, what can I use? And maybe the Python we get from the PSF, Python.org, could include some metadata. We don't have an embeddable distribution other than a Windows only that does not include the central library, and that does not include this and does not include that. So it's kind of a toy thing if you want the full experience, but there's no such thing for Mac, there's no such thing for Linux. And Linux is complicated because there's not the single Linux platform there. There are bazillions of different Linux platforms and dependencies and libraries. So I think this deserves thought, because this is the path that Russell Keith McGee three years ago identified as one of the potential black swans in the Python ecosystem. And that's the ability of building and shipping applications that run on your desktop, and of course, ultimately, on mobile platforms, because there are bazillions of mobile users and bazillions of mobile platforms. So people consume software through mobile phones, and you either work server-side and you're on the web, or if you want to use a local native application, whatever that means, you're not going to do it with Python easily. So I'll wrap it up. Thank you very much. I want to thank, first, the Euro-Python organizers and volunteers. Thank you very much. Thank you, Nicholas, for the inspiration. Thank you to the Mew Development team. Tim is there. Vasco is there. Carlos is down there doing amazing things with MicroBits. Thanks to Gregory for his Python Build standalone project. It's amazing. Thanks to Russell for his inspiration and beware and lots of things. Thanks to Donald Stuth. He works for the Python Packaging Authority, and he conceded the POP name for us because it was his. Thank you, Donald. Thanks to Glyph. Great words on how to sign and notarize on macOS. Great blog posts. And he is actually the first person in the world that ran a POP package, macOS, properly signed and notarized. Thanks, Glyph. And thanks to Alastair, who was up until recently the maintainer of DMG Build, an amazing tool to build disk images on the Mac. So thank you all very much. Bye-bye. We have actually only one or two minutes. But if there is a question, and we have a break now, so yeah. Come here. So thank you for the talk. Very interesting. It brought up a lot of bad memories. I packaged PyGTK for using Py2XE for a long time. So one of the things I used to struggle with when packaging PyGTK apps was all the various Windows dependencies, I had to download separately, put it in all the appropriate places. So it would be relocatable when Py2XE would find it, bundle it up, and it could go. Does POP have to deal with similar issues or you don't have them? OK. POP is really dumb and stupid. And it's the work of lots of people. I just put the things together. POP is as stupid as I told you. It lays out a good reliable Python distribution and it pip installs. So if in development mode, your thing works when you pip install into a local Python, if it works, the libraries will work. On Linux, it's a bit harder because dependencies don't come along. But you say you pip install something that brings a wheel that has DLLs on Windows for QT or for GTK or whatever. They should work. They should be properly. They will be placed in the Python library, dynamic library, loading path. So it should work. It will capture everything from the VAMV. Yeah. OK. It should. But let's try. And does POP strip out extra things that are not required by the pocket to do? Not now. Not now. It's an optimization that can be done. Now you get everything. You can get everything. The only optimization that POP does is it's stripped away static libraries because the Python standalone project ships static and dynamic libraries and all sorts of things. It strips away those. And it pre-compiles the whole standard library and pre-compiles the whole project. That's the only optimization it does now. But it could be clever. Like, Py2xE does that. PyInstaller does that. It's clever enough. And Foss the Graph that Depends throws away things and make it kind of tinier and faster. It doesn't always work. There are many times I would have to explicitly add. Yeah. So POP goes the other way right now. OK. Take everything. It's just an extra megabyte. For now, it works, OK? But it's very exploratory. Well, cool. Thank you. Cool. You're welcome. So first of all, thank you for the talk. It was really awesome. Secondly, I like the fact that POP is stupid and simple and you can't get the complicated dependency graph problems that are pain in the neck to debug and blah, blah, blah, blah. So simplicity is a super power as well. This isn't a question. This is just me sort of spouting, really. I know we're over time and things. And the last thing is you said that POP had no tests. Well, music has been downloaded hundreds of thousands of times since we started using POP. And given that number of downloads and the fact that we don't get that many bug reports about installers, I think POP is comprehensively tested by beginners coders, but by definition that means they probably don't know what they're doing as well, because this is their first attempt at installing kind of an IDE or something like that. So applause. I mean, the purpose, I mean, if there's a purpose and objective is making your life easier. That's the initial motivation that they are. Oh, it's a mess. What are we going to do? Well, it should be easy, right? It's not, but it seems to work, OK? So let's try it out. If you want, join me on Saturday and try and package something else. It will probably fail, and we will fix it. Or not. Or we'll go grab a beer. It's OK. So again, thank you very much. Thank you.