 Hello, so let's start We're going to talk about package managers, and I hope that's where you're here for if not. There's a door on each side This is actually not a cathedral. This is a church. It's a la Sagrada Familia in Barcelona Which has been under construction for over a hundred years and they plan to finish this by 2026 roughly That's the plan today now before we start Two questions for you. How many of you use a distro package manager on Linux? pretty much everyone How many of you use a language specific package manager like npm, pipi, ruby jams Okay, and how many of you Rely on the distro package manager to provision your language specific packages It's much fewer, which is what I was expecting so some do but most of them do not And and that's that's a bit the topic of the talk So a bit about me my mission And that's what I do in and out of work is to make it easier to reuse free Libre and open source software I'm the developer and maintainer of several packages A series of project called about code. That's my primary set including something called scan code Which is a tool for scanning for license packages and origins I contribute to the Linux kernel become a stress and and many many other projects the question that seems that bothers me Why each time so I run Linux on all my machines? Why each time I run? APT install APT upgrade things works 99.999 percent of the time mostly and that applies to to Debian based distros or or RPM based distros and Why when I do a pip and npm a bundle install things don't work much more often than nine nine nines, right And you're laughing so obviously that probably resonates a bit it's Since don't always work they don't often work they very often don't work at all And it bothers me because I was actually bitching against distro developers and says why don't they get the latest version of the packages I want and and Why do I have to not use the distro manager which works and use another package manager that doesn't work? But gets the stuff that I want Right, that's that's the problem So why did I call this talk cathedral? Talking about cathedral the cathedral in bazaar. I don't know if any of you were born last century I'm an old dog The last century there was somebody called Eric Raman family name called. Yes, sir The focus seminary say called the cathedral in the bazaar and This is not exactly the same cathedral and bazaar as we're talking about today, but this point was how come that a development process as messy as the development of Linux Where patches are coming from every places ends up being? Able to build a system that works and that works pretty darn well most of the time One of the things that things were released very often on the one hand a lot of betas and Everything was the negated Whereas his point was that another style of development is more of the cathedral approach where you are to take things nicely and You are a bit less open about accepting contributions now That's just just just a side note, but I reuse this because it resonates in in the world of package management there's really two two types of packages package cathedrals and That's the one for for this rose and and the bazaars that's the one for language language specific packages So the one we use for rust node Python Ruby and the likes and They're really not at all the same packages. They're very very different and we call them all packages But that's probably a misnomer The distro package works typically for a single operating system Right, you're building for Fedora Debian and so on you don't have to worry much about your code running on Mac and Windows most of the time that may be a concern of a Language specific package whether it's built in C or C++ or In an OS agnostic language that requires some native dependencies The the package management systems are general purpose for this throws and very specific for sorry and very specific When it comes to language and as I heard before, you know, okay Scornan is a C C++ package manager Can you build no Java package with it possibly and you can probably do that? put JavaScript in in pi pi and and Java in n pms, but it's usually a lot of hacking and tweaking that's required there. It's it's not their primary purpose The other thing is that if you think about a package if you're a library author or consumer It's really about authors and individuals. There's one or a group of person that provides a bit of code That's supposed to be very specific in terms of function in the context of these throws. We have usually a team of maintainers which were collaboratively together to Not package one but package many package that are related and of the same family There's there's another context also yours are a user or a builder Which differs? the testing is Interesting how how many of you do know a project called Apache gump? Okay, it's it's pretty obscure project. It's actually interesting and weird They're taking all the head of every Project at Apache and try to build them all the time with the letters of head So each time there's a commit they rebuild everything against the latest and greatest. Obviously, it's very often read and doesn't build But what's interesting to to understand is that when you build this throw you don't care about having Just the test of one package But you want to make sure that all your packages for a given version release work with the same version of leapsy and the same version of this and that And you release them together. You don't really is just one sometimes you release one that If it's usually that version it's going to be always in the context of a larger hole and the main destroy release The other thing on the dependency side the dependencies we expressed most of the time for language specific packages are pretty simple whereas the dependency language for This throw packages are actually pretty rich. There's a lot of things which you look at them says conflict that depends resolve absolutes There's there's a very different context of managing dependencies The other thing also and most of the time when we consume package from a language repository we do consume these as is No, you take a maven jar verb at him. You take an npm 99% of the time you use it as is the packages that are Provided by these throws in contrast are 99% of the time patched modified deconstructed reconstructed you take a spell which is spell checker one nice star ball of stream in destroy it has a being Possibly 50 packages one for each language for the dictionaries Maybe one for the library one for a command tool command line tool So we are not talking at all about the same packages Now if you take the the the builder perspective The bazaar is like a box of Legos and The cathedral is trying to build an actual saying with these with these Legos so now Thinking again about your perspective as a user Even if I'm a developer so forget your developing software for a minute as a user you want a system that's stable that works That installs in a bit for the time Right, that's what you get from a distro as a builder. I want the Lego bricks that I can beat my cathedral with and I don't care too much about The rest I want to make sure they're really fit for their function They have a single well-known purpose and I can use them as a brick as a stone and assemble my edifice So now let's dive a bit again into dependencies Most language specific package managers. I said most this there's always variants Usually two options maybe three options on how you express dependencies and how and when dependent packages are used to usually build set up Time or run time And that's pretty much it and it works actually quite well You rarely need much more than that some package manager. I think it's a PHP composer Exposed a kind of distro like more richer language with a lot of verbs and Type of dependencies. I'm not sure they're very much used But it's it's a rare it's more of an exception The other thing is that in most case language specific package managers have a fairly simple way to resolve dependency conflicts so dependency conflict is when in your dependency tree to Packages require the same package with a version that don't match NPM for instance as a slick way to resolve that by saying I don't care. I'll take all the versions Which can be a problem. I'm sure you face that at times In contrast This whole package have evolved to have fairly complicated and complex Way to resolve dependency using what's called a SAT solver and Boolean satisfiability Which is really something that's Very simple at the base is but extremely complex in terms of its implementation. It's eventually an NP heart problem and To the best of knowledge, there's very few if any language package managers that have or that that use any SAT solver based the dependency resolution One of them I think is condar In the Python scientific world But it's quasi distro like package manager rather than being very language specific There's no such thing in bundler NPM or pep for instance The other thing that makes a big difference is the kind of social contract that exists Between the language specific package management environment and the distro package management environment again In the language specific world, it's usually individuals When I say individuals that can be a team, but you usually producing one brick one Lego brick at a time You focus as much as possible on a single function There's no coordination that's required most of the time with other package managers You just consume their stuff. You don't care what they do You don't care about their testing and their whole stuff. You just care about eventually In the confine of your single project Are my dependencies and direct dependency result and working but you don't care about the rest and it's it's a bit having a utilitarian Look at things do you think about plastic bricks Legos you care about, you know the mold injection molding the quality of plastic You take in contrast when you're building distro It's more like art and architecture You go to the bazaar and you pick the bricks the Legos you want to make sure you know their characteristics in terms of plastics and else But there's a really extensive effort for collaboration and coordination and in fact at some level there's Little code that's produced reasonably speaking and relatively speaking to the the rest of the the scope and And and a lot of discussions So that's that's really a very important distinction now Let's let's turn the things a bit around in fact you could say probably as each package That's right as a library or an individual project as a mini-caterials of its own You care about testing you care about ensuring that your code works in the context of its dependencies and that doesn't crash all the time And think of it as if you were taking these small cathedrals and assembling them further up So it's really eventually not really strict separation between on the two, but it's it starts all the way down and And that's what makes eventually things work So there's there's a way to eventually reconcile that at some level so What happens today really is that there's really these two contexts either I'm a user or a builder and I'm usually both at the same time. I am Building components and package for them to be reused and I'm reusing mine and further package to build system and applications And that's building of system of an application is a system integration job. That's what the new distro do What's saying that that that's of an overlook is when you start using for instance containers You very quickly actually are becoming your own system integrator you escape the confine of the nice assembly and coordination work that's done by distro maintainers and You take the responsibility without really knowing it is Each container ends up being eventually as complex And as messy as a whole in distribution But usually there's not the same discipline and efforts of coordination and collaboration that's been put into it. So that's important and So I will stop bitching and I think we all should stop bitching about out-of-date distro packages and We should work with distros to help them in many ways One if we want to make sure that our latest language runtime or package manager is available Well, we can help you don't have to be the maintainer of the packaging tools to help on The maintenance on maintaining that package proper and actually rarely enough. There are very few upstream package authors That are effectively the maintainer of their own package in distros Because there's one upstream authors and many many distros eventually So you can help this way and The other way you can help is when you're building packages and I'm terribly bad at that Make sure that they're actually Easy to package upstream by distros and possibly you can provide a package manifest for your favorite favorite distros Whether it would be brew on a Mac or a Debian or a PM package Which can help the work upstream and you can collaborate The work downstream and you can collaborate with The package manager and distro to make your package easier and faster to package and So really I think that's important in sovereign over look Distros don't happen by accident It's a lot of hard work and coordination that's needed and and we all need to give a bit of love there And that's it Just a plug. I'll be doing a talk on Tracing build with a trace In another computing room now be back after that in this room here. Thank you any any question How That's a complex topic so the question is What about the repackaging of language specific packages in distributions? Yeah, and especially the number of packages that may exist Which are typically in the tens or hundreds of thousands in an upstream in a language specific package repository environment It's a difficult thing on the one hand distros being software often have direct dependencies on the specific package and package version and Take the case for RPM base distros for a long time They depended on Python 2 4 and then Python 2 6 even though It was a long time since Python 7 Python 3 were available So they have this need of dependencies of their own To actually run on the one end and on the other end they want also to provide some level of coverage for Language specific packages. I don't have an answer. I don't know what's to be down there The the reality is that most of the language specific packages are Often considered that outdated when they're in the distro even in the latest release of a distro and Folks tend to not do system-wide installation, but always project specific or user specific installation There's probably I don't think there's a really easy solution one would be to repackage everything, but that's like double packaging Probably mimic sense to Instead work with distros to ensure for instance to work with node that you have the latest node runtime and npm installed same thing for Ruby and Python and probably Let go at a distro level of Language specific package repackaging unless you have a direct transparency and a need for a system-wide installation for it That probably would be the best bet That probably would simplify the work too the difficult Lots of applications Yes, it's not an easy there's no easy solutions fortunately that Yeah, so the question is is it's does do containers and docker Provide a solution to the problem of distro versus language packages I don't think so they actually amplify the problem at some level What is a container it's essentially a slice of or it's a mini distro It's a slice of user land to have everything but a kernel But you have everything What it provides is isolation and what you give up for isolation is immutability, so your container is immutable It's not easy to to update eventually you need to rebuild it. Otherwise, you're just accumulating levels of our levels of updates The Thing is that you have one benefit which is you can trim down the set of dependencies To just what you need so that's a good thing You can isolate them that's also a good thing, but there's very simple ways to do that just Putting using user level installations It's supposed to help with deployments on a massive scale So that has benefits much simpler to deploy a simple file single file mini machines than than a lot of packages The fact it's immutable it's good thing in some case It's also pain when you have a security bug and you've deployed 200 containers each with small variations You have to rebuild 200 containers. It's not fun So it's I'm not sure it's really solving any of the problems there It helps in some case it creates a lot of other issues in the other end the other thing talking about containers is that When you actually redistribute containers or deploy containers you are the factor your whole stack Linux integrator you took responsibility for the whole stack above the kernel and and There's much more ceremony that's needed there in terms of compliance and license bug tracking security tracking Things that were provided to you as a service by distros that you may no longer benefit of all the time Yes And everywhere and can run everywhere and then new designers So so the question is Is this dichotomy between bazaar and cathedral something of the past or will there be something that's more language agnostic Yes, it's not a cathedral of bazaar it's the opposite that's kind of true But it's like a modern solution to something really really all like distributions Is something is a factor So the the difficulty there is It's hard to build package managers and actually there's I think there's There are talks dates. It's really hard. I've tried a few bits of time and I fail lamentably every time So it's it's a difficult job. I've built distros not Linux distros but Long time ago an eclipse-based distribution which was called easy eclipse and Which had at a very small scale all the problems of that you can have in the Linux distribution It's again the problem between the buzzer and not the buzzer. It's not so much a technical issue It's more problem of coordination and and this coordination. There's no tool that can actually fulfill it you have to have people that discuss and if you see all the the efforts that's deployed by a Distribution teams To have package trackers in ways that they can collaborate and track all the same. That's where the big effort Now there's maybe a bit of light on the horizon There are kind of two emerging things on the distros side One is next and on the package management side one is pack and Which which provides can very different take on the whole problem good Is the maintenance pack by the way So you said that we should give the distros some love I guess my question would be what can the distros do For the packageers and what can the packageers do for the distros to grease this kit? Because I mean if you think about what a distro is it's all the glue that was required Given the certain set of basic assumptions to build all these packages on top, right? So what do you do to automate the glue? Yeah, well, that's the problem you can't The glue eventually when everything is a package the glue is what you and I do to actually Take a bunch of Lego bricks and make it a system. Okay, so time's up. We can take the discussion offline or later I'm sorry Very much. Thank you very much Nick's which is a Linux distribution Nick's like this and IX and SPAC SPAC