 Hello everyone, my talks about the ideas that have been floated for implementation in the Pacman package manager As a disclaimer, many of these ideas might never happen. They're just ideas and May not actually be suitable for implementation, or they just might never get anyone actually interested in Writing the code that's needed to implement them So the first bit I'm going to talk about is parallel downloads So this is already pretty much implemented, so will be a major feature for the upcoming 6.0 release Which we hope to get out in the next few months, at least a beta release So what this does is it downloads all the databases and the packages for updates in parallel So this improves downloads times quite a lot for those who cannot max their connection Either due to not having a close local mirror, or having a local mirror that's not particularly fast At the moment we do download all the packages from a single mirror Because we've run into a lot of mirror sync issues otherwise This may or may not be improved in the future And another thing we also do is download all signature files in the background So you're not only downloading the package, but also it's PGP signature as well That means in your cache you can verify the packages manually at any time This is all controlled with a single configuration option, so we've got parallel downloads equals 10 I think the default will set is about 5 You can set it to 1 to turn off parallel downloads So here goes an example of me updating my system using the latest pacman from the git branch So as you can see all the databases downloaded in parallel there It was pretty quick, I maybe haven't updated my system for quite some time And you can see all the packages downloading in parallel So as I see it in the background here, we're also downloading all the package signature files as well And you can see it works along quite smoothly So for me this is quite a speed improvement because I use a mirror that I don't have the fastest connection to And still had plenty of bandwidth left Another idea we've started planning out is an alternative system So these sorts of things are fairly common in other package managers I think Debian and Fedora have sort of alternatives But the basic idea is we currently have a user bin sh sim link And that points at bash But people have often requested using a different shell for sh With dash being suggested commonly Because it's a nice small shell with not many dependencies and pretty robust Now doing this in Arch Linux is currently not possible without rebuilding the bash package to remove the sim link first And so the idea is to be able to change where that sim link points But still manage it with your package manager so it doesn't change over updates There's a wiki page there with a bit of details on the implementation ideas at the moment So here goes a basic overview of how we're implemented So in the package build we'd let it know that we've got an alternative for this package So this alternative continuing with the bash example would be sh And then there'd be a file called sh.alternative or something like that Where we say we want user bin sh to be a sim link to bash So the current sort of style of implementation is we can use full or relative paths there And we can provide multiple files that are to be sim linked So for example in python for some unknown reason you might want it to point at python 2 But if we do that we might also want to use things like the idle to point at idle 2 And not the python 3 version and there's many other things python 2 related We might all want to change sim links at once So sort of prototyping what the pacman output could look like So here we've got pacman listing all the available alternatives on the system So we have python as an alternative and sh So python here could be pointed to python 3 or python 2 And currently on my system python 3 is the active user bin python as all should be SH could be provided by bash or dash and I currently would have it active with dash there So we might want to change that say move the sh sim link back to being the bash package And so I'd say pacman minus alternatives update Change sh to bash and then it tells me user bin sh sim link now points to bash As expected So that's still in the planning phases but getting a pretty complete plan for how that will be implemented We have a few ideas for how make package could be improved To make the packaging process easier First one is improved package splitting or an alternative approach to package splitting So when the original package splitting was implemented it was very much focused around how KDE was getting split up into multiple packages Now KDE was quite nice that in that despite building many pieces of software from a single source table There was install commands for each individual piece of software So when we split the package you could just go install this split package and it would be nicely managed Now other packages aren't quite so nice so if you look at our GCC package build It takes quite a bit to get the files into each of the split packages So this is an idea where we have our split package We'd have a function that stores all the files in some temporary location And then each split package would have a list of files that are to be moved from that temporary location into the package So here we've got foo which has the binary and the include files And a docs package which includes the docs for foo Another idea is improved user group handling This probably goes beyond make package and into how groups are handled within package files So quite a few packages require that we have certain users or groups to be created before the build process Sometimes this is not an issue because the groups are already available Sometimes the groups have to be created and other times we just use a number The group ID number rather than the group name to handle this So here for example building the cups package we have some files or directories in that package that are owned by the group cups So we might say we want the package group cups to be created before we build the package Automatic dependencies, so we currently have some sort of automatic dependencies and provisions in lib defends and lib provides And so they look for libraries in the depends or the provides array and automatically add versions to those libraries That that's useful but is a bit limited and there's possibility of extending this to include a range of dependencies and provides packages automatically Now if we look at Alpine packages which are fairly similar to the packages created by Pac-Man They have a section of automatic dependencies or provides added So for example if we look at libjpeg turbo it might provide a command being cjpeg Also has a SO, so a library of libjpeg version 8 and a package config file And then you could just depend on this SO library or automatically depend on this SO library and things would go well I note that these sorts of provides and depends would work well for repo packages But not so well with packages you've built yourself and installed for example from the AUR Because that results in lots of having to ignore some of these dependencies during upgrades before you can rebuild the package So we'd need to be able to turn that on and off depending on where the package came from Another part of makepackage we'd like to extend a bit is the extensibility of makepackage So currently makepackage has been started to be split into smaller files known as libmakepackage There's been a decent amount of progress made on that but there's a lot more to do So for example we can extend parts of makepackage so source code downloading and extraction It's very easy to add a new VCS system there so we currently support Git, Mercurial and SVN I think And if we wanted to add a new VCS system it's a matter of dropping in a single file We have things that manipulate package options such as we remove libtool files by default We could remove empty directories and things like that and we could add additional options for how we want our package to look here And then we've got a bunch of linting checks for both the package build and the package Which just check things are looking like they should do So for the package build we check that things like arrays are arrays That you have the right functions in the package build and all the variables are defined For the package we can check that it doesn't have files where it shouldn't have files etc Doing things very similar to NAMCAP does And in fact I've got a proof of principle called package lint Which is a drop in replacement for NAMCAP but it goes into makepackage itself So that has some advantages in that checks can be done in makepackage When we know where all the files are sitting and all the packaging variables So it gives us a slight bit more power This package lint is a bit early stages at the moment It only replaces a handful of NAMCAP rules And probably won't go a hell of a lot further unless other people contribute We've got ideas around improving the repository databases So as I mentioned with parallel downloads That the PGP signatures for all packages are now downloaded alongside the files So we could actually remove those from the database Now initially when we added them to the database everyone was using RSA 1024 keys And they didn't take up a lot of space for signatures But as people moved up to 4096 or whatever More secure GPG hashes The repo databases actually become overloaded with signature files So that's one step we could take is removing those from repo databases And then everybody's download to check for updates would be far smaller Another thing we're looking at is adding a timestamp to repo databases So this is very useful because we could say after two weeks this repo database is no longer valid Why is that useful is because there's an interesting attack on package managers That a mirror could hold back updates for a package that has a big security vulnerability And ensure everyone using that mirror still has the out of date and vulnerable package Now if databases were assigned The only way to do that would be to actually hold the entire database back Instead of just holding the one package and after two weeks of no updates If that was the expiry time we get told our database is no longer valid If something's gone wrong to us and we could investigate So we have a tool for managing repositories called repo add It's an okay tool but it was written quite some time ago and isn't very extensible So it also duplicates a lot of what we do in libator LPM already Because it takes a package, reads the information of that package from the file inside it And then adds it all to the repository database So it's already something we do in libator LPM is reading the package information We already write a database in libator LPM being the local package database So we could really replace repo add with something in the pacman library itself Now we'd need to spend a bit of time thinking about how best to manage repo's At the moment we just add and remove packages There's been some experimental tools built for managing repositories that do slightly different approaches which may be better The one thing that rewriting this would do would make extending repo add quite easy And we could start doing things like creating a source package database So at the moment we can create source packages with make package source And it would include the package build and any other local files needed to build the package Now we could distribute those for people to download It's essentially what things like the AUR do is get those tables But they do it more in a get way But we could provide links to download these source packages in the repo database Now there's a lot of questions like do we want to include that in the main database or a separate database Sort of like we do for file listings And other ones like are we just providing access to download the source package So someone could adjust it and build it as they wanted Or we're going to manage some sort of combined source binary update That seems like quite a big project and probably not suited to Pacman itself But there used to be a wrapper around Pacman that would do this And it would also automatically modify package builds according to a patch file you had So you could have it automatically remove a dependency you didn't want in there Or apply a patch that you needed Now we could also look at multi-threading support So in Pacman there's some operations that are very, very parallel For example checking signatures and checksums We do one package at a time Even getting all file lists out of packages Installing packages is done one at a time But that's a lot more difficult to parallelize But we could split this over multiple cores for some speed improvement there I think the initial patches for that have been floating around for a while But it's never been polished off Improving opti-pens So currently we only use them for informational purposes But when they're first proposed there was a lot of suggestions that we could have options to install All opti-pens automatically will query the user where they wanted to install each new opti-pend Rather than just displaying it Things like that could get quite intrusive in the upgrade So it would need to be an opt-in sort of thing But there's probably more we could do there to improve how we use optional dependencies Universal transactions So the idea is to be able to in a single operation install and remove packages Now this has been requested in the bug tracker since 2007 But no one's really done a lot of work on it And I assume it would be very useful for graphical front-ends Because you've got your graphical front-end You click, I want to install these bunch of packages I want to remove another bunch of packages And click go At the moment I assume they achieve this by first doing the installs And then doing the removes as needed But we don't have a lot of people contributing that have worked on graphical front-ends So I don't know if this is a major issue But this is something that I think we can definitely improve So the question is when will these features arrive And the answer as I see it is when someone implements them Or maybe never These are just ideas, they might not be the best ideas Some of the things I have discussed are well on their way So parallel downloads are complete to some extent I've got a bit of polishing to be done before release But it's already going well at the moment Alternatives and repo timestamps both have had initial patches posted But got a lot of work left to do then And we will be switching to using Arch's GitLab instance For development of Pac-Man when that's nice and ready to go So I'm told this is going to result in an inundation of new contributors As it's going to be a lot easier to contribute But as I always tell people Taking one of these ideas and running with it's all good But do make sure you talk to people before you do Because there's no point spending a lot of work on a feature That may never land into the Pac-Man codebase And with that I'll thank you And let me know if you've got any other ideas that would be interesting