 the software should meet, and it takes a long time for software going from upstream to into the distribution and being shipped to the users. So mainly the AppStream always wants people to use the latest software, or at least have an option to use the latest software. So in case you are on a stable distribution, like they're being stable, and want to test a new version of LibreOffice, or a new version of some KDE or GNOME software, the AppStream basically needs to wait for the user to upgrade the distribution or to compile the software or to do some other stunts to get it available. If the user does not want to upgrade the distribution or upgrade to a rolling release. So the software usually goes to a distribution package who wants to offer too much certain quality standards before policy requirements and basically cares for system integration. Because that's what we do in WN, we integrate software coming from AppStream into the system. And we are caring for the operating system part that the operating system is stable. And yeah, this goes at the expense of distributing applications quickly. And the distribution itself should basically work solid because nobody wants parts of the distribution exchanged in between unless he is a technical user and knows how to deal with problems which might arise during up kernel upgrades, Xorg upgrades, or whatever happens. So a rolling release distribution which you might assume would be an answer to this problem is not the solution for the user I have in mind in this case which is one who has no idea how to fix problems in case they happen. So I think, can you still hear me? Okay, great. Then in order to fix these issues, the presentation issue and the distribution issue, several projects have evolved to deal with it with this AppStream metadata specification to solve the presentation issue. And LEMBA and XDGUP which solve the problem or which attempt to solve better said the problem of distributing software. There's also new stuff like firmware update which is able to update UA5 firmware which is a new emerging project and which I will not talk about although it's very interesting and you should take a look at. There's a working group and Debian forming right now to bring it to Debian so you will hear about it more in future. So let's start with AppStream and what this is actually because I made quite some noise about it so maybe you have heard about it already. AppStream is basically a metadata specification for every single software component. It's basically just XML which AppStreams create and ship with their software and which describes some information users find valuable and also some technical details about an application. Yeah, those software components can be anything from firmware to a desktop application to an add-on extending an existing software to codec to input methods and to fonts. This is already a distinction from the previous package model where you now have different categories of what the software is and which users can search for. AppStream metadata is also created by AppStream projects. Sorry about that, AppStream, the AppStream project and AppStream is kind of confusing if I talk about it. Yeah, well, it's created by the AppStream metadata, it's created by the AppStream projects and the distributions only transform it and review it so there's no duplication of effort anymore because the AppStream project will write all the details about the software, the description, even translated ship screenshots and stuff so we don't need to deal with it anymore. You just need to check if it's sane and can we ship to our users. And also the AppStream XML metadata does not replace existing metadata but instead extends it. For example, if there's a desktop file present already, we will not duplicate the information which is already in there but instead extend it. For example, a desktop file does not have a long description so you create an AppStream metadata to add a long description to your application. You will find this stuff installed into user share updater and one thing which is important for Demian packages is that you should ship this metadata file together with the desktop file in the same package and also in the same package where the application is in which should be installed in case the package is installed. Reasons for that will become obvious soon. So how does it actually look like? The AppStream XML, not sure if one can read this but yeah, it actually doesn't matter. So one key point of the XML ship by AppStream is that it defines the universal and unique identifier for a software component. This is multiple benefits for if you want to search for a software component and on a foreign distribution where I don't know the package name because this ID is unique among average distribution. So this also means that AppStream people can request a software component with this ID and ask the system to install a missing package in case you, for example, have a file manager wanting to access a network share via SAMBA and SAMBA is missing on the system. The application can ask the user if he wants to install SAMBA to fix this problem. This leads to less craft and less unneeded components while still being user-friendly and being able to install missing stuff in case it's needed. The metadata also contains quite some other stuff like a long description of the software or translation. So we do not need to re-translate the stuff which is already translated AppStream and those translations are shipped to average distribution so there's no duplication of translation effort anymore. Although we could, of course, extend it, but yeah, all the work for this metadata creation happens AppStream and is shared between distributions. It also contains some URLs like AppStream project links, links to manual pages, links to the backtracker, donation page links and everything which the user might find useful to learn more about the application or to get help or to get information about things. There are also some screenshots URLs included. So in this case, no foreign person uploads a screenshot for the application which might badly represent it in AppStream's view but AppStream itself creates a screenshot how he or she wants his or her application to be represented in the software center of the distribution. One other thing which the AppStream specification provides is technical items like binary ship with the application on MIME types and this can be used in case the user wants to open, for example, a file type which is no application is available to open it with and then the software center can search for a component providing support for this MIME type. So in general, it's a way through metadata to make the system smarter on what is actually needed to solve a certain task. So now there is also distribution XML which is the thing which we actually ship to the user and which we create from the AppStream metadata. It is a JAML file in Debian and XML on any other distribution. The reason for that are historic but mainly our FTP masters do not like XML and ask me to write a JAML specification instead so I created a different specification so we are in Debian and all Debian derivatives are the only distributions using JAML as an intermediate format to ship AppStream metadata. It's called Debian 11 because it's the Debian extension project 11 and you can read about it in the wiki. This might sound bad at the first but currently all tools are able to handle both the JAML files and the XML files so it does not matter what you ship if it's the Debian, JAML or the XML. This files make the data accessible for software centers like GNOME software or KDE's Mure and Discover which is right now in development and which will be shipped soon. Yeah, and it provides the component, the software component to package name mapping to install stuff which is required. What we also do at the distribution side is to merge metadata from the desktop files into the final XML so you will have one XML file containing all metadata or one JAML file in Debian's case. And we also extract icons and screenshots and other components. The main reason for caching screenshots is legal reasons because we want to be able to in case there is something inappropriate and inappropriate screenshots we want to be able to delete it and we also do not want some foreign URL to be linked against from an inside Debian from the software center so we basically cache it at the Debian side to get some control over it. The architecture looks like this now so we have this compose server at the Debian side which reads all the package files, extracts all the metadata and then creates the XML, well the JAML data in this case and a top ball of all the icons on the mirror. The mirror will be the archive itself and there are still negotiation missing with the FTP masses to get the data included into the official mirror but all the apt parts are ready so apt will be able to download this data additionally to the client where it will then be transformed into a sapient database which is then consumed by the software center. The GNOME software project will not use a sapient database but read the data directly and set but in the end it doesn't matter if the data is available as soon as it's downloaded by apt onto the client. We will not download the data unconditionally but it will be a setting which is likely set by the software centers themselves to advise apt to download this data. The software center itself interfaces with package kit which is a package management abstraction layer available in almost all distributions which is a distribution agnostic way to install software. There's also work going on on a feedback server where users can rate the software and review it but this is very early work. There's some communication going on with the people who are trying to take or working on the new open curve collaboration service specification to get this into AppStream later. So important for Debian, how does it look like? You can find all the generator services on AppStream.debian.org where you find an overview of how much metadata we extract and how much metadata we do not extract but could probably get. Right now it's about 15,000 different software components but there are a lot more and there are a few issues to be fixed in order to make the data extraction work flawlessly. There are also still a few bugs in the metadata generator which is the reason why I did not announce it publicly on the Debian devil announce mailing list but as soon as the app is able to download the data I will do an official announcement of it but yeah, this is some preliminary work. This service will show you just like Lintian or why your application was not included into the metadata which is I would say in 70% of all cases missing icons because for some reason the icons could not be found in the archive or are only available as XPM images which we cannot reasonably ship on modern software centers. It also shows information about the extracted metadata. This is the Debian 11 Jammel file and the Jammel incarnation of the upstream metadata and yeah, it will show you what the generator actually extracted for which package and for which component ID and it also shows which I could have found for it because sometimes GNOME applications get due to various reasons a KDE icon and we still are working on solving that. So this is what the users will see. It's GNOME software. If you look carefully you will see that the screenshots originate from Fedora but it looks pretty much the same on Debian if you have the data installed and you will have multiple screenshots even with captions then you have long descriptions which is translated to the user's native language. Some details like license is used, how much space it will consume, the categories which it is in and also add-ons to install in order to enhance this application even also with user-friendly descriptions. Also the software center is able to handle codecs automatically in case they are missing it can install them using the provide system and it is even able to install input methods in case they are missing and fonts. So yeah, this is about the presentation issue of software. So now moving on to the distribution issue which is covered by other projects. Just remember why we will want this. It's mainly to obtain new software versions which are not yet in the archive without having to upgrade the whole distribution and to obtain new releases of software which is in the repositories but we want a new version. This happens quite often with web applications but when I talk about applications I mainly think about desktop applications although all those methods, outlines, outlines here should also work theoretically with web applications although I think nobody tested it yet. So if you read this you might immediately think well we have a solution for this already it's called PPA, Personal Package Archive or Repository because even if you're not associated with Debian you are able to create your own repository build your software there and provide it to the users. However there are several issues with PPAs mainly they are highly insecure because during installation time already you are giving root access to a foreign entity on your machine which is like the worst possible scenario security wise. Also which is another issue those additional packages might break distribution upgrades because when we design our system at Debian we cannot account for every possible third-party packages might land in the repositories and where the user might not uninstall it before dist upgrading. So we might get bug reports about something blocking upgrades which is not ideal and which happens quite often since Ubuntu has a PPA so you will I think in every release you will find bug reports of users not uninstalling the PPAs properly or having something breaking upgrades because some third-party developer was not careful enough or the packaging at the Ubuntu or Debian did not account for third-party stuff. Also another issue is that PPAs are obviously distribution specific many third-party applications are now packaged for Ubuntu but not for other distributions which is kind of unfair not even Debian has access to Ubuntu PPAs they sometimes work, they sometimes don't and this greatly limits the usefulness of the PPAs and also shifts the focus of our users more to Ubuntu because that's where the software is easily available where on Debian you only get the stuff which is packaged by us and the distribution repositories. There are the Debian package repositories soon but those will only be available for Debian developers for obvious reasons mainly to solve the first problem so it's not a full solution to the problem because we are only few and upstreams are willing to ship the stuff basically on their own and do not rely on a package to do it. Also PPAs are quite an overkill if you just want to distribute an application. So in order to fix this or work around this depending on how you look at it several projects have been involved one of which is Limba, it's a software IDvelop which is a successor to two other previous projects which mainly failed because the underlying stack was not ready for it and they also were basically just proof of concept and experimental projects. However, Limba is something I'm pretty sure will stick around. So one of the key concepts of Limba is to split the distribution into an operating system part a runtime part and an application layer. The operating system is all the thing you would expect to be in an operating system like the kernel, whoops, sorry, XORC, Wayland, SystemD, PulseAudio, Mesa, D-Bus, et cetera. One thing which these components have in common is that they have stable APIs or are D-Bus demons which can be interfaced through stable D-Bus interfaces. The runtime, however, is something like toolkits or shared libraries which are changing ABI and which might need to be available in multiple versions in order to make the applications run because some might be linked against varying versions of a component. They need like FFMPG or different versions of GTK. So this part is handled by the distribution itself while the runtime bits and the application bits are handled by the Limba installation system. One key element of Limba is the so-called bundles and those bundles will ship the runtime parts and the applications. A bundle is basically just a tower ball containing upstream metadata, dependency information for other bundles this effect depends on, signatures and payloads data, which is just a file system tree containing binaries and libraries and all the stuff you know already. Those bundles can ship both components making up the runtime and components making up applications. So it basically looks like this. We have a bundle for everything and a bundle for the application. In this case, we have for example, Clementine as an app which depends on Qt5 GUI, Qt3 Gstreamer and whatever. Those are the runtime bits which are shipped also as Limba packages and we have underlying the OS which is Debian, Fedora or any other distribution. Yeah, those components are shipped by the respective upstream project. For example, in this case, the Qt stuff would be shipped by the Qt project while the Clementine bundle would be shipped by the Clementine project. So if you now have all these different components, the question is how we put it together and to form one thing and without needing the applications to be relocatable in order to find their dependencies. The answer is relatively simple. It is using OverlayFS to overlay all these different bundles and the application into one new file system. OverlayFS in case you don't know is a virtual file system which is able to overlay existing file systems into one new file system. And we then create a private mount point and for the application and run the application from that. So each app will get its own mount point will be able to find its resources and will be able to interact with the operating system. We also do some other new kernel features to contain the application a bit and to make it easier to handle. So with all those different bundles now and application bundles and all those things, you might think that it's well-be-confusing for the users to get all this stuff but it actually is quite simple. If the user wants to get a Limba bundle, they are the first place to go might be upstream to get the bundle from there and install it into the stable system. But those Limba bundles can also be combined in repositories and which has been integrated into any upstream compatible GUI tool. Because if you remember, every Limba bundle contains an upstream metadata file which describes a unique ID for this specific software component and which makes it possible to also extract this data and create compile a metadata file for it which can then be fed into the software centers in order to make the Limba bundle software available just along the stuff which is already in the distribution. So the integration is really nice and GNOME software already contains experimental support for Limba for installing and removing Limba bundles in case you have added a repository. So since these bundles can be installed in a conflict-free way, there's no issues with dependency satisfaction. So dependencies are always satisfied and all this stuff happens transparently so the user does not need to care about it. So although it might sound complex for the user, it's really simple. One thing which I think might evolve is the so-called Limba Hub which is a web application which is able to take Limba packages and create a repository out of it. The main idea is that upstreams create their software and then submit it to a sentry repository where some QA can be done on it like using the ABI compatibility checker to check if they broke ABI and then reject the package in case they did it or to do additional QI which the Limba package generator does not do on its own. So yeah, this is a one thing which must evolve in case we are going down the Limba road just to coordinate the efforts of upstreams to ship their software to their users. So the strengths of Limba are the conflict-free installation of software that you can share a lot of resources via overlay FS that it has some safety checks for ABI compatibility but the ABI issue is also a weakness of the Limba concept because upstreams need to care to declare correctly if they, when they break ABI and when they don't because if they do not do that there is a risk of breakage in the Limba system. It also does atomic updates which means that if you run an application like Firefox and then it gets updated while running the Firefox will still see all the resources available when it's started and will not use the new Firefox files and get confused by the new stuff there which is the real issue there. Same with LibreOffice and which is one of the reasons why the GNOME project started to do this offline update stuff where they reboot twice in order to install updates to ensure that the system is not, or it's still using software which is currently updated. So, and this is also a huge issue for the Chromium project where they go through its extreme length to avoid issues arising from software from Chromium being updated while it is running. It also has this tight upstream integration which has some nice side effects for example due to the unique identifier which is in every software component it is easily possible to map the install software against the database of CVE vulnerabilities for example to see if the stuff which is currently installed in the system is vulnerable. This is of course not interesting to the user we just should ensure that there are no security errors but it's quite nice in case you're administering the machine and want to check if everything is sane still. So, this is about Limba but there's also a second project being developed with this solving the same issue but using a slightly different approach which I also want to talk about. It is called XDGUP and mainly developed by Alexander Larsson of Red Hat and GNOME who was working on stuff like this for a while and yeah, XDGUP started to take off after Leonard Pettring wrote this blog post on how we put together Linux distributions where he outlined using better FS to ship containers for applications and stuff like that. So, XDGUP is inspired by that and is able to fulfill the requirements outlined by Leonard. So, Leonard is not involved so it's a project of mainly Alexander Larsson and many other people and currently developed like GNOME but they recently moved it to the free desktop infrastructure. So, the main difference between Limba and XDGUP is in the runtime part because on XDGUP every thing, the runtime and the application are always tree repositories. Always tree is in case you did not have experience with it yet is basically git for binary file trees. It means that you have a file tree and you can commit to it to this operating system tree and to switch back to any point in time. So, you can see it as a snapshotting service and it also has it on repository format where you can pull stuff from so you can get file system trees just like git and it's optimized for binary files and not for text files like it is. So, the runtime, first of all, XDGUP also makes a distinction between the OS, the runtime and the application but the runtime is an OS tree repository and the OS tree repository is built by KDE or GNOME or any other large entity and contains all the dependencies a software could need in order to run or an application could need in order to run to be more precise. And this means that GNOME will, for its runtime and SDK, add things like GTK or any other GNOME component which is useful while KDE will obviously add the KDE frameworks, files for it and anything a normal KDE application needs to work properly. They also ship together with the runtime and SDK which is the runtime plus all the development headers so you are building in case you're building an application in the XDGUP system, you're building it explicitly for a specific runtime with a specific SDK. So, if you are an application developer, you decide I want to target the GNOME 3.18 platform so I use the 3.18 SDK and target the 3.18 runtime with it. So, this also means that the user needs to have all the runtimes around which are required for to run the application. The application itself is also an artistry repository depending on a specific runtime and they are mounted together in their own, yeah, in their own file system again just like LIMBA, similar to LIMBA does but without dependency and overlay effects. Yeah, another speciality of XDGUP is the sandboxing feature which is in very early development but you can still test it with some games right now. The main reason why we want the sandbox is this. We want to protect our users from malicious third party applications which could access their data, start their webcam, invade their privacy and yeah, do all those bad things. So, we want to protect the user against it by adding a permission system and by adding a sandbox. So, this is something XDGUP implements and there's some discussion going on if it should be mandatory for XDGUP applications or if applications should be able to run without sandbox but so far at least as far as I know there's no decision been made yet. Maybe it will be a policy of the desktop environment or distribution. So, if you want to get an XDGUP application in case you're a developer you get this large runtime or SDK and those things are, as I said, OS tree repositories built via the Yocto project. Yocto project is an effort by the Linux Foundation to automatically build Linux distributions for embedded systems and it fits pretty well with the use case of building these larger runtimes and SDKs. So GNOME already ships one and for one for KDE is in progress right now but I'm not sure when it will be ready because there are a few issues with it. So, and the applications then, as I said, are built against, we're using a specific SDK against a specific runtime. The strengths of the XDGUP are just like Limba, the conflict free installation of software, atomic upgrades, whoops, but yeah, sorry, but also the ability to roll back the installations at any point in time using the OS tree repositories. So, just like with Git, you can jump back and forth in the history of the installations of the software by switching OS tree hats. You also have a stable and defined runtime in SDK which means that the risk of ABI breaks is almost zero since the upstream, as long as you trust GNOME to not break the ABI or to not break the runtime ABI but this is something which is checked by the larger entities and there are not too many parties evolved in creating the runtime. So, the risk of something, something, yeah, something problematic happen in ABI-wise is pretty, yeah, pretty small. It also has those sandboxing features already built in although they are not very usable and we are currently waiting for Wayland, KDBus and Toolkit integration to make it actually work. So, this might take a while until we have working sandbox because there are also a few other issues which could easily fill another talk. So, okay, so the good and the bad things about those solutions, now, Simba has this dynamic runtimes and a high amount of flexibility but at the cost of having software authors which need to care about ABI stability which is not sure if it, there's no certainty that this will work but I have faith in our upstreams that they can care about ABI stability which will also benefit us as distribution. So, this requires a bit more testing. Well, actually, software authors can break ABI but then they need to declare that they broke it. But yeah, this definitely requires more testing and there's always a risk of breaking things. But on the other hand, their software is shared between their high amount of shared software and the updates can reach a user really quickly. So, in case a library vendor gets a new security update out it will reach the users almost immediately because we do not need to integrate it into a runtime or to update all the applications depending on it, so we can just ship it unless it breaks ABI. So, Exigia has the static runtimes with, well, includes this ABI stability guarantee and a very, very low risk of breaking things but it also means that in case there's something not present in the runtime the application authors need to statically link against it or bundle it with this. So, in case there's a security issue in the library which was not in the runtime then we need to get every upstream to rebuild their software with this new fixed library which might take some time and there's some duplication in case you have multiple runtimes available and potential security risks in case you do not have the sandbox solving or working around some of them. So, what does this actually mean for Damian or Linux distributions in general? First, going back to AppStream. AppStream is really nothing you need to think about a lot because it's no change in any paradigm and it just enhances the usefulness of the repositories which we already have and offers some nice new features like being able for third-party developers or AppStream developers to interface with the great thing we have which is the software repositories. So, this is I think really not controversial. The limba and x2gup stuff which is really something which is important to me is not something which will replace the distributions or the PPAs because we will still need PPAs to switch system components or to update system components in case someone wants a new system, the PPA is the way to go because this is not always components are not something you will ever be able to ship with limba or x2gup because it's really stripped down. So, those solutions complement what we already have. They complement the repositories and they complement the PPAs. But another thing what we should maybe do is to focus on the OS part of the distribution and not that much on shipping applications. Stripping applications are still important because maybe some enterprise vendors are already happy with what's in Damian and then it makes sense to ship the applications and the repository to make sure that it's trusted and tested on Damian but the main priority of us as distributors should be the OS and not shipping each and every software component on the planet in our repositories. And also important in case the limba and x2gup things which are still experimental to take off is that we need to get a say in what's included in software center applications create a valid trust path from our distribution to those services and maybe endorse one which for example is offering only free software and say, yeah, this is something which you can use with Damian and which integrates well and ensure that it works well with our distribution. So users do not, which are attempting to use it, can use it, it's still an option, nothing which will be mandatory or well, some bit preferred way to get stuff but we should also not reject this new developments because they will happen since users are very excited to have this new opportunity. So in case you want to read more, there are a few links like Leonard Pertring's blog post and LWN articles on limba and x2gup and yeah, the project pages and also the upstream.debian.org to see if your package contains metadata which has already been included in the main upstream data. So that's it with the talk and do you have questions? What makes you confident that desktop file, what makes you confident that desktop file names are a suitable way of universally identifying package names? I'm thinking of things like epiphany.desktop or there has been some package called jit in the Debian repositories for some time and that's only within Debian. If there's something which is only in Debian then you will only have it in Debian of course but yeah, the desktop file name is unique among distributions because they are, well, no, if you have a desktop file and you have a file conflict between desktop files then the upstream project has already solved it. There are no desktop files with the same name but having different projects and it will be solved even more, well, it will be even more unique in future by switching to the new Debus activatable applications and using the reverse URL sheen for desktop files where I now have org.kde.kate.desktop. So with that you can safely say it's a unique name and universal name for an application. Would that be it? Two more questions? So in all this brand new world, how many copies of, for example, the Qt libraries are never used or going to have installed or how many copies of GTK? That depends on what your application depends on and the exeggup model you will have, well, have as many run times installed as applications are needing them. So if you have three applications, each of them needing a different run time, then you will have three copies of everything. On the limba approach, if you have three things depending on where three different versions of Qt, but the other stuff is the same, then you will have three copies of Qt but the other stuff will still be shared. But yeah, the repository services are mainly there also to have applied some force to upstreams to drop outdated dependencies faster and update their stuff. Because if we are in control of the application repository service, we can still throw out stuff which is not updating its dependencies or which is having security issues. So the one who's controlling the repository service is the one who can dictate upstreams what's to do. So I hope this will mitigate it, but yeah, the issue is there that you will have to duplicate itself. Any final question? I have a question about dynamic linking because in limba, for example, it seems that if you let upstream do it all, it's kind of scary that they will just, I don't know, just go by without dynamic linking and making bundles and not caring about security at all. Maybe they have not even, they don't have the knowledge to do it right. So how is it gonna be that mitigated? The idea is to do lots of rigorous checking of the resulting package. So you will currently already, the limba package generator or the limba bundle generator, is applying some checks for sanity of the upstream things and will reject to create the bundle in case there's something wrong. So the upstream will just not be able to create something invalid or something stupid. And by applying more of those technical things to and lint tools in order to get it sane, I think we can work around this problem. And also on the software store side and on the repository service side, we can also do additional QA to get upstreams in the right direction and teach them how to do it properly. So this is how it should be mitigated. Okay, thank you very much.