 Enrico Zini, who wants to talk about how to build your own modern app installer. Thank you. Hi, I'm Enrico Zini. As some of you know, I have something to do with DevTex. And generally I've been working trying to enhance package information and the user experience of looking for packages and so on. I've always had this idea that Devin is huge, right? There's loads of packages, which is a problem because it's hard to find things. So some people started to say Debian shouldn't grow too much, because otherwise it becomes a mess. And I started to say, let's find ways to handle the mess. And so DevTex came out and a few other things that I'm going to show you. If you want, I put the notes of the talk in Gobi. I hope it's big enough. I've improvised this slide. So if you want to, during the talk, I'll try to give as many ideas as possible and point to technologies and so on. So if you want to enhance the notes or just follow them or read them in five minutes and leave and find another talk, they're there. And use them as you please. And feel free to enhance them. Sorry, I hit it by mistake. So what is in an ideal application installer? Voices. I guess you can say it without the microphone now, repeated. Brainstorming session. What do you see in an ideal application installer, like package managers or the thing? It must install what I need. It must know what I need, right? That'd be good. We'll get there. Soon, within the last couple of hours. More. So it should give a visual, I know I repeat it because if you run around with microphone or Brainstorm, it's a mess. It should give a visual like screenshots or preview of the application because sometimes, like in case of games, the description doesn't help that much. Indeed. It should keep track of installed version and. Okay. So it should let me know of important changes in my packages and allow me to keep my system up to date, basically, with the most critical. So I am aware of the most critical things about the system. Right. If I search for burned DVD, it should just find the right application. Exactly. Okay. It's a few ideas. Yeah, you are imaginative, not ambitious enough. So I had a few more. So it should, you know, search, browse, explore, install, remove the usual thing. It should have all sorts of information, description, tags, screenshots. Even more information than that. We start to be used to social information like rating comments and package recommendations. There could be different installers for different needs. Not necessarily if you want to, I mean, there's a kind of user that would like just to install graphical application and there's like the software developer that would like to look for libraries available for a specific programming language, not necessarily the same interface will work for both. You don't have screenshots of libraries, although you could show like URLs to APIs. And at the same time, you don't want to clutter something about graphical information with information about development libraries. So it's very exciting. You know, when I said lots more, we're going to see some more later of information you can have. It is exciting because it's very easy to access unthinkable amounts of cool information. And the only thing that we need is interfaces. We have a lot of back-end code that is ready and usable. And it's having interfaces, having all these installers that doesn't have much work yet. Possibly because all the work that's been done so far is by plumbers like me, you know, like the Git world has been talking a lot in terms of plumbing and porcelain and where the plumbing is like the pipes and the porcelain is the user interface. And lots of people have done plumbing, experimenting with ways to index and deal with all sorts of information. But these people are not the best people to build a proper user interface. So we need the second step. We need to bring in people who haven't done the plumbing, but they can build on it and build something cool and useful. So some example mock-ups of random ideas that I think are slightly missing and that can be very easy to build nowadays. So you can have something inspired by a search engine with a search box and you type whatever you want and you get a list of what you need. And you can even do something like a preview that if you don't type anything, the software will suggest something that you may need. This can be done. Hopefully that will last us about the second step, but that will come, you know, wait an hour, she'll tell you about it. Or you can do a similar interface for uninstalling packages, your systems full of rubbish, or you fill up the disk, the USR or something, and you'd like to get rid of packages and you can do the same look for search only among the installed packages. And if you don't type anything, it will show those packages that you are not using, you know, the first package you see is the one that you use more time, the most time ago. That is also very easy to do. Or you can make something like two cows, Android app installers, something like this with, you know, screenshots, command, searchy thingy, ratings, maybe with a swirl, cows would work very nicely in Debian, but they've been used already. So that is easy. Or you can make something like online shopping sites, kind of Amazon things with the new type of something and you get categories on the left, I guess you're used to this kind of interface and screenshots as a preview and descriptions if you want. Yeah, this is a mock up that already works and I published a code on my blog and not many people noticed it, but that's to say that can be done. Just clean it, you know, you just need to show tag descriptions instead of ugly tag names and maybe, you know, put the proper description here and you have it. You just need somebody to do it and deploy it somewhere. Or you can have a browser for the packages that are already installed and which will, you know, show what's in your system and you can sort by show first that the last packages have installed. So you can like install things and then have a look. What did I install? Let's try them. Or show first the packages you use the most and you can have links to man page documentation in user share doc, something to launch all the binaries provided by and even maybe something to contribute rating and comments to Debian. So it can be shown in application installers. That'd be nice to have, right? We don't have it. We don't have anything to browse the power of an installed Debian system except like for the normal KDE menu which is just a huge menu of things. So why not? In order to do all that, we have several building blocks we can use and I'm going to show them like Appetix App in index which is this backend code I've done for all sort of cool and smart package for handling all sort of package information and very fast. I forgot to do this first. I apologize. No. Black on white. Okay. So Appetix App in index is likely to be already in your system because it's used by software center a few other things. So you may already have ACSI cache. Okay. The font is big. It doesn't show much. But for example, you can do, if you need to look for like something to program in C, you can do Appetix cache search C and it's a bad idea. However, you can do ACSI cache search C and you likely get... It knows that it's about C. You know, you get C, GCC and so on. Same about GNU R and other similar things that we have that have one letter names. So this one indexes words and interestingly it will give you suggestions for other terms you may want to look for or tags like DevTags tags that are related to what you are looking for and it can even give you spelling suggestions like did you mean G edit? It's there in your system today. It's even the first command line application that allows you to find something before you run it. So if you do Appcache search tab, you get a list of tags you may be interested in. Say I want a program. So program and then say what for? Let's say for image. So you get basically context sensitive tab completion and by the time you finish the command line, you know what you want and at the beginning you didn't. And there you go, programs to do something with images and how about raster images or utilities and so on and you get more. So that is Apptix Appian index. In my blog it's full of posts that teach you how to use it from code. The examples I posted in my blog are in Python but you access it using the Xappian libraries. Xappian is a search engine library and Xappian libraries I have binding for any language like C and Lua. So people still complain there's no bindings for Lua. I agree it would be nice to have them but maybe somebody's doing it but they get bindings for Per, Python, Ruby, PHP, if you want to do it online. And it's natively C++. And that's the first. Screenshots.bebian.net, you know about it. Who's never heard of Screenshots.bebian.net? Who's heard of Screenshots.bebian.net? Okay, great. Who has uploaded to Screenshots.bebian.net? Great. Cool thing of Screenshots.bebian.net, if you have HTTP and a package name you have Screenshots. It's, you know, any web page you make that displays packages you can just put an image source equal Screenshots.bebian.net slash something slash package and it becomes a screenshot. So there's no excuse not to have Screenshots in anything online because it's just, it's that easy. Where is it? Here. No. What? Okay, browse and somewhere there should be documentation of the API although it doesn't fit at the moment on the screen about maybe. There you go. How we can use the Screenshots. And there you have it. As easy as that. Screenshot.bebian.net slash package thumbnail screenshot and so on. With this you can make a link to a page with the information about the package. With this you get a small screenshot. With this you get a big screenshot. Any web page takes nothing. It's there. Up install data. It's a package that contains desktop files and icons for every package in Debian. So I do not have a W3AF installed in my system but with up install data I have the .desktop file and the icon which means if I want to make an application installer for every search result that has an icon I can show it. And if it has a desktop file I can show in the search results I can show the menu location where it's going to be installed and free desktop description and so on because it's all there. And it's also interesting because besides the normal desktop file information they have extra bits which is the package you need to install to get it and popcorn rating and this section. So that allows you to sort all applications in your results by popularity and you can only just search the desktop file information and you know you get the correspondence to packages. Software centered as that. Oops. What's that? Why do I press B and it becomes black? B, black? Right. Distorm match. I don't think many people know about it. Who knows about it? Oh wow. Not bad. Usually it's only me knowing what I do. Great. Distorm match is a new thing. It allows you to say oh there's this package in Fedora. I can't find it in Debian. Is it in Debian and what's its name? I'll give a demo. Again it's still no. Okay. So let's say Fedora, RPM in Fedora can correspond to those packages in other distributions. It knows about a lot of distributions. Basically all the RPM ecosystem and all the Debian ecosystem. And that's nice because now with RPM it's a bit, apparently RPM in Fedora has been split in sub libraries elsewhere and so you get all the results. But let's see who's got the package. Because Debian has RPM. But do anybody else have the package? It wasn't so slow. Oh yes. Interesting. Okay. So with this you can do cross-pollination across distributions. The idea is Debian has screenshots while the others don't. Then let's use this so that the other people can get screenshots. Or Debian has tags and the others don't. With this I can export depth tags for other distributions. And then I can contribute as well and then I can re-import tags from them. It is exciting. It's kind of just being built but I'll talk more about the project where this came from because it is very interesting. And that allows you to build web cross-distribution web services. If you build any website where people do rating of software well you can use that to export all the ratings you have to whatever distribution or to get contributions from whatever distribution. There's also OCS. I know it exists but I never used it or tried much about it but it's an arrestful protocol for ratings, comments and social features like that. And the idea is that you can have any sort of package manager talk OCS with some server that the distribution provides and that can be used for submitting ratings or downloading ratings about packages. The protocol is there and it allows more than ratings. Like friends, like Facebook inside aptitude. I'm not quite sure but I think they use it. They were wanting to use it for Maemo. That's all the documentation of the API. There was some demo about it. KDE people may know more than I do about it. There's some person that knows more about it. Is there someone in the audience that knows more than I do about it? We'll go and find the KDE people later. But that seems to be the way to go for the sort of social interaction, socially contributed package information, crowdsourcing and so on. Going back to our example mockups. So the example of the search engine kind of application, you can use aptX Appian index for the search feature and you can use a recommender that you don't know about yet but just wait half an hour to provide package recommendations. And of course, screenshots for screenshots. All the building blocks are there. You just need to put them together. Same for the one about uninstalling packages. Again, aptX Appian index to search for them. And if you look in Virolog popularity contest, if you have popularity contest installed, you have the last batch of information collected by popularity contest which basically lists every package you have installed and when's the last time you used it. And you can use that as a data file for yourself to sort packages by last used time. You need to have the file system mounted a time or rel a time if you mount slash usr no a time that won't work. But if you use rel a time, that allows you to say show me the packages I don't use, I'll get rid of them. It's there. In terms of the sort of fancy app market thing, again, aptX Appian index allows search packages, you get screenshots. App install data will give you icons. Those icons are just taken from app install data. And you can have a look at OCS and see if it works to provide ratings and comments. I think Ubuntu is a bit ahead in looking about it. I don't know if they built anything, but you can get in touch with Michael Vogt and he may be able to tell you something more or the KDE people upstairs. And then this one, you just need to take it over from me and make it beautiful. And it uses aptX Appian index for searching packages. aptX Appian index contains step tags and can already give you tag suggestions and all sort of information. And so this comes out of Xappian basically for free. And again screenshots, that's a web app I made and yeah, sure, take it, use it. And again, the browser for what is installed, it's again aptX Appian index for search. Am I liking aptX Appian index a lot? But it is really the thing for this kind of quick search. We need to implement last installed information in aptX Appian index. The information about when a package was last installed is not there yet, but we know how to implement it. There is already a hook in the package and anybody that's interested talks with me and I'll show them how to add a plug into aptX Appian index, how to hook into the package and how to put the two things together. I haven't had the time if somebody else would like to do it, yay. And again OCS for sending ratings and comments. And if you want to implement list, man pages, documentation and so on, you just have a look at the package info and it will tell you the list of files per package and you just look for the interesting ones and you show them in a nice way. I talked about this too much and cross-distribution collaboration. That came out of the upstream project which was a meeting that we had in January in Nürnberg at the OpenSUSE Novell headquarters in Germany and that's a bunch of people. From basically most major distributions I think some couldn't come but they have been invited and it was unfortunate they couldn't come but we had Debian and Red Hat Fedora ecosystems were represented, larger ecosystems so we had Debian and Lubuntu and we had Fedora, Red Hat, OpenSUSE and Magyam which is the new name for Mandriva and some other things that kind of lost track. We're there, unfortunately the GEN2 NR cleanups couldn't come which was a shame and the idea was to write some software together to do a kind of modern application installer and what we came out was after a couple of days we came out with a list of technologies that we kind of agreed to use and that is hopefully software center and hopefully because the Ubuntu copyright assignment policy is severely screwing up this process because an employee of Novel or Red Hat cannot justify that they contribute code to not too free software but to canonical in their pay time so that complicated things a lot and unfortunately stalled the project I hope that can be resolved eventually but I'm not sure what will happen so far that's an example of copyright assignments impacting cooperation in the free software world but something like hopefully having software center on the client side possibly package kit would be a nice thing to have once you get fixed a bit and the Exapian database the AptiExapian index basically to have the back end for advanced search functionality and to have some extra data in the mirrors like icons, package, popularities and so on that can be indexed by Exapian the AptiExapian index does not only index the usual packages information it can do all sort of numbers again like popularity it can index extra information that's not in the packages file so it's the ideal place to put all the extra kind of information hopefully then use OCS for rating and comments and well that's like when what I call it repo compose I think someone from the Debian ecosystem wrote this but basically when you build the archive you extract the extra information that you can download into the application installer so that kind of outlines technologies that we all decided to use there's no point in building another back end index for package information AptiExapian index is there distributions are now aware of it and the same about OCS there's no point building a new protocol for getting package ratings there is one which is a simple restful protocol let's use it and the wiki of upstream there should be the link in my notes in Gobi has all sort of information about kind of the decisions that have been taken at the meeting and that is exciting it was exciting it just needs again people to put energy into it to me the important thing that this means is that if you start to do any work in cool package interfaces you know what technology you should use you don't have to go around and spend a day looking for as anyone's done this that's all the starting points that you need for your work and you just start there and people know about it other people will be happy to build on your work so it's there use it, make it awesome that's the idea there is no excuse there is no excuse not to have very cool modern anything about packages there's no point in being jealous of what the iPhone and Android have it's easy to implement we just need to do it we just need to spend a couple of afternoons doing a prototype and decide to keep maintaining it basically my blog, my notes me get in touch if you like any directions it's all there any questions? how do you do this distro match thing how can you reflect these files and how this works I'm glad you asked I basically the input to the system is a list of source to binary package mapping and a list of files in each package for each distribution and then the idea of distro match is to implement one glue for all sorts of methods it just runs them one after the other so there's no particularly bleeding edge research on computer machine learning behind it the point is to have one place where if anyone does it can be plugged but at the moment it looks for dot desktop files as sort of markers because they usually come from upstream so if you have two packages that contain the same desktop file in the same path they're likely the same and that's kind of considered an exact match obviously if two packages have the same name then yeah and then it looks at the similarity sorry it looks at key packages, key files contained like if two packages have the same .pm per module or .py python module then they're obviously the same development and if they have the same man pages the same binaries and user bin and if all of that fails it will look at general similarity between list of files in the packages so it kind of keeps going and going and the interesting thing is that it's done in a way that it builds one index per distribution and you can go from A to B by sort of getting info out of one and feeding it to the other to get it out so you basically parse the .rpm and .dev packages and make this index the index is built by taking on the .dev side I query the information out of UDD and that gets you the whole .dev ecosystem so derivative distributions yes no problem so we can have one to different versions it's all there the .rpm world has something similar called SOFIE which allows you to find search packages by contents and they've been kind enough to give me a login on the SOFIE server so I could build an expert procedure for all the .rpm world information which is vast and get it out and the index it that was a fantastic example of cross-deaster cooperation there's a machine of the Magia people actually they're running SOFIE that every night contacts the Debian machine via SSH and sends data over that's great I have a similar but probably more easy questions is up to install data where you get all the desktop files it is generated automatically I do not know the details I don't know if it's dark doing it anybody knows manual right okay I think it's manual someone needs to upload it process all the devs and then upload the result okay sorry then it's not automatic if somebody searches for let's say Excel would it be possible to print out alternatives to that? I'm glad you asked it was perfectly well we also need to say that you want a program instead of there's no LibreOffice because nobody tagged it yet but basic there's numeric but we need to know well LibreOffice is tagged by having approved all the files in the distribution yet I think because I run I'm a bit behind with that but basically the way this works in APT XAPIEN index we have Excel as an alias for the depth tags for spreadsheet and we can do the same we've done the same for Word and so on where is it and yeah XAPIEN supports aliases and synonyms we can just feed that list and we can feed more so I noticed in your screenshots you seem to be using package names instead of app names which are quite different no in the screenshots before package names so I think users will probably be more interested in app names rather than as they appear in the menu rather than package names because that's kind of a technical detail that I necessarily need to know about true but on the other hand we are talking about unless they have something about app names and I don't know but we are talking about something that you use that you use in a package manager and in a package manager you work with packages and you can actually map from packages to apps and but then in that mapping you can do the inverse mapping and get screenshots oh I forgot talking about custom interfaces for different users oh I have a new laptop and I don't have it installed 65 okay there's one that's called GoPlay which will be installed in three seconds two one which does that GoPlay, look for game so you can see games already installed or are not installed or both and you can choose the kind of game and you can choose the interface and there you go a list of three dimensional arcade games with screenshots and descriptions and tags and so on it does everything except install it but it's an interesting example about sort of per task interfaces that is way more simple than any than any other package manager and it's only for games and you want to say like Doom and it's very quick and you get them and then the categories automatically change interactive Doom oh yeah it's probably mentioned Doom in the description and interestingly just by changing the depth tags that are used there's GoAdmin with well system administration so clustering, system boot benchmarking and so on and interface and fast search and list of packages but if I filter for three dimensional sysadmin packages maybe and we do have 3D sysadmin packages there's GoSafe for security things antivirus authentication forensics three-dimensionals, oh ok same thing again more example of things that is not hard to do that also guess what uses Crap in index and in terms of mapping applications to packages ooh this is an application installer a prototype application installer we built and you just type I don't know image editor list of image editors that uses guess what indeed it looks for packages but shows apps so only those that have desktop files and it just tells Crap in index only give me those packages that contain desktop files and there you have it and so you get fast search on everything using the package descriptions are more verbose than the description in the desktop files on the other hand it merges desktop file descriptions into the keywords so it actually searches on both it was an example that's quite nice actually I need to show you this other feature which is very intriguing although can I have home and ok it's a photo it's an image and drag it here I get packages that can open image and then I can just open it in a surf it's it took what two hours to do the sort of drag and show icon thing it really is now it's all easy to do it just needs ideas and throw some code at it do it I don't I'm busy with the plumbing I would like to do it I don't have the time so I want people to do it do it going back to the xapian index I was also playing at the same time as you and I saw the example you did that it was showing I don't know what we were looking for but it showed first the libraries and you were looking for Excel so it's that on purpose it's a bug or what because I'm thinking that as a user I would not like to see the libraries first 100% as a user as a user you do not want AXI cache you'd expect if you want programs to have like a GUI or something that can show you screenshots but if you want if you want you just say that you want a whole program search and then no libraries in any case it was a little weird to see that the library was more relevant at numeric or for the Excel example relevant well because of the description really looked like screenshotty now I added a whole program to it and excellent bifurcation right? ok we can work on that it's an excellent game but well yes matches the package name exactly so maybe inside it just says Excel and so but then in an ideal interface you get this and you say what the hell is excellent bifurcation? I want spreadsheet and you just filter I want spreadsheet and you would click on it and you can build a tag cloud that's a tag cloud and you know there for free there's an example called my blog in python as you type it's fast enough that you type and you get a tag cloud out hey do you have an example of web interface using these things? web interface? yes package slides this prototype I made I don't dare starting it because it probably would blow up since I haven't started it in the last six months and you know code will unwrite itself as you don't use it but this works again you can search I implemented filters like only desktop applications only programming libraries and so on that you can choose in there and by default it will only show you desktop applications and then you can shop by tags on my blog commented with descriptions and everything so I think we need to finish thank you Enrico very much for this inspiring talk and I hope you got somebody who throws code on your ideas I hope so too thank you