 I'm Franck and Piat, I've been involved in Debian for a few years, I'm maintaining some I'm Franck and Piat, I've been involved in Debian for a few years, I am very interested in how we can improve the support of our users, either through documentations, tools, other things. One of the areas where we can improve the current situation, in my opinion, was... Can you hear me now? Does the audio team feel like... This way? Thank you. So I think we can improve the way the experience for our user when they have to upgrade their distribution. I'm currently here going to present a project I've designed. It was basically ready for I haven't tried to get it in silence, but it's probably going to be in squeeze. I expect this talk to get some feedback from you, so if you have any questions, comments, don't hesitate to interrupt me or I think we should have plenty of time to discuss at the end anyway. So first of all, from a user point of view, the first thing the user is doing is to install a system. We have good tools for that. Debian installer, not to name it. We have the bootstrap or many solutions, which are very effective. Of course, when Debian release a new version, a new stable version, the user wants to get a new version. Debian has an infrastructure to perform this upgrade, namely DPKG and IPT suite of tools. And those use post-installation, pre-installation and so on scripts to perform the migration from the current package to the next package. Also, we also already provide inside each package some documentation for the changes introduced in the new package, as you all know, namely the news.wm file. It's up to the maintainer to provide all the useful information for the user. Also, during the actual migration of the package, we have the possibility to prompt the user about questions regarding those upgrades. Finally, we have the release notes, which has information typically on the problem that the user could encounter when he is upgrading the system. And overall, we have, I think, something which is already very good to help the user migrate their systems. So, what's the problem? Do we have a problem? The first thing is that we are a distribution. Debian provides software, which is for most of them not both inside Debian but outside Debian. And anyway, like every software, there are some new features, some features that are discontinued. Unfortunately, we try to avoid them. Regression, which usually are fixed before the release. What kind of other problem can the user face when he is upgrading the system? It's something like local configuration, which basically breaks what the upgrade scripts expect. Also, user who have made some choice to really customize the configuration of its system. Also, some package introduced some new behaviors. That happens. And also, Debian can choose to use a different program in different release. So, that's the kind of things we can change. And that has to be handled during the upgrades. And there's one thing which could be considered as a problem is I mentioned before that we provide news.debian file to provide information about what's going to change in the package. But obviously, since this news file is contained inside the new package, you can only read it more or less when you have started to migrate. So, what's needed? What can we do to improve the situation? I think one aspect what really interests the user is basically to have some predictability to be able to plan and prepare his upgrade. I mean, the user is clever. He has read the documentation we provide, especially the release notes. But we can help him for them. From the user's perspective, upgrading a system basically is three steps. The first step is he should get the documentation, release notes specifically, and make sure here is all of it. It's reading all of it. In large company, he's probably going to make some, to have some computers to test the upgrades and make sure he can upgrade the system without breaking the production. This is also part of planning and migration. So, the main step for upgrading a system is obviously to actually upgrade the system. As I said before, we have some tools for that. You change the source list, update and all this one. And finally, after an upgrade, since there have been some change between two stable distributions, maybe Debian says, okay, we don't use syslog anymore. We use our syslog anymore. But, of course, we want to be conservative when I upgrade a system. So, we are not going to force the user to change the system. There's a syslog, for example, during the upgrade. Those kind of changes are not performed during the immigration, the upgrade. However, they are documented in the release notes. So, we can expect the user to read the release notes. Also, some packages are not used anymore and could be removed. Things are maybe not always removed. So, you know, it's all those kind of small things that we can help the user feeling more comfortable that after he has upgraded the system, he has something very similar to a freshly installed system. So, I've designed a tool, which is actually a very simple one, which focuses on the first, on the two steps I've mentioned above. One is what we do before the upgrade and to warn the user of what he's going, what we'll have to take care of. And the other part is, after upgrading your system, what should be done. So, we can have a look. We can actually, yes, we'll try this one. So, here I'm running from a Git repository. Of course, most users will just use, you know, from the package installed. And basically, it's going to perform a few tests. And here we are in Preb, I'm going to, hold on. So, invoking the program just goes like this. And we can see he has performed values test for the system. The idea is to help and provide information about the status. Actually, the system here is not very, very interesting. But basically, it's a modular tool where you can just have plugins to test anything which is interesting for the user. For example, I'm currently working on a tool to detect a package that are no longer available in the next release. This is kind of tricky, actually. Also, we have to make sure that the user don't have some settings in APT that would prevent the upgrade from working properly, especially doing APT pinning. We can do a check for our fan system. We can, we make sure the user doesn't currently have some package that are on hold. We also make some tests to make sure all the package are currently installed. We don't have pinning installation and stuff like that. And it's also performing a few tests. This is, those tests at the bottom are really those which are documented in the release nodes. And we duplicate, in some way, we duplicate the work. The documentation of the tool clearly states that the user should not expect to have some tests which are 100% accurate. We have some false positives. We have some false negatives tests. But that's probably a good start to help. Here I've run the script in what I've called human mode. It says another mode which is intended to be easily run on a large set of machine and to be able to collect those information and just to grip the files. So each file says, you know, the usual logs and stuff. And on this test machine I don't have any very broken things. So we don't have big arrows. But here it just says, okay, we have a file here which contains some stuff which are not standard. And this is because I'm getting some files from the hosting company I'm using for this test machine. Right, yes. Actually what we, what I have done so far is that when something is, the question was Lin-Chan currently provides some help about the error messages. And what I've done is that for each test I recommend to have a link to the release notes where it's documented. But actually it's true that not all the error, all the test comes from the release notes. So we need to do something about it. That's a good idea. Well, you're going to have to, this is going to be a difficult job to synchronize this with the release, right? Because you basically want to have the release notes done and then there is the last upload of upgrade advisor. And then we release, ideally. Maybe it's possible to actually have the release notes in the package or have that data in the package so you don't have to have links. You know, we have that island test thing. When I'm going to upload it, maybe we will actually merge it inside the release notes. I should have talked with the release notes maintainer. Cool, yeah. Probably. Yeah, so executable release notes were, sorry, executable release notes were, was one of the ways that I tried to describe things when we were doing update manager in Ubuntu, which is kind of a similar idea. And one thing that I don't feel we did well in Ubuntu that we should learn from here is that we didn't do well at decentralizing things. So the, all of the update, all of the upgrade logic, all of the special cases that you have to deal with are all centralized in update manager. And the result of this is that the update manager maintainer is a hell of a time sorting through all of the bug reports and has to keep up with the entire distribution and all the weird shit that people do. It does seem like an extremely good idea not to duplicate that and to try to design something that would allow packages, that would allow individual packages to drop in something saying, when you are upgrading from Lenny to Squeeze, we need to do this weird thing that's not, you know, that's not expressible within the standard package system. Maybe we need to prompt the user in an unusual way, maybe we need to make sure that some other package gets installed and that's something that could be, that could be evaluated before the package is upgraded by inspecting the new things that have been downloaded. It looks like everything's centralized in upgrade advisor at the moment, but if that's not the case, that would be great. It's designed in a modular way, where actually each plug-in is, I'm not completely happy with the way it's designed, but it's hosted currently on CollabMaint, and it's pluggable so anyone can just come, I will see you later, just any maintainer could drop a plug-in inside the directory or, I'm not quite sure how it should be done today, either the maintainer contribute to plug-in inside CollabMaint or if the maintainer of the package should include a test inside this package. The problem of putting the test inside the package is that it means the user has to install the latest point release before... Yeah, that's definitely a problem, although not necessarily, you could, maybe this isn't the right place where the tests are designed, but you could, if the tests were written in such a way that they didn't depend on the contents of the package, you could extract plug-ins from the new version of the package that you downloaded before you actually tried to install it. If upgrade advisor were in control of the whole upgrade, then that's something it could do. It does mean that everybody would have to use it or they wouldn't get the benefit, but that seems implicit. I would be very happy to have some feedback from other developers of what's considered to be the best way to contribute this kind of problem. I'll surely post on WNDevol to make sure to have good feedback on this. That's probably a many scenario. I'm not going through post upgrade, but basically it's the same kind of, you invoke it the same way and it just makes sure that all the standard package are installed and then we can also contribute package. So if let's say we have decided not to install Apache 1.0 by default, but Apache version 2 by default, we could just tell the user, okay, you are running Apache 1.0 and just let, we want to let you know that the default now is Apache 2.0. It's really the same idea. In the next page, I'll use this convention. Basically, I've written the version N of the distribution is a current install. N minus one is considering the user has already performed on a grade. So let's imagine he had already performed from H to Lennich and then his goal is to get to N plus one, which is squeeze. Okay, so how do the prayer upgrade actually, I should have inserted an extra slide for people who want to understand how it works and who wants to contribute a plugin. For the prayer grade, we are performing three sets of tests. The first test is after upgrading to a release, as I said, we are suggesting you as a user to perform some, the release notes suggest recommends to perform some modification to the current system to get ready for the next release. I remember one which was to modify the configuration in Grubber, for instance. The other step we perform is just to make sure the current system is in the same state. Do we have broken package? Do we have some orphan package? Do we have some weird list of packages which are used and which could conflict? Maybe some tests about the bootloader. If we know if some bootloader needs some specific attention. And also basically we want to make sure that we would be able to modify the kernel for the next release. For example, if you're running a hypervazor, you probably can't change the kernel you're going to boot in the next boot. That's something we can warn the user about. The third part of this test is, as we mentioned, two tests for non-issues that are usually described in the release notes. Discontinued packages. All those who are becoming non-free because this can be an issue for some users which don't use non-free. In the long term, I would like to be able to warn the user about unsupported hardware in the next release. For those we can guess based on PCIIDs and stuff like that. Also, we want to upgrade the user about those packages which will require substantial reconfiguration or some migration steps during the upgrade. Maybe if you run a wiki or if you run a CMS, maybe the two upgrades are not going to hundle all the cases. Or maybe it just says that you have to perform some tasks manually. One of the user again. Is it clear about what the pre-upgrade is performing? What kind of test? I guess so. Okay, the post-upgrade mode. Again, basically we test the status of the system. Maybe the user has the impression that the upgrade went through, but actually there are still some packages with the old version. Those kinds of hints we should tell the user. Also, if the user hasn't rebooted yet, we can warn him and say you won't be able to boot from the new kernel. The other part of the post-upgrade part, upgrade action, is again to mention what is described in the release notes. Basically, there's nothing really new. It's just trying to get a tool which can get the attention of the user of important points. We were talking about writing script. I think it's probably the maintainer of a package which is the most capable of writing a test script. Because he knows exactly what has to be tested. And also, one problem is to know when, in which circumstance or in which release each test has to be run. For many tests which are version-specific, I've ended up using something which is very deduced to maintain, but it's to actually detect the current version and warn the user based on the version of Debian he's running. I'm not doing the test based on the version of the package he's using. This is because it's much easier to run this test. But maybe the maintainer of the package may sometimes know that it's actually this version of the package which triggers the problem. So a maintainer would be capable of saying if the user is using a version of FUBA less than 1.3.5, we can warn him because we're sure that the next version is going to be a source of problem. Yes, Martin? It's sourced. Yeah, it's sourced. I can't remember where I ended up putting BNSH, probably just for LinkedIn or something like that. I think it doesn't make sense anymore. Yeah, because the package is sourced then executed. So the benefit is that we can initialize some stuff when the package is sourced and actually run the test later. Something like that, yeah. No, just no. Here's my mouse. The main function of the test must be named after the name of the file. So usually a typical script is like first of all we test because if something is really known to be a test is legacy on a given distribution we can just quit here if it's something before any we don't have to worry about it. Then I've used many macros so we can precisely have different kind of output. So the next line should be title, and it's also used for internalization of the script. Then it's the actual test that should be made and that's really up to the maintainer to decide what he considers to be sensible for testing. Here we are testing for H2Lini distribution we had to make sure that the path of update was correct or something like that. And again, if there's a... I don't know how I can point all that. So we use gecko for internationalization and at the end of the line we use a pipe with either alert, extra or whatever for to be able to output with machine... in machine format. Inside the readme file I may be not going to go through all them but inside the readme file there is some guidelines of how to write a script. I don't know if you want to see some more tests but to perform some tests about whether the user is pinning some package I'm just detecting the presence of the file and warning if the file is here it's a very conservative way of testing. I didn't try to get inside the preference to detect if it's had pinning anything. What was in this one? For some distribution we had to test the kernel and tell the users that the kernel 2.4 is not supported anymore so this is a quick test to perform it. Because some tests have we've seen maybe all the tests regarding the status of the distribution actually run in both case if we are running a pre-upgrade or post-upgrade source of test anything with an B is actually run in both case for pre-upgrade or post-upgrade. I don't know if you have question and anywhere we would like to have feedbacks for but first of all I think it's a very good idea to have such thing for and I would think the quality group may be interested in helping you or keeping track what Colin said to distribute the checks I think it's also a very good idea the main problem would be to encourage the package maintainer to write scripts or if you already wrote to contact the maintainer and to include the checks into their package or their repository what I think what is needed is something like overwrite if I like to check a couple of machines and I know I have some false positives or false negatives I want to disable some checks so maybe you should write down what is the future that someone can disable checks maybe not disable checks but maybe you can overwrite the output in such a way that maybe there is a check that looks whether we have been SH or something like that all over in the scripts and you don't want to disable that check entirely but you might have a certain connection of files over there in user local or wherever it is where you're like I know that's fine those files are okay but go and check it for everything else but maybe it's also important to disable checks if they could break the system yeah is that Heisenberg this is really regarding being able to discard some checks and the maintainer can just grab the output but we can implement something like that and the other comment I had was about the distribution of the files which Thomas also mentioned I'm one of the maintainers of the lock check package and if you really want to try to get files into other people's packages just use your time differently I'd say because it's really difficult to do that and it's also moves the control out into someone else's package let's say you want to do a format migration to 2.0 format now you're going to be faced with the same problem that Petter had for instance in the last three years trying to migrate in its scripts now Colin you said something about Update Manager in Ubuntu I think correct me if I'm wrong but I think Ubuntu Update Manager actually doesn't use packages in the end you'd contact the archive directly don't you well what I'm trying to get at is that you might let's say we have a release we do release notes we have a release and then we find out that there is a problem and we write an amendment to the release notes I'm wondering whether there is something that we can do with Update Manager the way we do it with Update Manager sorry is that better the way we do it with Update Manager is we have a custom upload format that drops a tarble into the disk's tree so that we can update a post release quite easily it's still a package upload to do that but it produces something that can be downloaded separately that that contains the the sort of stuff that we might have to change post release depending on what problems we find so it would be I think in your model it would be new plugins whether Debian FTP masters would be happy with that whether they'd rather that you simply upload a new version of Upgrade Advisor or something I'm not quite sure but yeah that's I understand Martin's point now that's the way we do it in Ubuntu for obviously the rest of everything has to be done with ordinary packages the one thing that is a bit of a gotcha when you do that is that anything in that tarble needs to work with the previous release so we've had a bit of trouble making sure that things continue to work across Python version changes that sort of thing but it's all possible that's actually one of the problems is actually to make sure that that the tests the user is running are accurate for a system currently the way it's done is that if the system detects that let's pretend the the package is named version Upgrade Advisor version 1.0-Leni or something like that it's in any and it will refuse to test the system for the next upgrade and I intend to use Point Release to actually ship a version of Upgrade Advisor which is capable to detect the migration and not include so to introduce a new version Upgrade Advisor in a Point Release I think having it do it itself is likely to result in fewer weird bugs if you can manage it it does require a bit more coordination but it means you don't have to deal with infinite bug reports from users who have forgotten to upgrade to the latest Point Release first but you know I need to understand the process once again let's say we have this package goes into squeeze it doesn't really exist in Leni yet right let's say it goes into squeeze and then so we have version 1.0 in squeeze and then 1.1 because there was an update in squeeze R1 and then we release squeeze plus 1 the next version and that would have Upgrade Advisor 2.0 in it now before the the pre-upgrade checks are the ones that are run from the old package and the post-upgrade checks are the ones that are going to be run from the new package so I'm not concerned about the new the pre-upgrade has to be run basically before the upgrade but it has to be a new package in the old archive okay okay so that actually answers the question that I was going to have which is that maybe we should find a way to somehow make it easy for you know like the release notes start out with section 0 preamble if you have not yet loaded Upgrade Advisor and run it do it something like that so that we actually put I don't think we maybe the release team has a good idea how to do that how to actually integrate that process automated but in the end I think it's going to be the user that is going to accept this offer it's an advisor right it's not something that you need to do it's an advisor and download it download the latest version run that so that that was actually the question I was wondering how you were going to do the do you know yet what you can anticipate the Upgrade problems are going to be in two years and I would have been really surprised and amazed if you would know that it's really I mean any advice of how to be to do it in the most sensible way is welcome because we have to deal with people who are going to upgrade from CD and maybe the latest point release and eventually it will be nice to have it on the CD so the user can run the old version Upgrade Advisor and get some information from the CD or whatever and I don't think I found a perfect solution for that point release was not so bad but I'm not sure what we should do I've got a question which is I'm looking at the sample output you gave of running it if I think about what I would want to see I probably just want to see in the ideal case your system is okay and not checking this checking that and all that stuff so I suppose you're planning to work on that output a little bit in the future that's a broken something which is broken in the current design is that the test name of the test is always displayed and the other thing I want to ask is correct me if I'm wrong but doesn't Ubuntu's update manager actually drive the update upgrade process so I mean I can sort of see why you chose not to drive the process but you've also cut out this helping a certain percentage of our users because they're not going to be able to run this and actually make any sense of the output as opposed to having to actually go fix things so what do you think about that problem actually fixing the problem I mean about the question of whether we should have something actually drive the upgrade and changing what packages are installed things like that dependencies can't handle the same problem that Ubuntu ran into that's beyond the current purpose of the tool but that's sensible to you mean to actually perform the action yeah exactly it's a bit beyond the purpose of the current tool and that's the name says it all but I think you have to think about whether we want to do that if this is actually a big problem we have to deal with your solution is a good solution for some subset of our users but it's clearly not going to solve it for all of them I suppose we could have some another set of plugins where the user can say I ran my pre-grade tests and the actions that I recommended are okay and I actually performed the action or something like that I suppose it should be possible to do it I'll think about it but why not about the update manager part are you aware of that one of our small code projects is to pull back update manager from Ubuntu we write it partially so that it uses Python, APT and so that it is not dependent on UX or that kind of thing and provide much more modularity so that we can write distribution-specific module such as your tool in it that would be a way to extend the audience of your tool I'm not really a user of Ubuntu and I'm not too aware of the tool of that manager it seems interesting and if it can be used we could use of that manager to perform the action or I'll have to dig in that if I look at if I look at the upgrade processes of the season that I've watched my experience is really the details of the upgrade which program is run when the first upgrade or the kernel is decided very very late in the release cycle very very late means something like 2-3 weeks and we really see the issues coming up with one or the other way and I can still remember the upgrade procedures were adopted in the release notes while the CDs were building already so don't expect that anything can be changed in packages at that point in time anymore when I was tracking the release of Linux and added batches I've added many batches when the release notes were actually written which is really in the latest suite before the release I think when you point that it's difficult to get new packages in the archive is emphasizing the problem I think the problem with we need the newest version before the upgrade maybe it can be solved by a command that prompts the user for do you want to download the newest tests or checks or package version and then restart the upgrade advisory command so I think we do not need the newest version in the stable release but a command that can download this or maybe I'm not sure if maybe also volatile would be a good place to put new versions there another thing what I want to say concerning why not only fixing the checks that says there's something broken I think it should really be an option to say no I only want to check it and do not fix it at all because as you said you have some faults positive, faults negative and we could also break the whole system if we think the checks think there's something broken let's fix it but maybe they break it then so this should really be optional we could have some levels of maybe for those who are harmless to do and perform some expert mode by default I understand I think that's interesting point of view I was going to raise the same point I'm personally not going to be a person that is ever going to run a tool that automatically migrates my changes because one of the things he says is ETC is not going to be automatically changed so I build on that and I don't want that Joey's your point is valid Ubuntu needed update manager because the user the target user is a slightly different one than Debian's we might need that as well but there's nothing to prevent us from providing another program that possibly builds on top of yours to do it I was just thinking about the case about you have to modify a file manually after an upgrade so your tool let the user know that case so your tool tells the user that this file has to be manually so I was just wondering if there was a tool or if your tool was meant to do so in the future so that the user makes a package upgrade in a 3.8 route environment then it modifies the file after that maybe he can test the file to see if it works or not I don't know if we can do it and finally there is a command or whatever that says that this file should be automatically be installed in the new system after the upgrade so that we can automate manually modified files in those packages that I don't have scripts that update configuration files so I don't know if there is a tool that kind of automates that or if your tool for the change that has to be performed after the upgrade I think it's really a job of post ants to perform the action for after the upgrade I mean those actions that are specific for the package it doesn't make sense to duplicate and if the maintainer wasn't comfortable doing a modification in the post installation script it's probably not something simple to do it in a credit advisor either regarding the action that should be performed before the upgrade as I said currently it wasn't in my plan to do it but I'm not meaning those files that are modified by post ants I mean those configuration files that the format or the syntax of configuration files changes so much to make it a bright script to migrate from old configuration to new configuration so this file needs to be modified manually manually I don't know if the maintainer said for what has to be performed after the upgrade I really think we should rely on the maintainer script accepted if it's something you probably want to install the latest standard packages you don't have this one and we can't install it because it's not APT's duty to automatically install new standard packages it's not APT's duty to remove old standard packages that stupid example but so but some action has to be performed before the upgrade and those could be used by an upgrade manager as tools and finally my second or last idea is regarding the CH truth regarding the CH truth I think the problem is the space we can't expect the user to have the space to do any test I can maybe answer that very briefly one way that you can make that work that I know Michael Vogt was playing about with for update manager is to use a union mont so you do the whole upgrade in the union mont you let you store the diff in a temporary file system on a file somewhere and if the user even gets to reboot into that find out whether it's working or not and then can do a commit it relies on an awful lot of kind of chunky technology but it's worth looking at hopefully someday we'll have some snapshot system and that would be easy actually there's one message I want to base in this talk and I've skipped over it is that there really is not a return during the release and there's one thing that could be done is to find a way so we could use I'm not making myself clear currently the news.wm files I have one script somewhere which is not in what we have tested which grabs all the news.wm files and put them in the grad advisor so we are capable of saying you are running this package the new package will display you will have this news.wm file the problem is that many message in the news.wm files are not so important but some are very important and it would be nice to have a notion of priority in the news.wm files so the users know this is important and this is minor I think we could modify one of the reasons in news.wm files use a change log format is because that does have a priority field and I think if you look at app list changes it already probably orders them by the priority of the news item with high priority ones first and low priority ones last okay I have to check that I think this is absolutely great I'm very happy that you're doing it and especially in the way that you're doing it the advisor versus the sort of like enforcing the changes let's get it to the point where we can use it so that we don't have to make bash depend on dash because I think this tool could actually help us all the scripts are tested to be bash and bash compliant but that's not what I meant not your scripts because this is something that has to happen during the upgrade make sure that you have a shell installed like make sure that and this is what and I think for the migration for the migration one plugin that could be implemented is to pass the local system for bash scripts and if shebang is slash business slash doing check bashism and once a user about failing check bashism that's a plugin that we could implement for local scripts I probably didn't phrase this correctly I was only talking about the distro wide change like this is something that happens during the upgrade let's say that in squeeze plus one if you have dash the default one or in squeeze then this is something that the upgrade advisor can actually help make happen not check the actual files but help make that change happen on all of the installed Debian systems actually I need to say that if we really want to drop a package from the essential set of packages like bash that's nothing we can just decide in the last two minutes of this talk I know thank you all for your attention and for your feedback I was looking forward to it and I'm very happy of the feedback I had