 I'd like to talk about how we are using Debian in a larger, somewhat larger infrastructure as a German federal foreign office or another name would be the ministry for foreign affairs. In contrast to other talks I will talk about, even about some problems we have seen in Debian. I wouldn't mention any problems with Debian in other talks, but I think the local audience can cope with it. First of all, I want to introduce you to our IT department. We have four IT units in the headquarters in Berlin and Bonn, and these are our central IT units. So our IT units aren't gathered around different locations, so that might be different to other institutions. We have more than 11,000, almost 12,000 computers, or client computers, both right. And most of them are located outside Germany. I think 75% are deployed. We have way over 200 missions. These are embassies and general relatives dates. For instance, there are general relatives dates in Edinburgh. And we have two IT zone centers, two small IT zone centers almost in New York and Singapore that are responsible for giving IT support in the time zones outside or far from New York and Africa, especially in America and Asia. And overall we have very little IT stuff. Most of the IT is located in Germany, and all the administration is done remotely. I work at a very small unit called IT strategy. It's like a CIO unit directly working for the other IT. We are managing the IT workshop responsible for the IT of security. And we are making strategic decisions like using DVR in the ministry. And as a side note, I got my job through our formal project data. Martin Liebkemeier wrote a mail to Steven Private about the job offer, and I trust he got that job. So it's worth doing, working with DVR, or for DVR, if you would like to get a job. Okay, the next is I want to talk about the history of our IT history and the 2003. We had a lot of those in our missions, usually one, but that's all about the mission. We are using up to now IPsec or internet for a secure connectivity of our missions and we are using a German-branded order called ZINAR, a secure internet network architecture. It's a prototype of our national security agency called ESI. As a science, we have used Microsoft Windows XP and Microsoft Office as missions. And at the headquarter, we have still used Microsoft Windows NT as a service in the clients, including such things like Microsoft Outlook and Exchange. It's quite a classic installation except for ZERTA, so it's maybe unusual for this time. 2004, we have introduced DVR to the foreign office. We have used it for so-called internet terminal servers that are used to have some more secure access to the internet. So there's no direct, at least no direct take-back to our network, to our mail servers, or to our client computers, because everyone has to use a terminal server to access the world like that. And we have started installing more and more internet applications on DVR servers. In 2005, we have refreshed the hardware and software in the headquarter. Since this year almost all servers are DVR servers in the headquarter, and the clients have multiple configuration, where Windows XP is installed and DVR is installed in the clients, and the users can choose which operating they want to use. And we have removed the Microsoft Outlook stuff completely, and replaced them with Sonobot and Firefox, and installed Office at least in addition to Microsoft Office. In 2006, I would call it the Speaking Up phase. We have started installing the placed servers in the missions, because they are tested to old and empty clients, not re-absorbed or re-absorbed even clients. And all new servers, all DVR servers, and all new clients are multiple clients like in the headquarter. And we have started to install DVR only on every notebook. So everyone who wants to get a notebook has to use DVR, there is no option. And these installations use this special CNA based virtual private network product, or a special journal supporting such CNA weekend, and the user tools that are necessary to fill up their weekend. This year I would say is the start of the final phase. We are starting to migrate some missions to DVR only. About one week ago we have migrated embassy in Oslo to DVR only. It was quite easy, I would say, after that there are no big problems. And currently we are migrating the embassy in Cairo, which is quite a bit more larger. And I hope that's our problems. And we are using the virtual box emulation for legacy software, like not very popular instance. That works quite well too, I must say. One question, so the final phase has to go to migrate all missions completely to deviant? Yes, all missions and in-depth portal. After that you will not have work anymore? Yes, my talk is done then. Probably not, because there are still a lot of things to do. Before I come to problems in DVR, I will talk about the quantities of DVR. I think I can keep it short before it knows more or less what are the quantities of DVR. But I will just mention the major points. With DVR I have a large package pool and a large developer community here. It's not business driven, so it means we are dependent. We don't have to fear that the DVR organization is making contracts with Microsoft. That's a big advantage. It's a really big advantage, so it should be the wearer. It's in the real resource distribution, nobody cares if you are recompiling your kernel. You can just do it and you will get support from the backtracking system, even if you are using it. Users can easily contribute to a good or a bad advantage. You have the backtracking system, you have collaborative development or packages. You can contribute to translations as you can start and you maintain the process for yourself. And what's interesting is that DVR still has a focus on end users. If it's not that easy to install DVR, it still has a focus compared to other community projects. Especially if you compare us with other large community projects. And then maybe Gen2 or the 3BSDs that are community projects. But I think they are less focused than DVR. And yes, the high strategy is always the thing with the backtracking system to the strike release process in such a commercial installation. Now I will describe our development infrastructure for our installation infrastructure. Our goal is to package everything we have in that format for easy distribution installation. We have been using a package like this of his templates in that format. When we use them in the virtual hops, it's just easier to distribute them in that format. For that we need installation and development infrastructure. We have our own normal, we have components like archive with upload views, a built-in server, we have our own policies of packages, and we are using the usual software. But instead of the version, we get more and less main list, media requests of this and the 3BSD system. More therefore, most for support to end user. One problem is that it's still too hard to set up such an infrastructure and to maintain such an infrastructure. So I think it costs a lot of manpower to keep it running and to set up. For instance, all the tools I have mentioned don't have proper ADAPT support or value limits with ADAPT support. But ADAPT is trusted as a very important organization. We are still missing a lot of help tools. We have to provide a lot of help tools for archive maintenance. You want to have such things like a package tracking system for yourself, but you have to do it more or less for yourself. It doesn't look like a package tracking system or something that works. It would be nice to have such things in Degan that make it easier to set up your own development. We even have our own installer. It's not because Degan installer is left. Degan installer is a good tool, but it's not fit our needs. It simply does not fit our needs today. At least some years ago we found that it's too difficult to fix Degan installments. It's easier to justify your own installer instead of fixing it. Maybe we should change that to future and fix Degan installments so we can use it directly. What we are doing with our installer, we are doing automatic setups. Of course, I've looked at some hard-peaked clusters. That's not easy to do with other types of installs. We are setting up a dispute with onboard dock devices. It must be done in sync with an every partner. That's the logic avoidance on top of the OVV. I'm not aware that any other tool is able to do that fully automatic. You can buy two scripts, of course. That's what we have done at these two important scripts and package some Degan packages. That's what we're doing there. We can create an installation making up any client you need, the ROMs, DVD ROMs, network-based installations, whatever you need. We can support torture machines, what you did for it, like Xen torture the box. And we can support image-based installations, which is necessary if you want to install other operating systems that don't have proper installer like Windows. Or if you want to buy your hardware pre-installed with your operating systems, it's usually easier if you get your hardware and the operating system is already pre-installed. These two are faster to adapt your machine. And we can separate the installations from the data process. That is more or less the same. You get the hardware pre-installed with software and you're just setting up the machine before the user gets a little server. It starts working and you're just changing the OS name, the password, whatever is needed. It's been more or less of an automatic way. The next point I would categorize as policy-like, roughly as policy-like problems. One problem with a lot of P1 packages is that we need to run a daemon twice or more than that on the same machine. You have a low-balancing hardware cluster, and if one of the partner breaks, the other problem is that you need to run all the daemons alone. That means you need to avoid outputting paths too fast and directly. The head problem is with Zambar, because it had some outputted paths, so you couldn't run Zambar twice in your machine. It was simply necessary. You had to change that. Other daemons supported like Postgres, and it's not Postgres multiple times in multiple instances. That's not that difficult. But Postgres takes a way to automatically bring configures such as installations, so you still need some kind of strats that's just doing the setup. That would be nice if the packages would be able to or have a way to pre-configure them, so if you insert them, you automatically have two Postgres instances. One is automatically restarted, or automatically start via hard-read, not via instruments directly. Another problem is the user ID, group ID mapping on Schertstorch. Debian uses a lot of dynamic user IDs and group IDs. During installation, that's not possible if you're using Schertstorch like DRBD we are using, or any kind of storage area network that we have in our headquarter. Most of all, those in that quarter are just directly putting from the storage area network, so you have naturally a Schertstorch there. At least in the Schertstorch, it's necessary that you have some fixed mapping between numeric IDs and names. And we're nice if Debian groups would have a way to pre-sead such methods. The way is just to create all the necessary users and groups that are all in the installation process, so the picketers themselves don't create users and groups dynamically. But that's some kind of dirty fix, so we're nice if they have something that is picked up by the policy in Debian or something. Debian does not define a central configuration database. You can build something like that with deep confidence and things, but there's no definition for a central configuration database, so you have to invent one for yourself. We've been nice. We had a mechanism that just works in Debian. In the configuration mechanism, it's not extendable, but I mean here, the other picketers that can be pre-seaded by Debian, like OpenSSR or something, the SSH server, OpenSSH server or something, but if you want to configure more than what the main table has intended, then you have to invent your own. You either have to change the picket, the SSH server, or you have to justify the configuration database. There's no way to extend the mechanism that the main table has built in Debian. You have to go or create the configuration by adding more detail because that's what we need at least, because we have a central configuration database and we don't configure any sort of line directly at everything, every configuration item that needs to be changed needs to be changed from the central head-up server and needs to be replicated to the actual configuration file in Debian. One advantage is, of course, if we have plain text configuration files, that's a real advantage, but we have no good mechanisms to replicate the configuration from the central server to the actual DGN machine. The package-grunt files are not very flexible. For instance, you cannot present such things like force-configured and force-configured for UVTL files, that's sometimes a good thing. UCF at least uses some environment variables, like the UCF force-configure and UCF force-configured. It uses step-con, but UCF is not required to use for every given package. It's not even flexible enough, I think. So it's a bit better, but it's not good enough. Another amazing tool, I think, is some kind of de-cursion tool for configuration files. Don't have such things. And the conferencing of the package is not even fully documented. I have an example from the policy. I found a trust search, for example, on the CIF file. UCF explains what happens during perching the DGN file. It mentions that some verifiers get removed, but ends at, etc. So you don't know what really happens during perching. So it ends somewhere. There's no documentation on what happens during perching. Except the source code, the package. Moving configuration files from one package to another with name changing or without name changing is another problem difficult to handle. Especially if you have a lot of your own packages and sometimes you want to move just this configuration file from one to another or you want to split a large package into sub-packages to move the configuration files around and you always get headaches what happens during upgrade. At least I found it difficult to find out what happens during such stuff. Another mechanism that would be nice is to use run parts for configuration files. Especially, for instance, so the sudoers would be nice to split up in small files and have a directory sudoers.d The trust contains all the files and it gets combined into one configuration files, sudoers and during hostings or whatever. And it would be nice to have a default mechanism for every configuration file that even supports migration from the old scheme to the scheme with multiple files that gets combined. It's a difficult part to split but it would be nice to have it in the end. Very good. Another thing is that with configuration files you have a number of backup for instance of the old files and you can choose a diff to see the difference to the old version. Yes, I already mentioned the central database of configuration data. Probably we have a lot back in of that one but that's still experimental as far as I know and it's very limited in what you can do. So it's not really useful for any practical purpose as far as I know. The best option is to try as the end-up files that you distribute around in some way via HTTP or whatever but it's not even a standard mechanism. So we are missing a standard mechanism and the mechanism that we would like to have is a template mechanism where you have a template with configuration files and some items when the file gets replaced by data from a central elder or whatever and the final file is written by placing parts of the version template. And that mechanism would be sensible for us so it shouldn't be limited to what's the main theme so it's usable for every user to be able to be able to easily extend the content. And of course if you would have a central database you would need some kind of user on the front and some user interface for instance an application or something like to change the database. That's missing too. Okay. At the end of my slides already took a bit about maybe I'll update the data solution but we have questions. I think the ID how to handle the configuration file with the DPEC HD Birdlight tool is very interesting. I talked to some other guys and the ID was how to handle for example also the custom Debian distribution. Currently as far as I know Skoda Linux for example is creating an Apache package on their own so they can include their confiles for the web server even. And I think it would be much better that Debian has only one Apache package with some basic configuration and other packages for the environment package or you would create a package only with the config files so you can create an Apache web server with the plain Debian files where the binaries are mostly included but the config files comes from your package. And currently I think one great problem is that the policy says that a package cannot overwrite or change the config files from another package. So this has come up as a question or issue several times over the years and I think that my idea has always been very similar to what you're suggesting but the right answer was supposed to come up with a standardized way of handling this to meet everyone's needs and then it would be completely appropriate to change the policy to support such a mechanism. The challenge, I think the reason that the policy is a different way is now is that it would be complete chaos if many different people invented different mechanisms of overwriting each other's config files and we would all have a really difficult support issue. I hope that people don't read things like that in policy as things that can't be changed just as good indications of how we should behave until we have a good idea or come up with a good solution My first idea was if there would be a tool like the package rebirth for the config files that for example the Scholar Linux configuration package could take those config files that they like to change with this main version and label it with the Scholar Linux so the blame package system would know how if I do an upgrade of the plain departure package I do not have to touch several config files but the Scholar Linux package if there would be an update of the package it says these are under my control so I can update the versions if, except the user that makes some mental changes to it I think doesn't sit in the policy somewhere that just diverges some config broken not actually intended to work it seems like this is something that could actually work and you just sort of get this message I didn't really get the future that it sort of forever did Even if it's not allowed currently I think this is the second of the third time I heard that we need something like this so I think it's a very hot topic and it would be very good maybe to have a buff or conference to make this a buff so we can talk about this because a lot of people can use this feature a lot of different interest would actually work pretty much wouldn't it the same solution and I think there's another idea that you have in your list involving this notion of more configuration files coming out of directory more deliberate fragments under our policy I've been talking now for at least three years about the idea of config file layering and it seems to me that some combination of this there are some configuration files that I think are structured in a way that layering and sequencing that have these override defaults works well and there are things like behind the config file that really just have to be a place to take so some combination of these methods well articulated in the default file it strikes me that if we're going to do something in my back it really does need to be done with the actual involvement of other Linux distributions as well though for those of us who are for one reason or another running a few genius environments because we have to do one myself you know I don't want to have a big configuration scheme for a procedure that I have for Debian as I really can't get something that I don't think we can do in isolation so that would suggest that in addition to coming up with a good proposal we should figure out how to engage the LSD folks and other distributions who worry about this in the process why do I feel it would just look like it's objective to develop a distribution so we have enough information on a bit of luck what was your current solution for this problem of the config file handling we have found out that if you have five solutions the bottom is worse than the other one so we don't have a resolution for that sometimes we are just setting the DH compact version to back to two where that purple does not define every file below as a config file sometimes we are just moving the Debian config file sometimes we are just overriding them by post install on something like that and setting dpcp where is the right place for us to continue this? it's a post config it's a Debian package that does not override our configuration files I think all the involved people what about the requirements is this something that wants them to go or someone suggest a place for this kind of conversation to continue on? I would say Debian Devil because a lot of people that needs some sort of this thing don't change about this would be a custom Debian distribution that they are creating my last question to Debian customer they are so focused in producing a CDD that they have too much they need a different approach to change confines what we just want to be a personalized Debian but I think myself I need to a personalized Debian idea something that can work now or a new very new feature for now I do what I'm using I install my package with scripts and patch that changes confines after the installation and it's working enough for me but I'm looking too far some other way to do this my idea was to enhance what you have in a way that is usable or and try to convince the maintainers to use UCF for installing a variation files but I've learned that my knowledge is a bit strict about changing UCF adding more features to UCF so probably you need an artwork to work fixing deep package might be difficult but you can create a package called UCF next generation that was important as a starting point I was going to suggest that sometimes the right way to convince all of us of these things in fact that we're hitting on in the last night is to at least build a prototype of what you're talking about and demonstrate how this will make the world a better place and then it's much easier for people to rally and support and it's difficult sometimes for a paper design to achieve much sense of consensus certainly the kind of consensus that's required where they can find a lot of changes or changes in policies that's probably what I might do next year when we have switched everything to VVM so I have more time to do some stuff I've even tried to build a prototype but it's far from being presented somewhere because it's too limited even if you use it, don't use it but I think the first step would be to collect the people that are interested in this topic to document the limitations I think that was the idea behind the talk start with documenting the thing maybe even the key and as a starting point and if you could document missing features or the problems you had I think we can people will get involved and see, oh, there's something documented that's also interested and interesting for me so people will come together in the future yeah, maybe I will start with documenting I think it's very important because if somebody only reads the title of your talk you would never expect that this kind of topic would be a part of your talk and I think it's very, very interesting that you also had the problem and talked about how to solve it so first thing we have to point out I tried to explain this as I expected but I didn't you wanted to open because usually in the public we don't talk too much about dig and pop there it might sound like maybe on the surprise but it's really easy it's not about problems it's about new things that we will invent in the future hopefully I think it's completely appropriate for even for those of us who truly love heavy hits of this distribution talking about where we need to go next because in fact I've been chatting with some person I already seen in the last few minutes I think are a little sad that they didn't come out to hear your presentation at this point but we'll make sure to see it later so hopefully it will sound like just out of curiosity but how do I do it? how do I do it? and so was 8.4 our little with the Siemens PCs and think that's not that's more or less for storage we are using EMC in that part you mean something like Nagios no in the box is do that by using the serial mode do you go to homes and know we don't use any promotion tool for touch things I don't really understand what you do in those you are using what HP provides is IOP cards for onwards that's actually a pretty good tool set and the only thing I would caution you about is that if you remember you have a service you want to think about including the advanced pack idle option because we have that to be able to take your mode access to the it sucks a bit it's using this H2XS it's good enough the good news is it does support remote SSA jacksets and you can do things like firmware upgrades from boxes using the LDAP back in the deck the LDAP back in the deck you understand the LDAP LDAP the LDAP the LDAP again the LDAP again I do try to use it as only as a remote without the device as a way to press it in the package no, not really because it can be published in other ways that are only less complex visiting and we have just I think 9 deck quantifiers that we are exiting and that's just a clean file that we are distributing and including our deck quantification as well so it's simple that's quite simple when you were talking about something like a compile directory to deliver aggregated fragments with an example I know you were thinking this is a more general mechanism for the example and you used the suit have you ever tried the suit held up back in support that would be an option in that case but that's not a job I was just curious but we haven't used that we haven't thought about that but I don't have any reasons why maybe it was easier or it was too complex so whatever I don't really know but we have considered that I was just curious how to maintain that I think one of the problems is that the suit must work even if you don't have network and you must make sure it works and so as the suit of maintainer and someone who does not personally have any LDAP I'm always curious to know if you have some experience with it Indonesia's motto is unity and diversity and you can see by the news how well they're getting along the problem is that as the country has grown people are educated in different areas to everybody learning everything they learn their unique area and that causes conflict with people in one area thinking that the problem is in some other area as an outsider this is my first heavy in conference my analysis is that what is being imposed is somebody that is unbiased very creative very good documentation writer to start at the very and document all of Debian and how it should come together I think we're documenting individual projects but I'm not hearing that we're committing a whole system and I'm hearing conflicts about one scholar versus another scholar and all kinds of different things seems to be that some documentation that brought people together would be the greatest thing and that's just an outsider's opinion in the public what do you think? I don't think there's any suspicion that well I didn't remind you I think the more you get into something the more biased you are if you're in one particular area but not in the other area but isn't that just like any other area way too big for anyone who has been motored across the whole thing but it's just more and you cannot create one solution for all it's about diversity for sure it's better to to unite some things into a more general solution but you cannot do everything for all and I mean it's basically a good idea but you don't want everything the same you can't start all using Microsoft because that's what already it's just a stupid idea to use Linux and even have multiple distributions I don't agree right there there's ways in which diversity is better and unity in diversity I think is the greatest concept but my observation is that as people become more and more specialized they tend to think that their own area is the most important and conflicts arise with other areas but you have to remember in the open source community there's quite a strong element of natural selection of work and actually having people are trying different things all the time and sooner or later their consensus is right that it may not happen in a specifically organized and directed fashion but I think it's a very natural selection of work I don't see it there are so many good conference free conference organization software around and this conference is based on five moments and I haven't been able to even find anyone to ask me more than a dozen people how to solve the simplest problem with pet-to-bar you know I don't see natural selection of work I see somebody having a pet project and poisting on us the worst conference software I've ever seen that's true but it's not changed from today to tomorrow I would assume we all realize that hand-to-bar might be quite bad and now should we try to reprogram it in one day for the next conference I believe it was as natural selection already proved to be very good rather than trying to rewrite pet-to-bar natural selection takes years so and stuff they didn't change from one day to another to be able to live in the water they needed thousands of years to do such things I have never had the idea to do it myself I have looked for the pet-to-bar installation by myself because I've seen the problems and just fixed it with the idea of a hand-to-bar you're right I had many problems with it so in the start I wasn't even listening from the log in to see the schedule and back but yeah you cannot change it in one day but everybody who has some time can keep in mind that there is a system that is not optimal but it works in general and you can help to make it better and if you come up with something a thousand percent better for next year then I think a lot of people if you show that to some people a lot of people will help you to get this for the next year this is every single process and half of my pictures have been filled in quite often and T is quite responsive so that's how this all starts I don't think your knees are bad in general but yeah everything takes a lot of time for sure and no one single person will document the whole Debian project this is really impossible but there are people working on unifying and documenting stuff but even the existence of other people who create another wiki about Debian maybe in another language already then you have splitting of information and there are so many interests that you cannot you cannot just say nobody is allowed to do with Debian wiki somewhere else for example I don't see how these things can be done practically for sure we can remind people it's better to use Debian wiki instead of create your own and it's better to work on general solutions but everybody cooking his own soup but sometimes people have different needs for sure the idea could involve something very very good at automatic installations but I have to meet today and that's why I'm using FAI for example maybe I could invest very very much energy in 5 years the I can do the same as FAI today and for the I people it's exactly the other way around they need something what is manually capable of doing a lot there are not so much interested in automatic stuff they need it today for sure I can say to them if we work on this in 5 years we have all the manual interactions you have now in the I in 5 years in FAI but I am not interested to work on it and the people who want it need it today that's why we have 2 different installers for example maybe even more but that's the 2 most important I think but that is the reason for diversity so sometimes diversity has a reason yeah you're way over my head about the I versus FAI but I asked somebody else about which one and he says why make a choice why not the I support FAI and people work in FAI and finally why not hook the 2 of them together if you want that yeah that's why it's open source there is somebody who needs to hook them together then you can hook them together currently I see it more as 2 different solutions which do something in the end might look the same in 2 different ways and because people have the need to do these things in 2 different ways the one that wants to click everything together because maybe he has no 2 computers of the same type in hardware or software installation so you don't want to create an automatic installation for that but yeah I have an idea why you want to do that because it's retro visible but this is just one example of why diversity has a reasoning and yeah we could try to put them somehow together but you might end up with a jack of all trades master or nothing you probably cheat one together so I think we'll start another talk so no other questions we can come to the end and discuss it yeah