 So today I talked about Fie me which is a build server for images and first Anybody that never heard anything about Fie Okay So I started this project in 1999 I'm not sure if or I'm now I'm sure that during those times The Debian installer did not have the preceding stuff. So we needed something automatically and Yeah, I installed the first cluster with the Fie and I always do talks on Fie or like today in the lightning talks I talked a little bit about ray cut which is used in Fie So what was the motivation a? Neighbor of mine She came to me with oh my windows laptop is my windows desktop is broken Can you and re-install it and in the end? I installed her Linux and I was shortly thinking about Should I use five for installing her desktop with Linux and in the end? I I did not choose it because I thought Fie is too complicated like the Debian installer I guess it's not really that easy for beginners because there are a lot of question But also fire is not really for beginners. So this was the motivation about thinking about Fie The target group was always advanced sys admins, but I thought maybe it's possible to make Fie usable Also for people that are not that advanced sys admins The idea is Installer should cover most Installations the Debian installer is really perfect because I think It covers like all different kinds and strange environments you can do a lot of things you can Configure very strange combination of language keyboard layouts and so on but I was thinking about an installer that covers like 90 or 95 percent of the installations so a lot of special cases can be ignored and Since the Debian installer As like I think more than 20 questions I thought oh it would be much nicer if they were only like three to five questions And I looked at Linux Mint and Margea installer sent us installer and they all ask much less question and In the Debian installer we sometimes have also things that are asked During the installation so it's not everything is asked at the very beginning For example the task selection where you select your desktop is done after the base installation So this was also very important. I would like to have something that asks everything at the very beginning and Then maybe some tool could create a customized installation image and this Installation image should run then completely unattended so you can get yourself a coffee and when you come back your machine is ready So and there were three things the customized installation image You just boot this image. You do not have to touch anything and then it's ready where I thought oh, yes This is fine or fine may maybe if I can do this So as I said fires only or was until now only a tool for experiences admins and you have to adjust Several config files. These are eski files, but still you have to touch Five to ten config files to make a customization customization so how can I make fine usable for beginners and that's the beginning of fine me and There's a web page. I will show it in more detail later Where you can just click some things and then you can you get your customized image? This image can be put onto a CD DVD or USB stick just with DD and and The customization is just by using the web interface So there's no need for you to edit a text file a config file inside fine I Hope I covered most important things that you want to adjust are a little bit customized You can add additional packages I think that's the most important thing that people say I want to have the normal Debian installation but with some additional packages and You can select different distributions. So it's not only the installation image for the stable release You can create three variants of the installation So this is the web page and Thanks to Yuri He did a great job During the first and second day he added a new feature that we now have a toggle button So here is it big enough or should I zoom in? Okay So we have a toggle button So what you see now is just the bare minimum of questions and we can toggle it to more advanced settings So you have to select or just leave this as it is Username if you do not enter a password a password will be generated and shown to you and sent by email I will now just type in the password It's here in clear text for me. That's fine because Yeah There's also a command that you should change the password after the installation and I do not like to Enter passwords twice. So you can see what you typed in and hopefully do not make any Wrong Mistakes. So for example, we could select the stretch distribution with back ports So we will get a four point fifteen kernel with stretch There are some buttons. We can say we want to have some Debian developer tools. This is Does what I defined in the fight configuration. So just a list of packages and here you can enter your own packages you I will Select the desktop so you can have an installation without any desktop. So a very small installation I will select the Xfce desktop, but all the other desktops are here with the language These are just the task packages that are I think Debian has much more task packages I just search which are the most common Languages and what I do if I say I want the Spanish language also the keyboard layout is Spanish So I know there are different combinations and with the local time It's getting much different more difficult These installation will install the clock with UTC. So if you want to set your time, you have to do this manually So this is this I want to cover the most common installations So we select English us the desktop and As an example for example the midnight commander and gimp I can add An email address so if it would take longer for example if this service will have Success and a lot of people are using it. You may wait for some minutes So your job will be finished So here are the commands how to reconfigure the keyboard or the time zone and Then you just click create installation image And now in the background. There's some Job script Looking. Oh, there's a new job and there's a summary of the configuration of the web configuration Down here. You see these are the five classes. I will explain a little bit more about this But with this information Phi configuration is generated. That's what normally the experienced sys admins have to create but here you just click on some buttons and It will be done for you. So In the meantime We have some more advanced features, which I will also show you later For example, this this very simple installation just creates one partition But you can also select that you want to have a separate home partition or using LVM just by selecting this on the web interface and You can also add your SSH public key for logging as route without a password or what's very nice I saw this there's a new Ubuntu installer does this you can give you a github account and Then there's a command which receives the public key from your github account and put it into the route account So you can log in without password. I think that's very neat and if you have a Repository with your own packages you could also add this and say please install those packages from my public available repository So let's see as we see this job finished in 74 seconds So now this customized installation image is available for download. You can also download the log file since this is an installation image I first have to create a Partitial package mirror and This is done by the command fine mirror and You can also read the lock of this Call of the fine mirror so where a list of all your packages with all the dependencies are available So you see these are the list of packages and later they are downloaded and In the end it says it created Mirror of one gig of packages and Since I have a local mirror. It's very fast So this is the one part on the installation image a partition mirror with all the packages and The other is that the config space which you can also download So this is the config space that was really created for you by clicking the web interface so if you want to do more things with fire you could set up your own fire server and use this configuration space and That's also very new the two commands that are used for creating this ISO image are now also Listed there. So first create the partition mirror and then create the installation image Okay Copy link location Let's see how good the network is here Yeah, baby because in it includes all the packages and with Xfce Libre office and so on and And the installation environment is maybe about 200 max That's not much bigger than the Debian installer what you need to download so To one done I have a little wrapper Which which calls? Fresh KVM machine with an empty disk and boots this ISO image and Then we will see how this installation Runs So this is stray cut putting the image and Now you see the there are already some part it commands executed and now the packages are installed And everything runs on and in the end some customization scripts. We we use Only shell scripts for doing some Customizations and you see the files are downloaded from slash media mirror So this is local on the ISO image It would also be possible to create an image without the packages and then give another sources dot list file So the packages would be downloaded From the internet, but this default in the file servers. We put everything onto the ISO image I Guess it will run for like four minutes Okay What I will show you now is So this was the simple one now I toggle this web page and you will see that there are some more questions you may answer for example you can Give a root password if you leave this empty Zudo will be configured Here you can upload the SSH key or Give your github account that would be mr. Phi for me With the partitioning schemes We have one partition or one partition and slash home in a separate or this version this two versions with LVM Phi itself can do much more. We could do software rate Setups script setups, but here I want to cover the most common installation So very simple. We have only four things that you can choose All two two encryption. Yes So This that was the partitioning things This is the new feature where you can add an URL for your local package repository and The rest is the same you can add Packages you like your email address and then also create an installation image so I Normally said By default I include the non-free Linux firmware This is because My target audience is an end user and I want to make it very comfortable for them So yeah, they can just install it and do not have this problems And since this is not on an official Debian org website That's Yeah, I can do this with this default And Let's see the installation is still running. So advanced features The next thing after this installation I will show you how to create cloud images So this is currently we creating an installation image when you boot it the installation is run fully automatically the other type of service phi me Gives to you is that it creates a raw image or some other formats as you see here Cuckoo and whatever Which you can just boot and the installation is already done But first See if the installation finish Teche ah, okay. These are now The shell scripts that are executed for the customization of like ETC message of today ETC network interfaces is written and so on you see the installation took 236 seconds It says there are some errors, but That's not really true and it stops here, but we can also disable this This is only for showing aha everything went well and now we just reboot the machine and You see the grub Okay, the XFC desktop Debian was a user with password FAI and We have you name minus a this is 4.16 the backpots kernel was one was installed there We have only one partition no LVM and I Told it to install gimp, which is not installed by default gimp is there so so this is nice and The midnight command is also there. So yeah, and Now we just throw this machine gone and What's very nice with this wrapper script? it creates the That the local disk of the virtual machine and slash TMP, which is a RAM disk and I love RAM Yeah, it's so nice and fast So this was the installation image and now we look at the cloud image First you can say how big should your disk image be So here I say eight gigabytes It's you will see it's not an eight gigabyte image that you have to download later by default Z standard compression Anyone who does not know this compression? This is very fast very new Created by Facebook if I'm correct and so yeah, it's for very big files and What what you should never use is? Gzip with sparse images the disk image is sparse and Gzip cannot handle this So if you compress it and uncompress it, it will be very large and all the other xz Z standard Can handle sparse files very nicely So the host name is set the root password Username with the password now we want to install buster Maybe with no with no No, we also do the Xfce desktop Any packages you like to have in this cloud image? Desktop in cloud image does not make that much sense maybe Emacs emacs 25, okay and now Create disk image This will take a little bit longer because we are doing the installation inside file image But no problem. I can tell you What other IDs I have so currently we have the installation and the cloud or Virtual machine images for AMD 64 file itself can also do cross Architecture images so it would be some work to extend the web page to say please create an arm 64 image and We it would be very nice then to have Predefined conflicts for a raspberry pi or all the very different boards, but that would also be possible I guess the next thing I will implement is other distributions Because I know people are always asking it not you but the Ubuntu guys I am yesterday I did the first test with Ubuntu Bionic the LTS release and five just works out of the box with it So what I have to do is to integrate it in these Fimy processing scripts Ready to go cloud images for the big cloud providers That's only as a different five config space that I have to use currently for example in what I call cloud images I do not install the package cloud image That's needed for all the ones I'm also working in the Debian cloud team and This team decided two years ago. That's a Toolchain in the future for the official Debian package will be fine Amazon is already using it. So if you Boot or if you use a Debian cloud image in Amazon Noa Meyer hunts did this and he's using the file to chain for it Google is Not yet using it because there was a very small problem in the config file We had one space too much which causes grub to hang forever and That was the reason why they decided for stretch to use their auto toolchain But the things are working. So we have the config space also for Google and also for Azure some people from credit thief did this and So then the Debian cloud team already has the five configuration for all the big tool providers cloud providers We could also think on a more generic five Installation image So it's an image that you would boot up and then enter your job ID of the web page And then the configuration would be downloaded and the package would be just received from the internet That was one. So the image would be much smaller because the package Do not need to be on the on the Installation image It's also possible to create live images with five. This is a little bit more or Yeah, currently there you need some manual work, but That should be also possible to use five for creating live image And then also to provide this on the FIMI web service If you want to customize much more inside the image you say, oh, I have some ansible script that I want to execute at the Very end then I say, okay This is just a starting point use the FIMI service and if you're happy with the five tools Then set up your own five server Create your own configuration space and then you can do all the crazy things So how does FIMI work internally we have a web server where there are some CGI scripts And this is not the build server. So on the web server you click submit create my image All the input is validated. So you cannot Make nasty things and then the CGI writes or creates a sub directory and puts two files in it a config and a meta file and writes a status Waiting for processing and then the other server the build server reads this config and This is just an NFS mounted directory and sees. Oh, there's a new job. I have two processes it I In this Processing script we pass for some errors. What's happening very often that people type in a package that's not available and This will be detected and then a new version of the web page will pop up and say oh When creating the package mirror there was an error because this package was not known Sometimes I have to or every night I create new NFS routes for Buster If there are security updates, I have to create new NFS routes for stretch and backpots I have some cleanup So if a lot of jobs are created the images are on the disk after normally I say after one day I Just remove the images. So you have one day to download the images There's three different Configurations ETC five stretch Buster stretch backpots. We need for the installation image. We need a different NFS route But the config space that is shared about all configurations. So it doesn't matter if I install Stretch or Stretch backpots or Buster I can can use the same five configuration also for building the cloud images I use the same five configuration The a new job is Is detected then a copy of the configuration space will be made and It's will be customized a little bit. So that are very very few changes. For example, I have to put the SSH key Into your customized configuration space or the list of packages or the user and route password Then we have two things if we want to create the installation image I first have to create the Partition package mirror and then create the installation image for the cloud images We do not need the NFS route We just need the configuration space which is customized a little bit and then we can just create the disk image So there's one step less compared to creating the installation ISO The starters on the web page will be Updated log files written and if the user said oh, please send me an email if my job is ready. This will also be sent to the user Then we have the ISO or the disk image and this will be copied back to the web server where the user can then download it and Since I have a lot of RAM in this machine everything is run in RAM and yeah very very nice So as I said we need an NFS route a configuration space and five classes This is very central component in Phi and this is just a list of names So in home LVM, this is the class name the Phi class We describe and I think this is that example home LVM describes how to partition the local hard disk This is our very flexible tool where we can do LVM crypt setups Software rates and so on but for the FIMI service I just created four different types of partitioning and this is the home LVM example So we have a list of classes and As I said just two commands for the installation image with a list of classes and For the cloud image. I have to say how big should the disk image be the list of classes and what's the target file? That should be created So let's see if this is ready Yeah, it's ready. So it's 1.1 gig This really the raw. Yeah raw. I wonder But no problem, let's download it Should be fast So This is the normal architecture if you use Phi in a client server setup So you should just look on the left side Where you see you need the config space and an effort NFS route and a mirror and these parts will put onto the CD And if you set up a network installation thing This is how things gets from the server to the client For the software installation We have another sub directory called package config and there you also see of Several files where the file name is the Phi class. So since in the FIMI service every Client belongs to the class Debian it will install the packages that are listed on the upper top and Here we have another class non-free So these packages are only installed if you also said please install the non-free Packages and this is mapped to a Phi class called non-free and there's another class for AMD 64 and so on some References in the past it looks more like this when I said oh who's using Phi and During the last months I collected some Logos just because it's just much nicer and Let's see the download was ready We unz the Phi me Image Phi me 0 1 that 3 Z image I On the website I said I want to have a 8 gigabyte partition So now let's see how big is it? So the file is 8, but since it's a sparse file. It's only 3.5 Geeks and The compressed was 1.1 gig now I use my Rapper And I say boot from disk and this is the Phi me Raw image disk image that should be booted up. That's it Debian FAI let's see if emacs is installed. Yes Gimp is already there hopefully and the blue midnight commander So let's see Questions I'm using the on yes I'm using the pre-seed file and for the normal installer Do you have a conversion between your syntax and your configuration files and the pre-seed file? Or maybe can you add a download button for the pre-seed file to your website because I think it's rather nice to have it Displayed in the website first. So I'm not using the Debian installer And I use preceding. Yeah, the Debian preceding for the normal packages You can do this also in Phi and it's the same format you get with Debian get selections and What you get is you can download your own Phi config space and this includes all information you need to set up To do this Phi mirror Phi CD or the Phi disk image command, but you cannot convert This config into a DI preceding or vice versa. That's not possible Because for example for the partitioning part, I do not like to create from my disk config Partman preceding file You can pay me a lot of money. I will never do this You know the department preceding is very ugly and very heavy For other things Yeah, selection of the for example the selection of the language. These are the normal preceding we use So and the list of packages tasks select. I think it's much easier if to do this in the Phi configuration then to create a Debian installer preceding and Why use DI if this works for you? Yeah, it works as well for me. Yeah, yeah, then fine use it Hi Thomas, yeah, thank you very much for this new feature in Phi project It's very nice, and I found very very great that you have the output of the of the commands that you use to create a the ISO image or the cloud file a question that I have is in in which Servers are located the The file that we create the ISO or the cloud is a server that you own host or trust me So Yeah, currently this is also Both the web server and the Fimy processing build server is run on two machines at the university Where I work as a system system administration. So that's also why we have a very fast connection The CGI script and the shell script that is processing these jobs is currently not open source So there are plans to do this. I'm not sure when If You want to reproduce the things you have the config file and you can download the file software and use this one or two Commands to reproduce it Some people said oh very nice service. I would like to set up in my company then please yeah contact me and Currently there are no concrete plans to make this background scripts open source, but it will be in some future but Currently you have to trust me and as you also have to trust all the package maintainers that will be installed there But you can verify it or say I do not trust Thomas But I will just grab the five config space and do this then on my own Thank you There's a There's a question from the internet. Yeah Why not use a proper job queuing system like grid engine or Similar and I'm using I'm using grid engine at work for different things It's it started as a very simple project. So in the end it's just a loop which Checks if there's a new job or not currently I do not process jobs in parallel currently. There's no need for it If this project will be very successful. Yeah, I have to use a queuing system It's yeah, a very simple script, but it would be also possible with a proper queuing system More questions. I have a bunch of questions. Yes So first what is it that you use for partitioning? I'm using a poet script that we wrote several years ago in Phi and we we defined this config file this package config and and The pearl script parses the scripts and then executes the part add and mkfs commands Which you can see in the log files So if you want to see what does Phi do after parting this which commands are executed you see everything in the log files Right, but so you turn this Text into partitioning Comments. Yeah commands, but the text looks like this like with the spaces and everything Yeah, or you can use more or less basis or do you like should I convert it to XML? Okay, then my next question is what are you using the NFS route for when you're generating the images Yeah, the the NFS route is used Only for the installation image That's when I do in the installation. I need to boot the machine as a diskless client So it's it's just what the Debian installer loads into RAM. You need a running Linux system This is our NFS route on the installation image So when you boot the installation image this NFS route with all the commands We need are started without using the local disk and then we can do everything on the disk change route in Target and so on but this is the NFS route is the system that is running during the installation Okay, but there's no need for this to be NFS. It could be a local It's called NFS route. So it's then this is a very common That people call it NFS route and if you have this network installation thing It's really an NFS route, but on the you're right on the installation ISO. It's not NFS It's it's just a local file system. Yeah All right. So and then I think it's my last comment So you have you have like the the ISO from which you install and when you install from the ISO You're installing then packages like on the machine and then you have the image Which is like a disk image that has the packages already installed. So you like you skip the installing. Yes Have you thought about having like an intermediate thing where you download an image that already has the packages installed That's also possible what Then when you do an installation before you can change route in the new system for adding Packages you I have to call the bootstrap What we do we call the bootstrap once and create a tar file out of it So this is our minimal in the former days. It was the floppy disk or base tar GZ file So you could exchange the minimal tar file with whatever tar file you have That's for example, what we do if we install Ubuntu We boot the the installation system, which is a Debian system and Then create the files a local file system and extract Ubuntu base image and Then we can change route into the Ubuntu or the same for Santa s and so on then we can change route into the other Linux system and add packages there and if you have already a bigger image with some more packages Edit there. It's very easy to say do not extract the Debian stretch image, but use my image which also Includes other tools and if you are fine with that you can just extract the tar file Okay, any more questions Yeah, the heading is in German What? The heading is in German Because it's a copy of my German slides Thank you for this and what's also missing that The the web page where you can select German or other languages would be nice if people are interested to help to translate them So that it's more easy for people that do not speak English To use the website and create their own Installation image with their language Some someone on the stream said that the fight dot me website is not yours and it's a it's a Hack thing. It's our it's a scam you go there and you get hacked Do you have any plans to try to buy the domain because it's pretty confusing the first thing? I would have done by seeing that talk You go to five dot me and yeah, I Was thinking about I Thought longer about which name should I choose? I didn't check which web Domains are free and in the end I thought oh do I like to have a different domain name for the service? But since it's only a part of the five project I thought and technical it was also easier just to host it under a sub directory and Yeah, if people now Yeah, grab the five me domain and do other things with it Yeah, then I think one question would be to use a debian dot net or maybe debian dot org Domain because people trust much more I get a lot of commands. Oh, this would be very nice if this would be hosted on a debian machine But this would be much more complicated because the dsa team has much more restriction What to execute on their machines? Currently we need root access because we mount some things and this is not er DSA will not would not give me root access on any debian machine That's the same problem We have in the debian cloud team where we want to create their official images for the cloud Providers where the debian cloud team will also not have root access and so there's much more work to get empty virtual machines starting up Putting data into it creating the images Recesing them from inside the image Yeah, and since on those machine I have root access That's much easier for me Right. We are out of time. So thank you Thomas again. Yeah