 So welcome everybody. This is a talk about bee builder and to make sure that everyone is on the same page I'll try to explain who I am and why I'm speaking here Does everybody know my name is Junichi Uekawa. Of course everyone can pronounce it Yeah And I was surprised the other day that Steve Langus I could spell my name. So that's a very impressive And I usually live in Japan. I Have been developing developed process around the year 2000 and I my usual interests are about all your processing I Play around with the localization stuff and also some package quality assurance stuff And P builder is one of them on this week. I have been hacking on Well for the last two days, I've been playing with the Ups list bugs for playing with the BTS So I'm doing that kind of stuff. You usually Okay, so what I wanted to do was to talk about What around P builder is happening? Okay, so how many people heard of P builder today gravity hands, okay So everyone knows about P builder. That's great How about P builder UML? How many people know about them? Only about a half ish Well, that's a bit minor because user mode Linux is less easy to set up How about how many people know about cow builder? Well, there's more people that's impressive And how many people know about game of builder? Okay, one person. Oh three Great. So I think there's something that people don't know about that. I can talk great So I think everybody knows about this. What is the concept of P builder? How would you explain? To explain P builder to a new person. You probably need to explain what is a cheroot. I Think everybody in this room is comfortable with the concept of cheroot. Who is not comfortable with the concept of cheroot in this room Okay Well, I'll explain anyway So if you're using P builder you're by default you're creating a cheroot inside a directory Well, which is called ver cache P builder build something something and you have a whole debut installation under the directory So you're usually working underneath it and when you're using the cheroot command You're doing cheroot and directory name and the command you're using in which case And you usually use something like a bean bash to get a shell prompt inside cheroot So that's very simple and the normal operation of P builder is to Have a base.tgz file Which is a tier gz of the cheroot image So you have the debian installation under a sub directory which is extracted and recreated from the base tgz every time you run P builder So that's the 30 seconds you wait for every time you type P builder Oh seconds, okay, you have a fast machine then okay, how do I use P builder? How many people here? know Do not know how to use P builder Okay, so I will explain it's good to set up P builder. You just get up to get install P builder debian is great So you just install the package And you have to actually configure the package because I haven't actually gotten around to setting up through deepcon form So you have to edit the I think the mirror place In the P builder or see you which specified by default FTP JP debian org Which is a pretty bad default, but it's one of the full mirrors FTP debian org only has the major architectures So if TPJP debian org has all the architectures available But it's very slow from the rest of the world except for a few countries including Japan and then you might want to start by reading the fine manual Which is in user share dog P builder people dog PDF How many people have actually found that there is a manual great So I think there's about 10 people who knew about the manual and the rest of people didn't know well you do know now so great So we do have a man page There is a pretty big Documentation about P builder which has all the tips and tricks and how to do when such errors occur And what kind of commands do you use when you run P builder? And I drew a diagram just for the sake of it You usually run P builder create once Then you do P builder update every time Something changes in the distribution And you run P builder build to build a package inside the base.tgz Few years ago you only needed to do P builder update once every day however because The archive is updated twice every day you have to do it twice every day Which is a bit annoying because P builder update is a bit slow But not too bad So for anyone who hasn't seen the P builder create output it starts with something like this and goes very long How many people haven't seen P builder create output? I Think there are about 10 or so people here, but P builder create runs the bootstrap Which essentially does most of the installation So it takes Several minutes. I think on a fast machine But you only have to do it once I will create the base.tgz. Then you will do P builder update Which you will update the cheroot with a up to get update and upgrade So that you have the up to date system every day or twice every day Then you will be able to build a package inside the cheroot with a P builder build where you specify that Description debian this DSC file and it will build inside the cheroot It's kind of simple. How many people think it's very difficult? Okay, a few people that's good And there are some other features such as P builder login where you can log into the cheroot Initially, I didn't design P builder for such work but it turned out it's very useful and There are other dedicated tools like S cheroot For having a login but P builder login seems to be very useful as well How many people use P builder login regularly? Okay, so it's a needed feature. I Try not to break it Okay, and there's a tool called P. They build which kind of mimics the build command So that you don't have to type in P builder build DSC every time you have to build a package How many people use P. They build? Great One thing is that P. They build is not exact copy of debut command and it does things slightly differently than they build So that's one problem, but I've been working on it to be More closer to debut so getting better sometimes And to run the build you have to be in a debian source directory Well, you have the debian directory so How many people do not know what a debian source directory will be? Okay, so I don't have to explain that Well, it's the directory where you can add a debian rules and you find debian rules file And you run debil and you will try to build a package inside a cheroot We create a source Table and other stuff Then create Then run the P builder on the source on the source package that's created So it will run Debian rules clean on a directory And it will run people debian rules clean outside of your cheroot if you don't like that it can run with the P. They build Use P. They build internal command How many people know that obscure feature of the use P. They build internal? Okay, about five people There's that feature of running Debian rules clean inside the cheroot so that you don't have to satisfy the dependencies outside of cheroot It's sometimes useful So I was wondering how do you maintain a package today? You receive a bug report and probably get an upstream package then You edit your package build it fail edit build Install locally tested fail edit build test install Then maybe P builder comes along and he's build the package and say well looks good It can upload it and upload. Is that the way you work or something else? Mostly yep, okay, maybe you should test it after a P builder build. That's a good point How many people do that? Ah Sorry, I am naughty I haven't put it there Okay, so you should test your package after your P builder build Then well when you think the package is ready you should probably do a Dev sign. How many people not know there's a Dev sign command Okay, so there's Dev sign command which is for signing your Debian package And then you can use the deep command to upload your package to the Debian archive Then I would like to talk about the different backends P builder has there are Different possible backends like LVM, UML, CalDance, and QMU How many people have used more than one of these? Only about three or four people, okay The motivation is that P builder based TGZ creation is sometimes slow and also The cheeroods do not have segregation on the process space So if you're running something inside a cheeroot and it's running as root privilege It can affect outside of cheeroot theoretically So to make the segregation better There are tools like QMU and UML and to improve the speed of Turbo extraction I Can use LVM and UML and CalDance and QMU and other solutions So that's the basic motivation for other backends and exploration. So for UML there's a Process space is completely different from the cheeroot and it has a Cal block device support which means that I can Run P builder on a block device and It writes to the block device. We've written a different file and I can throw that file away To keep the original state Which means I don't have to keep on extracting the table every time And it's very fast However, UML itself is not very fast. So that was the first try which I've done five years ago some people use it But it hasn't been very widely used because it's only available on I386 and Only recently being enabled on MD64 and is not available on PowerPC and other architectures And there's this CalBuilder thing Basically, what it does is do instead of extracting the table it has a Cheeroot tree extracted already and we do a CP-AL of that tree What CP-AL does is to create a hard link of every file and then I run a cheeroot And instead of a bash I run CalShell What CalShell does is to protect file writes And when it detects there's a file write It will try to copy and unlink the hard link So that it will preserve the files so this tool CalDancer is pretty useful for other purposes than PBuilder and I have received a patch For using CalDancer for deep patch edit patch So when you have need to have some temporary stuff and you want to maintain the differences CalDancer is pretty interesting choice The CalDancer, however It uses the concept of LDPRE load of doing a CalDancer.so Which overrides the functions which are likely to change files The problem is that Open and Other stuff is preloaded through LDPRE load, but it's not perfect. So not everything works The implementation is lightweight in a sense that you don't have to have a block device Dedicated to the copy and write file system, and you don't need a kernel space file system Currently, there's no mainstream kernel space file system which supports the Cal Support, I think. I don't think any of your FS has done it yet so Currently you need a kernel patch to get a Cal file system and Or you need to do your LVM snapshot, which means you need to have an LVM volume So CalDancer is pretty lightweight because you just have a directory and Have the Cal directory on top of it And you just have to have to run the CP-AL and Cal Shell into it and after working you can just RML-RF the directory, which is pretty simple And then there was this craziest idea about QMBuilder which When I woke up one morning in the new years this year I just wanted to have a QMBuilder Because there was this cool thing called KVM Which is exciting most of the kernel developers, so I wanted to Implementation P-Builder which runs on KVM How many people know about KVM as in K virtual machine, okay, so Well QMBuilder is essentially Command line tool to run P-Builder on KVM virtual machines It will create a KVM virtual machine or QMV virtual machine and Update DBM packages into build packages inside that virtual machine and as a side effect You can because QM supports multiple architectures You can run build packages for other architectures Which is an interesting side effect However QMBuilder is not that easy to set up because you need a kernel To set up QMU. This is an example of a QMBuilder configuration You need to give it a kernel image and an RDA image So you need to get that Unfortunately for MIPS There's a DI image which works So I can use the DI image and also for MIPSL I can use a DI image For other architectures, I cannot really use the DI image, so I'm Sort of wondering how I can Find a kernel image in it already suitable for QMU Because it's not currently readily available. We have the kernel images But we don't have the RDA images available in the Debian archive Is that correct? I think there are some team members of the kernel team But I think that's the situation So what QMBuilder does is that it creates a Base QCAL image, which is a raw device file, a raw file of The QMRowDeviceFile format, which is just a file Which can be loop mounted and I format it at x3 and loop mounted build Directly and I just run the bootstrap with a cross option What this cross option does is that I can boot into the file system and Run the bootstrap second stage to set up the rest of the system So I can run the bootstrap to obtain the root file system of that architecture Without being that architecture, being able to run dbg in that architecture So I just extract the files into that directory into this device Then as a second stage QMBuilder, I have an init script which tries to find an HDB device file and run script from it. So I can just stuff a new file system new loopback file system which has all the commands and scripts and Dc files in it so that I can script easily. So this is the architecture of QMBuilder on kind of works How many people actually run QMBuilder and go working? Okay, nobody because it's very difficult and manual page is lacking The next very upload should fix most of the problems When you run QMBuilder, you just invoke it like QMBuilder create and a config makes that config that kind of stuff And I have a wiki page To document all this stuff and for update you just you can build an update config MIPS.config Simple then to build a package you just you can build a build on config and mix with something desk file The only problem is that Because I use loopback mounted file systems I need to be root to do all this QMU shouldn't require root privilege, but some parts of it The current implementation QMBuilder requires root privilege, which is not too good I was hoping to get it fixed And another thing is that because it uses the QMU system emulation It's not that great in performance I've been told that QMU user emulation might be a little bit faster than the system emulation because there's overhead in MMU emulation However, I have not yet explored very deeply into that. I Have done some exploration and there are some blockers. So we will need some playing around with it Then you have the QMBuilder who up on working so you can just do QMBuilder login Unlogging to prompt which might say just So you can have a shell prompt With the QMBuilder create and QMBuilder login and you have a prompt which says uname-a is a MIPS Which sounds pretty cool. How many people think it's cool to have a MIPS shell prompt at your fingertips Okay, we're geeks and there are other ideas about build that Because you're running PBuilder and You're going to upload it to debut archive You want to test it? right