 Good morning everyone for the last station of this morning. We'll have Philip Kaluza with we will talk to us about Bootstrapping another IK texture using LLVM IR Please welcome him. Yes. Hello So in the last session Sylvester told us about Yeah, mass building the devian archive using Clang standard C compiler for the LLVM project and I want to talk to you all about Yeah, what I'm planning to do on top of this work and taking it in a bit of a new direction This is very experimental and much more work in progress even than the rebuilt with Clang work, so Bear with me. I I'll introduce what I've been doing and Hopefully this will be interesting to some more people than just me Yeah, so I'll structure the talk Talking again a little bit about LLVM for those that I did not follow the last session Maybe in the stream What I'm planning to do with it And what this would give us and what problems we will have I will talk about the pinnacle project That's been done by the Chromium developers I will show a little bit what what my experimentation has yielded and In the end I invite you all to discuss with me If this is interesting and if we should take us in new direction Just a quick show of hands who here has had experience with building stuff with Clang Linking with the LLVM linkers Okay So I'll try to introduce the concept Since my last step conflict talk was a DC7 allow me to reintroduce myself a little bit My name is Philip Kalutza. I did study at the technical University of Brunswick in Germany Since 2010 I finished there and Teaching there still a bit courses about system administration Of course promoting Debbie and doing that Since 2011 I've been freelancing. I'm now based in Berlin. I'm just yeah general IT consulting work So what is LLVM? LLVM stands for low-level virtual machine, which is not really anymore It's they started out with yeah, just describing an abstract register based machine for Having some some code within a compiler tool chain that is Above the the emitters for certain hardware architecture and but Quite below the SSA trees in the compiler So a number of medium and low-level optimization steps you would want to do in a compiler tool chain You can do very well on the LLVM bitcode or the Compiler internal binary representation of that It will evolve from there to become infrastructure for building compilers optimizers static analyzers you can emit C code again beautified and And Yeah, the many research projects are taking this in very many interesting directions at the moment The kind of assembly language for This very virtual hardware architecture is called IR immediate representation if you Assemble this assembly language into Some binary code they call it bitcode because it's not quite aligned and Basically Except for the compiler itself or the lower parts in the tool chain can read Either both formats or at least the bitcode format Again and do their optimizations transformations and so on in there IR and bitcode are roughly Mappable one-to-one mappable so For a certain IR you can actually generate a number of different bitcode files But you can always get go back to the same IR Sounds formatting That you had before doing that Yeah, the LLVM tool chain As I said contains emitters for Different target target architectures MD-64 x86 for 32 bits arm Those are the ones I played with there's quite a number more So of course there's just in time compiler for that and also an interpreter which Can just run through the bitcode interpreter wise If you use the LLI so LLVM interpreter binary to run something Normally if it's not on a very obscure architecture, it will try to to run through the JIT engine and so I'm interested in using this whole system and Taking it into the direction Yeah Can we use this for code distribution? Let me Okay, before I wanted to show the compiler pipeline Again quickly, it's pretty clear. So we have preprocessors that's included in the Clang binary Compiler you can also GCC style Call out to different preprocessors compilers if you like to both of them Will if you if configured correctly output bitcode and The whole tool chain can call out to different linkers They've kind of standardized now on using gold the shipping and plug-in for gold that can read the LLVM bitcode And if you link together a number of object files using this This gold plug-in it can actually do the first number of Link time optimizations there. So link time inlining is one of the big topics, of course that Will be very interesting to do there They also have a LLVM link Which just links together bitcodes With with other bitcode outputs bitcode again LLVM LD which is a bit of a hybrid can can run in both modes Yeah, so and so we can we can output ELFs dot SO files or something or a bitcode Which we which are also not named dot SO in the tool chain really quickly The LLVM compiler and the Tool chain so the Clang compiler and LLVM tool chain Have BSD star licensing Apple didn't like the GPL anymore at some point The GCC also said was very skeptical if Making the whole core of the compiler more modular Would lead to people circumventing the GPL there so They for a long time. They said no no plugins for the GCC tool chain Reluctantly they're opening up after they found a nice legal hack on how to ensure that most GCC plugins will still be GPL and so In in GCC something is slowly changing and it's becoming more modular, but only after big players Especially Apple said well, let's Go with this new Interesting research project LLVM Back that and we have BSD licensing licensing and we can do whatever we want with it. So at the moment, I don't care about the Politics here one way or the other but it's important if we're talking about LLVM to know historically that this was a factor in Why this ecosystem system exists and for some Debbie and developers I imagine it will be important in Thinking if they want to support the whole thing So what I want to do I Want to use the compiler tool chain let it output LLVM bitcode and Ship these in Debbie and packages Yeah, so we would have Debbie and packages for a new architecture That targets this Yeah abstract low-level virtual machine and Then we could do things with this on the target machine LLVM bitcode in itself though is not a hundred percent Architecture independent I will come to that in a bit, but as far as possible I would like to Yeah, target generic subset and be architecture independent in in what we are creating there How do I go about this First it's possible to register bitcode itself as an executable format with the linux kernel That breaks in very interesting ways once you're inside a chrude or chrude or Other build system build infrastructure But in general this is possible which means that the old cross-compile problem of One of these binaries is supposed to run During my package build and do something useful and the other of these binaries is supposed to be generated for the target target architecture and my whole Automate setup is not really set up to do something sensible with this or Differentiate between the two of them this problem goes away if we say well, we have bitcode we can Execute that so this is a problem that we should solve but Would mean changing every debut source package there is and making them all really clean cross-compile clean and I'm a bit skeptical on the prospects of that, so I worked around that Yeah, the packages we are getting out of there we can look at also a bit like arch all packages for Java bytecode or the CLI so Don't that bytecode monabyte code We ship we generate them also on one machine and ship this this bytecode in devian packages and We already know and assume on the target machine We need something that helps us execute that Excuse me But yeah, yes, yeah the the the code is Cal and the Common language infrastructure is the CLI But I want to generate this from existing source packages. I want to use existing see later C++ code not change it as far as possible and Basically, we're just targeting a different is a instruction set architecture that are very Different but not as different as these stack based Systems are like Java and CLI So what does this give us? So first I'm a developer. I like to fiddle with my development environment and I like to to get this Yeah Into a point where I can do new and interesting things with it and That's certainly It's the case it has been has proven very interesting Also, but annoying We would get a large repository of software that we have compiled under very defined conditions that we can say This software we have Indebian as LLVM bitcode. We know that this bitcode comes from a clean built the and and Should do what it does. Well On the target machine not on the target architecture, we can start Optimizing there we can decide on what functions we want to inline Based on how much cash pressure we expect. Yeah, so Even if I target and the 64 an atom and Core i7 have vastly different properties there and It is interesting to Yeah, try optimizing this Really when we know what machine it's going to run on we could Use this to try to bring up new ports faster. So if there's new Architecture like arm 64 being created The LLVM guys usually are there pretty fast and so once we have the Linker and LLVM interpreter just in time range and so on running there We could have a bunch of our toolchain available on the new ports and Of course something like the Raspberry Pi port Need and have happened in this way if we had a toolchain where we Do very late post installation optimizations LLVM has been looked at for GPGPU computing I'm not really too deep into that topic, but if we can reuse parts of our toolchain there That might be interesting Yeah, I talked about the late inlining already LLVM is not really well suited to doing all of that so Yeah, it sucks a little bit and I'll try to explain why it's not Aimed to be a code distribution mechanism originally We until now there's no compatibility guarantees in the LLVM bitcode That has been Pound also to be a problem in the LLVM community. I'm not sure How much better it's getting the compiler? Assumes a certain target architecture before it outputs the bitcode we will need to work with the compiler To make less assumptions The bitcode includes a lot of details about calling conventions and Liga details So the The things is that that the whole toolchain should work against linking parts of a Microsoft VisualSeq project so all these options for Linking details and so on are included in that bitcode the LLVM bitcode allows for uninitialized variables to be present there Which you wouldn't do in Java or that net bitcode bite code But they said well see so it's undefined we want to be able to carry this This is undefined state around with us So let's put this in the syntax So in a way the LLVM bitcode is a bit too low-level to do that and Of course as a caveat The old Java adage of write once debug everywhere Could apply here as well Yeah, the front-end compiler Generates just quite a lot of details in knowledge of What linker would link this together and What was the target architecture to run this so actually we want We will have two linking phases one during the compiler for them in package one on the target machine and For the first first linking face. We want to make the linker a bit Dumber Other things that might happen Just linking together LLVM bitcode is more like a static linking Until now we don't have tracking of this library Will be linked in later. You can just leave some symbols open while linking but the information of Well on the target machine Link in this thought as a whole please We don't have so This is something we can work around but if we want to make it really clean so that We don't have just a lot of static links lying around on the target machine later And the target machine can can simply Apply security updates. This might be addressed very detailed might need to be addressed very very detailed on our tool track Early preload is not a thing at the moment. It is however a very interesting To work with LLVM IR. There's just lots and lots of papers coming out in the last years Just working in this modular ecosystem playing with it taking the tool chain in new directions And I really want to get in on that for Debian. It is possible to define a LLVM bitcode a bi that's architecture agnostic where the bitcode doesn't really Include so many details about the target machine that this will run on the chromium project has done that They have the now NACL maybe you've read about that native client system Where they built a really nice sandbox on for running natively? Compile C or C++ code That targets This sandbox so you cannot do system calls or a very limited number of them you cannot exec or link in arbitrary arbitrary code from the target machine and Then they know as well this actually really annoying for our developers if they have to compile everything three times Into 32 bit into 64 bit and I'm 32 bit. That's one say I'm supporting at the moment so they started to Think about also shipping LLVM bitcode just targeting this architecture and The compile if I understood the paper correctly actually is happening within the sandbox, which is interesting And so they also went ahead and defined of just a number of Constraints on the bitcode so that they could say okay, we're confident that this bitcode will really be able to to run on different machines Just from the document. They have and they're saying well Yeah, the in our pinnacle machine the address space is 32 bits Addresses are always 32 bits wide They're the integer but bite order Low-end and I think these are reasonable assumptions To make about your target system. Yeah, it does mean once you start making these assumptions that you cannot just link in Lip GTK the native version on the target machine. Yeah, this this needs to be clear to us if we're doing this work If we want to link against 42 bit MD 64 lip GTK We need to tell our compiler to target that if we say well, we want to link In on the target machine other LLVM bitcode Which also falls within the set of constraints. That's very possible to do that. So I see this Project within Debbie and Possibly if it goes anywhere at all going in one of two directions The first I've dubbed architecture LLVM Which means we assume that we run let's for now say on a Linux kernel We have system calls that we can call we generate LLVM bitcode with an API to be defined Like I just showed on the pinnacle slide similar to that but not assuming we run in the pinnacle sandbox so What would be the point of doing that well the universal OS as they been as we like to call them in Has just a whole bunch of software and we would have the option of linking and optimizing that on the target That would give us the advantages for porting and so on they were talked about earlier We might constrain that project even further though I've dubbed this Possibility architecture of pinnacle and we just assume that we run in the pinnacle sandbox So we can call we can use less C code unmodified We can not do And in a limited number of system calls We might have a lip see running in there and the pinnacle guys actually have something like that That tries to solve a lot of problems with a very small set of Sus calls and that would mean that we jump onto the bandwagon of the pinnacle project and just give them Yeah, lots of Debbie and love Export lots of code that we have into that environment and for now pinnacle is just a browser sandbox, but I'm Pretty sure that some people at Google are thinking about also using that to a sandbox code on the server. I Don't have any proof of that, but I would if they I were them So if that becomes popular It would be nice really to say well then we have lots of Debbie and coach to to work with a third option would be To compile a lot of As many packages as possible and then Lindsay and Jack or something Which packages could live within the pinnacle constraints and mark them just to be complete, but I think for now The scope of this should be limited to one or the other we should find a clear way to go forward on this Okay, I wanted to Give two little demos so the one is really Some small code a little secret my course So I just copied it from from some example here just does a multiplication does a print F That's So It's cut off a little bit, but I think you can get the gist of it. Yeah, sorry So this is not very interesting code, but it does compile into Some bit code That would be the mullet dot oh and the dot s is the representation here No, that's not actually so this this is the LVM IR and I broke my demo I had a copy of this Compiled on on a Raspberry Pi and It's possible actually to to run this on We're using the LLI to run this on My input machine, so Okay, yeah, the bitcode does include the triple that this is compiled for and You have to override so this the a out BC is the one compiled on the pie and I don't know So if you want to call execute the bitcode that is That is from a different machine. You have to to Overwrite this. Okay. Oh all these tools have to M Help yeah, all these tools have nice different command line options. Was it m-target? I Think I'm out just right. I should have tried this again this morning So if I override this to Run on The PC here that would help here. Okay, this demo doesn't work I Did get something sensible out of that so but This was just to to show that this it could be compiled so I just took the LVM I Are or bitcode generated on an arm machine and Rendered here so Unfortunately, I lost my bash history a few days ago. So that's why I have to The other demo I had been hoping to show And maybe this one goes better is I Now will try to build in a shrewd and Yes I tried to build in a shrewd which gave me a lot of trouble and now I built it on this MD64 machine Hello binary and so what I did is to set up the the clang and Also, I had to do some some hot linking hacks like Sylvester before me set up clang to call a custom linker that is a wrapper around LLVM LD and I called Yeah, after a bunch of Bunch of tries I got it to run the whole automake system within the debilt and I'm not going to show that yet because it's pretty fragile And it generated me for me this nice. Hello package. So If I do this in a 32-bit install or if I set some more parameters, it will generate the hello package targeted at a 32-bit style bitcode and If we Look at user bin. Hello. This is LLVM IR bitcode now So the whole compile to chat toolchain at the moment Really with a lot of hacks, but it's starting to work that I really can get a devian package out of that So of course this is arch MD64 for now have defined the new architecture and if we Run this then we notice actually that new hello is nicely internationalized so That's the stuff I have to show for now. I've started compiling some Some Libraries, but as I said for the libraries there's still some theoretical work to be done or file format to be to define to be defined for carrying some some more information on to the target machine What would make my life easier is? We've talked about that a little bit in the past if user bin should be multi-arched I would like to have an alternatives mechanism for Yeah, at least for the toolchain The multi-compiler problem that we talked about before This is something I will be also working on next because I think this is important That not that you don't have to manually install some simulings into your your built environment It's better if we have a standard alternatives mechanism here and For me it would be interesting to install to be able to co-install a new hello from With LLVM bitcode and natively combined and AMD64 can do hello But this is just something I think we can think about in general in the demian project if this help if this happens It would help along this project a bit if it doesn't Well, then it doesn't I would like some help in general though When I started working on this it seemed like Interesting simple idea It's not simple and it's a lot of work and if somebody wants to help with this Please contact me. Let's get in touch with the other people For now we've just discussed Just using reusing the package LVM mailing list And you can get in touch with me personally and talk about this everybody's invited if you feel like your employer might be Interested in doing something with this once it's there Yeah, I'm available for contracting. So if somebody pays me I can put a lot of a lot more time into this of course Yeah, and I would like to end with a quote nice quote that I found on the LLVM mailing list About people doing weird cross architecture stuff with LLVM and I Didn't note down who said it but the quote is it isn't LLVM's fault that people want LLVM to magically solve all of C's Portability problems and I think that's important to remember. So we cannot Just do magic here, but we we can use this new and exciting tool chain into new directions and yeah, I would like to get this running and Hope this you guys will be so as excited about this as I am Are there questions? Do you think it's worth all to do this? What do you feel is it more important to do the concept arch LLVM? Is it more important to do the concept arch pinnacle first? Are there related ideas you want to talk about? Yeah, as I said we yeah as I said, we would need to target a specific subset IMAI we would define in our Forbide yeah little and you and 32 bit as we do when we target a specific 32 bit machine Yeah, one day we might decide okay. Everybody has armed 64 machines now or Intel 64 But nobody has 32 bit machines anymore Then at that point it might be okay to just Yeah, target Say, okay, we make a new archic architecture LLVM 64 and now their size off is Yeah, I think what you're trying to do is that you still can link at Against as I said earlier the native natively compiled natively running libgtk and with this concept We will not be able to do that. I totally agree for me It's not so interesting to do that for me It's more interesting to see what we can build how we can build a little LLVM universe out of all the Debbie and sauce packages Yeah But if upstream moves there and then says okay We try to make this really architecture independent then of course that would be nice and really exciting The past discussions that I've seen Do not make me hopeful Hello again Yeah, so yeah, this is a really interesting idea and we talked early already earlier this week and I did a thesis on a more compact encoding for this IR So I had to struggle with a lot of these details and I had to generate a lot of IR so that I could Get a benchmark for my compression basically and I think that It's really ambitious to get a single IR for All the being architectures and you could easily just make it easier by just targeting one architecture at a time Or you could just generate native code like a silver service ruling with Selang And that's just save the IR to the side because then you could still use that for debugging you could generate the Debug information for that You could do static analysis all sorts of stuff with that and Then one by one you could maybe start running that IR also Okay. Yeah That would be equivalent roughly to changing the compiler tool chain and Debian to use Clang and Rebuild all of them in on multiple architectures I think it really depends on the use case and and this Idea of keeping the bitcode around for for debugging is actually I added after I talked to Lindy because it's I think it's an interesting idea So you use the whole tool chain you generate a target specific binary For me at the moment it's out of scope because I feel like that would be too much doing too much at once With the resources I have but the important thing is I think that we get all the stakeholders in this talking And if we as a Debbie as a Debbie and project feel we can do that Then let's do that Thank you very much. We are out of time. So thank you