 Good afternoon, everyone. Welcome. Here we can see that the three drivers are clearly superior to the closed one because the Apple one which I was using didn't work because it apparently wants to use HGCP so that you cannot take copies of my slides. Thank you. Mr. Apple person, whoever chose to do that. Anyway, my name is Jussi and I'm here to talk about Mezen and with it the changing Linux build landscape. Just to get a view of the audience, I'm gonna ask you who you've been using it before and if you can raise your hand. So who are you are already using Mezen for some projects? Okay, it's about one four. How many of us are not using it but would like to? It's about four, maybe fifth. How many are not using it and don't plan to in the future, near future? Three people and how many of you don't really have a good grasp of what Mezen is? It's about about 50%. Okay. So Mezen is a build system and it's built on like a few core principles which are listed here and the main thing is that it's a build system that should support all languages. C and C++ are what we focus on because they're used the most but we want to support everything like we have support for Java and C sharp and Rust and D and a bunch of other ones. And we want to be a multi-platform and that means supporting Windows and OS 10, Android, iOS, all of those things. Some of these things are quite tricky like Windows has its very own special kind of worms and it has very different limitations to what you would have on a Unix-based machine and this is one of the reasons why build systems usually work only on Windows or not all on Windows. And the scalability that we have is that it must scale to tens of thousands of files. So this is roughly the size of what Chromium is which is the biggest development project that we knew of at the time which people are constantly building on their own machines. It should scale to the sizes of what you have at Facebook or Google or something where you have hundreds of thousands of files but we haven't really actually tested it but there's nothing in it that should be preventing things from working. Another thing is that it should work for all layers in the stack so you should be able to compile applications all the way down to something like libc if that's what you want and to do this it has to have minimal dependencies so in this particular case we depend on Python 3 and only Python 3 and only stuff that's in the standard library. If there's anything that's not in the standard library we are not allowed to use it. We have to implement it from scratch and the point of build system in this particular case is to do all the tedious and boring stuff so that you don't have to. At the very core is the language that you use to define your build. This is what you as a developer spend most of your time with and this is a DSL that we designed for this use case specifically and it's not Turing complete and this is by design and it will never be Turing complete. If someone files a merge proposal adding Turing completeness like functions or something it will get rejected and there's a very simple reason for this is that if you have programmers and you give them a chance they will start programming. I spoke to a person who works for a very large commercial company and they're using scones to do their very very very large and complicated build and then I asked him so in this thing that you do does your build system spawn threads and he was silent for a while and said yes and this is not a thing that you as a developer should really be doing if you need to spawn threads so that the build system can work your thing and build your code then the build system has failed this is not there's many places in the stack where you should really be clever and use all of your skills and all the tool and things the build system is not one of them it's just like make stuff happen and don't worry about it anymore. There are also some other things which are declarative and we looked into this but then we specifically chose not to and the reason for this is that if you have something that's filler declarative then debugging it is really difficult because you have magic bundles of states and then something doesn't work and it's really hard to debug but if you if you have something like which is imperative then it's a lot easier to work with here's a simple example kind of like a hello world where we build an application and sadly the transitions went away because we had to export this to PDF but if you go from the top there's the project which is given a name and it's using C and and we'd say that by default it's C99 and then there's the next line which is dependency glib 2.0 so what this does is that it looks up in the system so is there a glib 2 available is using package config for this and then returns and if if glib 2 is not available then this will just error out and then we will executable which consists of one file and and then you assign the dependency and then you have one test which is just to run this executable and and that's pretty much it and if it returns zero then it's a pass and this there's four declarations and this is hopefully fairly readable and and even if you've never never seen this before it should be fairly clear what each one of these does and because readability is the thing that we focused on the most because that's what you're going to be doing most of the time is you're going to read what this thing is actually doing and further so taking some cues from the functional programming people did okay okay so there's objects in the system but they're immutable so you can create them with some specific settings and then they don't change after that and this allows you to actually reason about things without having to know the entire thing the spooky action at distance if you done stuff like using make and there's late bindings and you have strings and then they get evaluated later but someone might have overridden the thing and then it's broken and you don't really know why and we wanted to like make those things not possible and and if there's something that's that's strange just error out and let the user handle it and rather than try to do the the clever and usually the wrong thing and and the goal really is to be out of the user's way as much as possible the the best possible build system would be one where you don't actually even know that exists you just write files and then executables magically appear but we're not there yet unfortunately so here's here's like the one of the main reasons why why people are starting using it so this is the build logs for system d as done by debian and version 234 the they change from using the old one to meson and this is the meeps build log and the build time went from one hour and 49 minutes to 49 minutes so more than 50 percent of the time was wasted wasted in the inefficiencies of the build system and which we could now get rid of so system d is already a user then d streamer d streamer perhaps the biggest user at the moment so all their plugins all their core infrastructure and they were one of the first first people to start seriously using it their specific use case was that they wanted to support windows so that they can sell sell different kinds of service because people wanted to run g streamer on windows with visual studio and using the existing order tools build was making it really difficult and many other known projects are starting to use it there's builder there's gtk there's glib and a bunch of other ones x.org is using it i'm not sure if it's a default yet but i think that's that's what they're planning to do there is port for weyland i'm not sure if it's landed yet and then there's a conversion ongoing to converts mesa which is the the 3d graphics library they currently have free build systems and they're trying to get rid of at least two of them and there are some close project every now and then our isc channel people come and it's like hey we're doing this sort of secret close source project and and we have these sorts of problems and and so on and and then apparently it's being used unfortunately i don't know if any like specific examples of those so when you do the transition and these are the the comments people usually get tell us later is that you get a big reduction in lines of build definitions like 50 percent or possibly even more an increased developer happiness it's a very big thing people actually like like their tool and you can tell like immediately when you talk to people is that this is actually a thing that improves the quality of their life which is some it's a weird thing to be able to do with just piece of software and then things you can do is that if many build times go down you can start doing a lot more stuff as an example you can run your ci tests on platforms that weren't aren't as fast or you can run different configurations that you yet you previously couldn't do and then there's the windows support and and other other close platforms which become possible to support and we also provide out of the box support for things like address sanitizer and memory sanitizer of all sorts of those sorts of things where you don't have to think about how the compiler flags actually need to be you can just toggle an option saying I want address sanitizer and then just we take care of all that for you this is code from Richard Hughes that's a bit cut off from there who's an all-round cool guy I had to do a bug fix release on an old version of project he was maintaining that was still using the old build system how could I have tolerated things being that terrible and this is very very common thing which which people say it's kind of nice to hear so some stuff about this oncoming we don't really have that many like major things that we're working on is a steady stream of improving things as they come on but one of the things which we have been working on which been asked several times is support for FPGA so currently we have this branch experimental branch with which you can take your firmware code and you compile it with the soft core core compiler cross compiler and then you get a blob and then you compile that blob into very log and then you compile that blob into an actual hardware description and then you can flash that into an FPGA and it works it's it's not like the well smoothed experience but but it's kind of nice the main thing that makes this very difficult is that the the tool chains for very long and vhdl do things like you know crazily different way like if you compile c it's pretty much the same like visual studio has completely different flags but the basic idea is the same but for these sorts of things like it's a complete madness how like every single tool change does everything differently so it's kind of hard to think of a like one one simple way of describing all of this so if there are people here who have FPGA experience and want to want to help with that please please can you touch with us it'll be kind of nice to get this going so you could create your free hardware with the fully free toolchain the other thing that we're working on is dependency management and since I have you here now I'm going to do a bit of a feasibility study on a potential feature that we might ask at so the problem that we find is that if you look at the build definitions that that people write then you have something like this is that you have a dependency and then depending on whether it's found or not there's a bug it should say optional there but if it's found then you create an executable or library or something and then you have to test again if it was found to build the actual test and on its own it's not really such a bad thing but if you have many of these these toggle options then it gets gets a bit tedious so we thought of ways of making simple there's been there's been lots of discussion on how to get how to get this working and and so the thing we're kind of thinking about is having something like this is that in medicine you can have options which are just like random options that you can have and you could specify as a dependency that an option is some this kind of of that name and from the outside you can specify that the value of that option is either required it's optional or disabled and the interesting thing is that if you choose disabled then what happens is that this system that this function call will return an object that's called the disabler and what it does is that you can ask for it is found and will return false and that's the only operation that you can do but the thing is that if you then pass this into any function it will immediately it will not evaluate that function at all it will just return the same disabled object and the thing is that if you have one dependency that has this is disabler you put it inside any a function called to create an object you get another object which is also disabled so all the way through the chain it is not all of the things that would depend on this just go away and and if you if you're familiar with not the number concept in floating point math is pretty much the same so then your build definition file would look something like this is you have an option that's called patented and then you could say whether you you have the patent license to use this specific one and if you say patented things are disabled then this all of this becomes a no-op the only thing that remains is the project call at the top and this would be kind of a simplification of things so how many of you would would find this sort of option oh we see it's already a hand if you would find this useful okay that's okay this is a good good show of hands that's good to know oh so we might might be adding something like this in the future so if you have any ideas of how to make this it's nice a piece let's know it would be nice to get get this so we can get things simpler so now everything is is fine and we have have our new build system and and things are fine fast and simple and everything is fine well since we are in Germany let's do a thought experiment one of the things which is changing currently in the Linux ecosystem is that there's all of new programming languages which have been coming up so let's create one of our own for as a like a thought experiment that's called Dost which is completely random and which has all sorts of cool features is notably it compiles a lot slower than C doesn't have a stable ABI it doesn't support package config and short nor shared libraries and everything that you do is that you always compile everything from source all of your dependencies all at once and then you create a one statically linked object and inside this thing there's it has its own dependency downloader resolver and all that stuff and all of your dependencies are inside your tarball that you ship and they're compiled possibly with your own compiler flags which are not the same as for everyone else using it okay so what if this thing exists so the static linking is a thing that's that I saw some some very disappointed faces in the audience when I mentioned static linking them but static linking does have its uses so if you're doing bare metal development then static linking is mandatory because that's just how you how things work if you are working on something like it like would be called an app so like an iphone app or a flatback app which is its own unique bundle thing then static linking is kind of useful because then everything goes into one and it's kind of nice there are lots of talks about containers in this conference so if you have a container with one static link executable to do just one thing then it's really kind of nice because then if someone compromises that there's no shell they can escape and they can't do all these nasty things and there's a bunch of other reasons why you would want to do that however so what happens to distros so I'm going to ask my assistant here to wave her magical paw there you go and now we have completely rewritten glib and all of the packages that depend on it in the DOS programming language and then we do a glib update now what happens um existing glib compiles on a regular machine in about a minute so this is not an exact number but it's like a rough rough estimate of how long it takes to compile so what sort of slowdowns do we get so first of all you get two x slowdown because in these languages that we have the new and our copying ones you have to compile your source code twice you compile it once to get the executables or the libraries and so on then you have to compile them second time with different flags so that the unit tests that are embedded inside of it get generated so that's two times two x slowdown then about 10 times slower is the compilation speed and there are 2906 direct dependencies uh dependents of glib in debian using the the numbers that are used so what does this mean this is a slowdown factor of about 60 thousand uh which it comes to about 40 days of cpu computers running uh this is not just cpu time it's actual wall time um so it's a bit of a like a jump from one minute to 40 days but like okay if you have have lots of hardware and and all that sort of things it's doable um but what about if you don't have have mips in your thing so mips is usually one or two cores and and laptop machines have four or eight so it's about four times longer and then the cpus are about 10 times slower um so this is where the the broken transitions unfold oh okay so the white text at the bottom actually says that glib update takes four years and 140 days um and as you can probably guess from the text this this this is not actually true um the glib that's the four years and 140 days is not the time it takes to do a glib update it's the amount of time that it just takes to recompile glib for every single one of your dependencies that use it because you can't use the system one because it might be special and even this is incorrect because you don't compile just that library that use glib you have to do the transitive closure of all the dependencies because if there is no stable api then you can't rely on it and you have to rebuild all of the things that depend on it and again the lack of traces uh is going to be unfortunate but by i did the math uh i don't the package information there was about 28 000 something packages in debbie and unstable at the time of counting anyone in the urans wants to guess how big is the transitive closure dependency closure of glib i hear 12 000 anyone else so 28 000 this is about the amount of source packages 26 okay 26 840 9 uh is the number of source packages that are in and once the thing detects okay okay so so i'll just continue because this is short of that so there are 26 800 source packages and the transitive closure of glib is 26 500 so there are about 300 packages that don't depend on build depend on glib either directly or indirectly and like the one of the the only major piece of infrastructure that is in there is go okay so but this problem is not really limited to just the new programming languages so this year at cppcon there was a presentation by google people about google app sale which is their low-level library which has it has all sorts of cool stuff and and it's the presentation was called live at head and the idea is that instead of having releases what you should do is that you can just live on the head of all of your dependencies so that's not the exact description but it's like as a rough estimate and at 120 minutes into the video and you can check this out if you don't yourself if you don't believe me there's a question about what about build times and then tides windows holding presentation says buy another cpu also he advocates the use of their competitors products which is a bold choice very good and a bit later he says that build times are annoying but they're also a solved technical problem it is one of resources is it if you look at this presentation there's a presentation at flutter flutter release a strange loop 2017 and in the youtube video over at five minutes and six seconds there's a person presenting saying we have spoken to google developers who have tens of minutes dev cycles waiting for x code and eric seidel also from google so even if you have spare data centers to do your builds in it sometimes you just can't use them it's okay so eric seidel saying here that had tens of minutes of dev cycles this makes me kind of wonder that that one of these two people didn't get the memo and i'm not sure which one it was anyway but if we were to do this so let's let's say what we do okay static linking that depending embedding all the dependencies do all of that what would it actually mean well if you want to participate in linux userland development you would require having a spare data center because that's how much computing capacity you need to do stuff and as an example there's the reproducible print is reproducible builds which is a great thing and hopefully they'll get there we'll get the next version of of debut and fully reproducible it was like not done with a data center there was like just a few people it's like hey we should do this and and with the machines and resources that they had they were able to do that because you could do each bit in isolation and this this becomes harder or impossible if everything is embedding everything the then you get the the resource things that the carbon footprint of development this show goes by a factor of one million is this something that's totally fine is acceptable any platform that's not x86 power pc or 390 can only be cross compiled because the cpu's of that platform are not fast enough to do the entire thing if you even if you have the entire production for one year of all the fab fab machine plants in the world so these are only only cross-compilatable is this something that's okay is it acceptable any platform that's not x86 is not actually supported because most of these new programming languages only support x86 and arm and then when you ask them about it they say well i'm sure we can put resources in this in the future when this is an it's an issue and this is for in planning is what this means is we don't care and it's not supported but at least we have the dependency problem solved right no so since if we're going continuing with this this completely hypothetical things let's take a hypothetical product project that's currently written in c and and people who have been converting it into dust it has 15 build dependencies in debian if you if you check for that and if you take the partially to dust convert version it has 65 new source dependencies which none of which are in any distro and if you look at the log file that's in there the hypothetical of coil in that's in there you find that there are some dependency which are there multiple times with different minor versions and some are links to random people's github repose with questionable license declarations with light or and stuff like that and what this means is that the future of linux user and dependent management is no j s this is what it's it's heading towards and then you might go to a conference where there are people presenting these sorts of things and advocating these new programming languages and then they would tell you that well we're just going to do this and the distros will just have to deal with it okay so the obvious question follow-up question is is can this trend be stopped and this is a question to which no one has an answer but do we actually want to do this after all getting dependencies easily from the internet with very little fuss is a good thing that's what actually what we want to have especially if you have a multi-platform project and you need to build on something that's not to be an or federa you need to have a way to get dependencies if you are developing like as an example gnome developers use ch build to get all of the new dependencies because they're not in the distro simply because because of the reason that they are not being released yet and this is the thing that you need to have so in meson we have a some sort of a solution for this or a proposal for one is that the build definition for your project should be identical regardless of where the dependencies come from so this is this is kind of what it looks like which you can use today so in the the middle line which has the z deep dip and they have dependency on zlib and then it says fall back and this is the nomenclature that we have is that you should look up zlib on the system and if it's available in there just use that one and if it's not available then get it from this sub project called zlib and and the zlib depth is the internal dependency that's defined inside of that dependence and then you have your executable which uses zlib and it just says that it uses the zlib and it it doesn't matter where it comes from but the build definition is the same and you can just get it built like that it can come from the system again the package config you can ship pre-built library package sorry you can ship pre-built libraries as a as any like a sub project that you can just put in there and use it can be built from source transparently during the thing and whatever mechanism gets invented in the future for delivering dependencies we just add support for that and things work behind the scenes and so and then it gets even trickier because if you have one language like if you have an entire project in C and multiple ones it's very simple but if you have many different languages in the same project then things get really really difficult and the future is always hard to predict but what will probably happen based on current knowledge of things is that there's going to be different components like shared libraries or something like that that's going to be communicating with each other over a CAPI but it's they're going to be implemented in various different other languages so you can have a dependency where you have something in C then your dependency on everything that's written in C++ and then you have something in D and then you have C++ again and then you have Rust and so on but the thing is that if you see API is the thing that's that's actually stable and you can you can talk between different languages so this seems like a fairly probable reason of how things are going to go and then if you have a project that looks like this it should be buildable using one dependency resolver and one build tool you can have possibilities different build tools but like if you have more than one dependency resolver inside one project don't it's it's it's not going to go down well as an added bonus there are some languages which are actively hostile towards using a build tool that's not their own and this is unfortunate so some sort of cooperation mechanism needs to be invented here just so that all of these different languages can work together and all the projects can work together I don't know what have a really good solution on what that should be but there really should be some sort of discussion on how this on the best way of getting this done so in depressing conclusions well first of all the the process of building software has gotten a lot faster it's gotten a lot more pleasant and it's just not me saying that because I'm the the guy who wrote the thing but actually people other people who are using it and they've been very happy on the other hand there is going to be some sort of dependency disruption on the horizon and we don't really have a good grasp on it and and something's going to give because the the approach that the modern languages are using don't really fit well with the way distros work today and so so something's got to give what's what's the thing that's going to happen we don't know but hopefully we can work on something that's actually a good thing that's a good solution for everyone rather than just everyone crying in their own corner and complaining that other people still want toys that's that's the end of it and if there are any questions Martin will bring you the mic mison sounds very similar to basal apart from one being written in python and the other in java what are the major differences okay so the the major differences are as you say that it's it's written in java so if you have a package that you want to include in the bootstrap set of an operating system you can't really have a java dependency on it because it's it just blows up we get comments every now and then that having a python dependency is too big for many people so if that's the case java is just like it's just a no go another thing is that it's very optimized towards the way the google does things so if your use cases are very similar to those their ones then okay it works pretty well but it's it's again it's optimized for the static linking cases that they have and they don't like all the things that you need to support like installation installing into unix directories and lots of stuff i haven't looked too deeply at how that how that works it should be possible but i don't know if it's it's as smooth as what we have and also their windows support was traditionally non-existent but they have been looked putting a lot of effort into that but i don't know what the what the status of that is they now have default targets for the bn packages rpm and torbols so they can spit them out directly okay let's go i wanted to ask if you could expand a bit on how the example you made on declarative making it hard to debug you're right wasn't very clear to me so if you could expand on that okay so why are declarative things hard to debug so how many people here have been doing qml okay a few so qml is the the declarative thing in qt where you can define your your user interface using a declarative approach and if you use that when it works it's absolutely awesome you just like tell stuff and everything magically works and then every now and then you get a bug where your window is slightly off and you can't tell what's happening because it the the positioning is not done by any of the code that you write it it's done somewhere inside a black box and and like okay now i have to debug this somehow and it gets very difficult but if you you can because you can't do printf debugging because there is no sequence of steps that you could take where you could say okay now in this case this is what things should be and you can't test for that so this is a this is a tooling problem so if you have a really good tooling debugging story for your declarative thing then it's it's it could probably be a lot better but we don't have the resources to to create both a debugging system for this which is it's it's not very common to have very good tooling for these sorts of things and to also write a build system so so we decided to do do this thing and i have another question i was wondering if you had looked into nick's and if so what your opinion on it was and so you nick's os i haven't looked into it too much but i know that there are people building mess and file projects on it and it works so so that's that correct me if i'm wrong but nick's does the thing where it installs packages into their own locations right yeah okay so we so we support that so if you want to lay out your output in a different way we totally support that and all that should work as far as as how it so from our perspective as build system developers and it's not really that much of a difference it's just their choice on how they want to lay out their files as far as the deeper dependency resolution thing go i haven't looked into it that much okay so i'm told there's one minute oh no sorry sorry okay thank you