 All right, let's start. Welcome to my talk about PTX tests. I'm Roland and I work at Panagraphonics as a systems and integrator. And I have been using PTX tests on my daily basis for the last three years, I think, yes. But it's much older than that, as we'll see. So yeah, what is PTX tests? PTX tests is a built system with focus on building embedded Linux images from source. So that means if you're on PTX tests, it downloads all the sources from the internet and compiles them and builds them into a nice image which you can flash onto your devices. And under the hood, we're using proven technology which you probably already know, like Knoomake for the dependency resolution and Kconfig for having a nice user-friendly menu and then some Bash and AWK code as glue code to couple them together. We are doing monthly releases now and the first version I could find from PTX tests was from before August 2003 and then the repo was converted into SVN. So yeah, it's already quite old. But there hasn't been a talk for a while so this has to change now. PTX tests is GPL licensed as you probably already know this license, yeah. And we're also having a hot page now. Let's have a look at a typical BSP. First you have this folder structure which is quite usual. You have some configs in your configs folder. For example, the PTX config contains all the software that you want to build in your image and then we have a platform config that defines the hardware you're using and the kernel config. Yeah, this is basically the config file for the kernel and in this case, we're also having a barebox configuration which configures the boot loader which we're using in this case. Then we're having some files in the project route directory for example, an ETC FS tab and an ETC shadow file which we want to install on the target. And then we also have some custom rule files for our BSP. Yeah. And the philosophy here is that your BSP looks just like the PTXTIST install directory and you can override and extend the files from PTXTIST locally in your BSP just by copying them into your BSP and then adapting them to your needs. Okay, first look, what you will see when you're working with PTXTIST are these two menus. As I already said, we're using K config for that. On the left here, we have a menu config for the user space configuration. So there's everything in it. I don't know if you can read it, probably only the people in the virtual first line, first row can read it. Like shell and console tools, scripting languages, communication utilities, applications, editors and so on. So this is where you would select the software that you want to have on your target. And on the other hand, you have the platform configuration menu. This is everything bare metal, like configuring the Linux kernel or a device tree compiler and some console options and some bootloaders and the images that you want to build for your hardware. Right, if you're working with embedded Linux, you often have to apply patches to your packages. For example, in this case, we're applying two patches to Linux and one patch for Busybox. And as you already see, we can just drop them into these folders in your BSP and they get picked up automatically by PTXTIST during the extract stage. So in this case, the Linux patch tier is in the configs platform v7a folder. Because it is platform-specific and the Busybox patch tier is just in the patches folder so it gets supplied for every platform that you want to build, which we'll see later. Yes, you can edit the patch queue. Maybe if you want to develop on your package and you can just extract the package and then change into the package tier and use Git or Quilt to work on your package and then it will synchronize these patches in these directories. For packages, there are different types of packages. So for example, you want to have some software on the target. This would be a target package, of course, and is built on the host for the target architecture like for example, SystemD and Busybox and Corridors and whatever. So you have to use a cross compiler for that. And on the other hand, you need some host packages too which are compiled on the host and run on the host. For example, in this case, maybe you want to build a VFAT image or something needs to use Python for the build system or CMake as a build system. And in this case, we would build the respective packages for the host to be independent from the host system so we can run on, I don't know, year old Debian and send OS hosts and we don't have any problems finding the software that we need. And the third category is image packages which is basically defines the image format like EXT or VFAT as I already said, or I don't know, you can also, for example, build images for Rauch like software bundles, software update bundles. Yeah, things like that. And there's a nice wizard in PDxTest for creating new packages because I think in upstream PDxTest there are about 900 packages pre-packaged but you will probably have some software that you want to use on your target that is not package upstream in PDxTest. So you have to do this yourself and you can just use this PDxTest new package wizard to create new target package and it asks you some last questions like what is the package name you want to use? What's the version? Where can you download it from? And so on. And in this case, what is the build system? And then it generates you the right files and the rules directory. In this case, the my package.make is the make file that contains all the recipes what you want to do to build your package and then the .infile is the kconfig snippet for the menu that gets included in the menu. Let's have a look at the infile first. This is standard kconfig language. In this case, we're first defining the section for PDxTest so it knows where to include it. In this case, it's project specific but you could also have one of the categories that I was talking about that I was showing on the previous slide. For example, editors or communication utilities. Then you define your config symbol. It's usually the same name of the package in uppercase. It is a tri-state option so you can select yes, no or as a module, we'll see about that later. And then you have to select your dependencies. In this case, we need to use the cmake house tool because we're using cmake as a build system so PDxTest needs to build this one first before building my package. And then also we're selecting libusb as a dependency. Yeah, and then some prompt and help text. Then there's the mypackage.make. It starts with a list of definitions as variables. So for example, here the version, the package name where you can download it and also this mb5 sum because when you download things from the internet, you really want to make sure that they are the right things and there hasn't been any corruption. Yeah, these are mostly standard and you don't have to change them except of course for the mb5 sum and the version and you can just use them as generated by the wizard. And the make file also then afterwards contains the stages that you built your package with. Every package starts with a get stage which is not shown here because we can just use the default get stage in PDxTest and you can always let it out if you don't need it, if you use the default one but you can always define it if you want and it gets picked up by PDxTest. So in the get stage, you would tell PDxTest how to download the image from the tarball from the internet then it gets extracted in the next stage which is the extract stage which is also not shown here because it is also the default stage. Yeah, it basically extracts the tarball into the build directory of the package. Then the prepare stage is partly defined here because we have to tell PDxTest to use CMake to build our package and use these config options for CMake. So for example, you can define here CMake in this case or just also use auto tools or if you're building a Python package just use the Python build system and everything else for these build systems at least PDxTest knows how to build. And also if you don't have a prepare system you don't need a prepare stage or something you can just set this to know and then PDxTest will just do nothing. After preparing the package it is compiled. This is also not shown here because we just need to call make basically in this case. And after compiling it it gets installed in the install stage. So PDxTest under the hood installs the package into a directory just using the default install rule from the make stage from the make file from the package. But we mostly don't want to install everything on the target which the package installs. So for example, maybe the package installs documentation or example configurations or stuff you just don't need on the target. So the next stage is the target install stage in which we define first the variables for our OPPG package which gets built by PDxTest and then we just cherry pick the files that we need from the files install location and tell PDxTest to throw this into our OPPG. And this way you can just cherry pick all the files that you need on the target. Right, I think that's everything for this stage. Let's have a look at another feature which is layers. There are a lot of layer mechanisms in PDxTest and layers are really, really useful if you want to customize software. For example, in PDxTest we have alternative config files so this is just about customizing one config file or a few config files for a package without having to copy the package rules, adapting the package rules to your VSP. And in this case, you only have to copy the single config file and adapt it. Then there are platforms as I already mentioned. Platforms are about different hardware which you want to run your VSP on. Then there are software collections which defines different sets of software. For example, if you have different product lines which are quite similar but not the same, you can define a different collection for each of them and just use the one collection for the one product and the other collection for the other product. And there are K-confit diffs. Maybe if you want to have similar kernels or something, you would just need to maintain a diff between them. And then there are base layers which are basically the most pure form of layers I think and you can basically do anything you want with them. All right, let's have a look at alternative config files first. Here we have a snippet from the upstream rule for cups from ptxtist. And in the target install stage, it uses this install alternative macro to search for ETC print cap. And if ptxtist sees this macro, it searches in different locations for ETC print cap. For example, in my BSP project route, so in my own BSP first, then in the config folder project route, in the platform folder project route, then in ptxtist itself. And if it still isn't found, it will go on to the cups installer and the cups builder and search for ETC print cap. And this way it uses the first ETC print cap it has found at the first location it has found. And usually ptxtist includes a default version of these config files and you can override them in your BSP. That's easy. Let's have a look at platforms. As already said, platforms are all about different hardware. So in this case, we're having a platform Raspberry Pi and it uses the RM 1136 JFS cross compiler and a kernel 4.19 and it builds an HD image and the Raspberry Pi also needs a boot partition, which is VFET. But we don't want to use only the RPIs. We also want to use other platforms. For example, here we have a generic V7A platform for our V7A and we're using the kernel 589 in that. I think that's the current at the time of recording newest kernel. And we're also using barebox as a boot loader and also the HD image. And then we have a V8A platform for 64-bit ARM and we're using an RC kernel here and also the barebox and an HD image. So this way you can develop four different platforms at the same time using the same user line software. So for example, here systemd, busybox and cups. And you would then select the platform you want to build before building all your images. And all the configuration contains the same software but different kernels and different boot loaders. Then there are collections. This is all about different software configurations on the target. For example, if you want to have GDP and S trace and stressNG for debugging, you could have a debug collection and then you would need to set these packages to M. This is the tri-state value as I mentioned before. Then you would, in your debug collection, set them to yes. So they are built into the image and in another release collection, you would just disable them so they don't get built into the image for release. So there are different scenarios here. As I already said, you can select them before building your image. So you would just use PDXS collection to the path for the collection config and then build your image. Or the image that you build can also use the collection by using these packages variable here and then the image will use the software from the collection. And in the second scenario, every image that doesn't explicitly use a selected collection just uses the collection you selected with this PDXS command. Right, next feature, Kconfig divs. The use case here is you're having a different kernel but you want to have a similar config but not the same config. For example, you are using a debug kernel package to develop your kernel or to develop your BSP in general. And you don't want to roll out all these debug config options to your release. So you're having a different kernel package in, yeah. And then you can set in your kernel package these refconfig to the config from your release kernel and the other line here, Moscow config tells PDXS which config to use for the debug kernel itself. So then you would go and configure your debug kernel package. And in this step, PDXS already uses the refconfig from the release kernel version and predefined everything in the menu that is like in the release kernel. But if you select different options it afterwards writes the div between the release kernel config and the debug kernel config into this .div file. And the .div file in this case only contains two symbols like for example, DMA, AP debugging. And the first line is always the MD5 some of the reference config. So in this case, the release config. So you can always, PDXS always knows that when the release config has changed it needs to recalculate the debug config from the div. So this has the implication that every time you're updating the release config you also have to update the debug config but this is just another PDXS command. So you just tell PDXS, old config, kernel release, kernel debug, sorry. And yeah, it's basically one second also. But it has the advantage that you're solving any merge conflicts that would occur during the update at an earlier time. So for example, if you deferring this update to a later stage, for example, you only have a config snippet and apply then regardless on what the release config is then it could lead to merge conflicts during the build. And maybe the person that's building the BSP is not the person who is developing for it and it doesn't then someone doesn't know how to solve these conflicts and leads to confusion. So in our case here in PDXS test we're opting to solve these conflicts early. This Kconfig div mechanism is also used for base layers. And in base layers, all you need is basically this sim link named base. It can also be a Git sub-module or just a directory or something else. It just needs to be named base. And it's points or contains the base layer that you want to build on. For example, here we have finished BSP named distrokit and it contains a V7A platform with a kernel config and a platform config and a PDXS config and some rules. And we want to build on this image. For example, because we want to enhance our product line and add another product without changing the original product. In this case, you would just create a new BSP, set the base sim link in this case to distrokit and this is basically all you need. And if you now go into the menu config, then PDXS searches for the first PDXS config it finds and it will go into this base layer because it doesn't find it. If this one doesn't exist, it wouldn't find it but the base layer has a PDXS config. So it uses this one. Then when you change something in the menu, the diff is calculated in the current layer which you were currently using. So in this case, the diff is between this base layer config and this config in the current layer. And for example, if your current layer also contains another package, then PDXS will find it when you're using this layer. But it will also find any rules that you have in the base layer. So there are some caveats. As we already saw in the previous section, this .diff kconfig diff contains the mv5 sum and I already explained this. So yeah, they are pinned down via mv5 and you can run old config to synchronize these diffs and it has the implication that you solve the update conflict early. And PDXS is always used as the lowest layer implicitly. So for example, this district layer doesn't have a base link but in this case, PDXS would just use its own install directory as the base layer. Okay, there are some more features in PDXS which are not about layering, but which are useful nevertheless. For example, we have code signing infrastructure. And the use case here is a verify boot or high assurance boot. For example, the iMix 6 processors or I think 8 and 7 also have this feature that contain some code in the ROM which you can use to pin down some fingerprints or certificates to sign your boot loader. So every time the ROM coder is loading the boot loader then it would first verify this boot loader. So if someone else wants to verify this boot loader then the verification would, the signature would not be the same and the ROM coder would not load the boot loader. And then in the next step, the boot loader loads the kernel fit image and verifies a signature and then the kernel would use the signed file system. And on this step, maybe you can use, there are certain applications like Ubifest authentication or EMAE4M or something. But the important part with German distressee is that it is a chain of signatures. So everything from the ROM code on, and the ROM code you can change and from the ROM code on, then everything is verified. And the other use case is you want to sign your update bundles. For example, I think Raak can do this if you want to use it. So you can download your update bundles from the web and make sure that they really contain the right software released by the right person. And maybe you want to also have some key set for development and some key set for the release phase. So you can configure your hardware in the development phase to only trust your development keys or your development signatures. And you want to make sure that the development images are not used for the release phase. And for example, in the development phase, you can also use some example, something like a soft HSM, which is easy to use. And in the release phase, you can then use maybe a hardware security model, a really hardware security model or a cloud signature service. And PDX tests splits this in two steps. We have providers and consumers. And the providers are really about talking to the HSM. And yeah, I don't know. In the case of soft HSM, select the right keys from the right locations in your BSP or in the case of NitroKey prompt you to insert a NitroKey, yeah, something like this. And you can select different providers in the platform config menu. For example, you have a provider for your development keys and a provider for your release keys. And we also have a package result for doing your own code signing provider. And on the other hand, we have these consumers below here. And yeah, it's basically, they use the abstraction in PDX test to select the correct keys or select the right key material to sign their images or their images basically. Recent PDX tests also have some nice features to get to know your BSP and debug it. For example, we have a BSP info command which lists everything you need to know about your BSP where it is located, where the PDX test is located, which platform config, which PDX config you're using, which images you're building and so on. And we also have the same for single packages. For example, here busybox is at version 131.1. It has a GPL license. The source is located here and it's download URL and where the patch folder is located and the rule file and the menu file. And you can use this to debug. The internals of PDX test, if you want, or if you're not sure on which layer the package comes from, you can find this out. Ah, yeah. You already saw these license variables here and I'll quickly talk about this. Every package can have these under score license variable and under score license under the files. In this case, we are setting them to zero BSD and this really means that you're defining the project license for the package itself. So PDX just knows that this package is available under the zero BSD license. And it also knows that the license is contained in the folder license in the packages build directory and the MB5 sum of this file is the following. So we are also pinning down the MB5 sum here. So we get notified by PDX test if this license changes. And in the next step, you can use this to build a license report for your BSP. Yeah, you would basically just call PDX's make license report and then it collects all these license material from the available and selected packages and drops them into a nice PDF. So you will see here, DOSFS tools, C2FS procs, GDB server, G-Lib C and the copying files for all of them. Yeah, and they also contain this dependency graph. So you know the open SSH, which is license under BSD and BSD three clause and MIT and be aware and ISC license. It depends on the Z-Lib, which is licensed under the Z-Lib license and OpenSSL, which is licensed under the OpenSSL license. So you can find software that maybe depends on GPLv3 if you don't like GPLv3 or something. Yeah. You can also use this as a template for your license compliance process. All right. Next topic, reproducible builds. PDX test tries to build packages reproducibly by default. You can select which date it uses and the goal here is if you have the same source input then you also have the same binary output after compilation. So you always know if you're having the same binary on two build holes, then it has to come from the same source. And PDX test does this by setting this source date epoch variable during build and the packages build system has to respect this, but I think it's quite standard now. There are only, yeah, I think it is quite established right now. And of course, the date is only one thing we can pin down, but there are various different variables that go into the packages and which the packages itself have to make sure to use reproducibly. And this is really an ongoing effort, but yeah, PDX test does everything it can. Next, we're also having a mechanism for your BSP to document itself. And in this case, you have a doc folder inside your BSP, in this case, district it and it contains some rescripted text files and PDX test is then using swings to build a nice, in this case, HML documentation or it can also use PDF as an output format. Then there are some development helpers which are really useful if you develop with your BSP. For example, if you want to develop your application and when developing the application, you need some container to run it in, right? So PDX test in this case has, for example, an NFS root support. And if you call this NFS root command, then it first builds all the packages, of course, and then exports the sysroot folder where all these packages are installed on the devilhost. It exports this via NFS and sets the correct access rights too. So you can just tell your kernel to use this NFS root command line. And if your board has network, then it will just use this exported NFS folder as the root file system. And this has the advantage that you can use VIM on your development host and don't need to SSH to the target and use the, I don't know if you have VIM on the target, but yeah, it's mostly not a fun experience. And you don't also need to reflash or rebuild or reboot your target. So you can just edit files and plates. For example, if you just want to test the config change or something like that. Next thing is we also have a GDB wrapper. You can use this with a GDB server or just to debug core dumps from your target. And you can just use GDB on the host with this ptxts GDB command. And it's basically just a simple wrapper that sets up the rights sysroot. So GDB finds all the debug symbols and yeah, something like that. Also quite nice to have. Yeah, it's a summary. What is ptxts? What makes it unique? It has a quite useful menu interface. It has under the hood, it's using stable and non-technology. The focus is on reproducibility. So we pin down most variables which we can control as much as possible. So if someone else is building the same BSP, then we hope it's the same output as someone else is building it. And we also like to solve conflicts earlier rather than later. We have nice code signing infrastructure and some development helpers which help you developing your application. If you want to try it out, you can have a look at district it, which is a pre-configured BSP for a variety of dev boards. I think at least Raspberry Pi one, two and three, then some older development boards like the Nitrogen six boards. We also have a QMU emulation for R and V express. And also I think for some MIPS platform and x86 64 bit. And I think we're also having some 64 R and V8. Yeah. So yeah, you can use it to try ptxts or maybe just use it to build a BSP for your own development board. Yeah. Have a look at this homepage here. Here are the references. You can start from font awesome and the Tux logo is by Larry Ewing and the Gimp. Right, that's the end. Thanks for listening and time for questions.