 Thank you for joining me today. So today I'm going to... First of all, I'm Alexandre Franges, I work for Collabora, and today I'm going to talk to you about an RD project I've been working at Collabora called the Pixel Format Guide, subtitled humorously, of course, to the Galaxy. So I'm going to tell you what the Pixel Format Guide is, what it can do out of the box, and also how we can leverage its functionality to do some more custom things with it. So let's start. So in the Graphics Wilderness, we find strange beasts we call Pixel Formats. We give names to those Pixel Formats in order to tame them, but the thing is that each Graphics API and each project uses a different flavor for this name. So for example, we have the Deron Formats, SDL, GL, and many, many, many more. The problem is that just by looking at the name, we often can get some idea of what the Pixel Format actually is represented by the name, but we are missing a lot of information, and often the documentation is either lacking or vague, sometimes even misleading. And the only way to figure out what's going on exactly is to look in source codes, and even then it can be very, very crazy. So are we doomed? Do we have a solution to this? And the solution is this, or perhaps something like this for the Pixel Format guides. And actually this is what the Pixel Format guide is all about. So it's a collection of information about what the various Pixel Format names mean. So it describes the layout of the components for Pixel Format names, so the number, the order, and the size. And also it describes whether the format is represented as a native integer or as an array format, as we call it. So bits and bytes in memory in some fixed order. This is very important because it determines whether the Pixel Format is actually, the way the Pixel Format is stored in memory is affected by system maintenance or not, which is a piece of information that's often missing in documentation. So let's start. So the first part of the Pixel Format guide is the documentation itself. It's a collection of documents describing the Pixel Format names at a high level. So for example here, don't have to read it. It's the start of the webpage describing the German Pixel Formats. And the documentation is posted as a GitHub website. So it's automatically created from the repository. And it can be very useful when we want to get a broad overview of how to interpret Pixel Format names for a specific family. The issue here is that, although it's quite useful, it's actually not the most exciting part of the Pixel Format guide because we can do much, much more. With the power and the magic of computers, we can encode all of this knowledge we have about the Pixel Formats and we can automate the process and automate it too. And this is the second, perhaps the most exciting part of the Pixel Format guide. So we have the Pixel Format guide tool, which is a Python tool, which supports some operations which I will describe shortly. So starting from the bottom, you can see the list formats. So at least all of the formats we know about for a particular family. The list families, it lists all the families we know about. By family, I mean SDL, DRM, Vulkan, GL, etc. And then we have the document operation with print out the documentation for a particular Pixel Format family. And this is actually the same documentation that's used to produce the website. So it's basically a marked-down file. And then we have the most interesting operations, Describe and Find Compatible, which I will describe shortly. So starting with the Describe operation, the Describe operation gives us in an easy-to-consume manner all the information we have about a particular Pixel Format. So, of course, first of all, we'll get the name, then we get the component data type, which describes how the bits in the components are actually interpreted, what they mean. So I'm using the Vulkan names for the data type. We also have unsigned, normalized, signed, normalized, signed-and-test-float, etc. Another crucial piece of information, as I mentioned before, is how is this format described? Is it described as a native? So, it fills in a native type, or is it described as bits and bytes in memory in a particular order? In this case, we have a native type, so we get the description of the native type. So we get for each pixel what each bit of the Pixel Format means, to which component bit it corresponds. Then we get the description of how the Pixel Format is stored in memory, both for little-endian and big-endian systems. And you notice in this case the little-endian and big-endian descriptions are different. The bytes are just swapped, because that's how this works in this case. Moving on to an example of a format that's described as bytes in memory, we don't get the native type description, but we still get the little-endian and big-endian description. In this case, the descriptions are the same, but in some cases they may differ. That's usually the case when we have pixel formats where each component is multiple bytes. Each byte, its component in memory is in the same order, so for example, RGBA, but then the two bytes of each component may be flipped depending on system-endianness. That's described. The next interesting operation we have and perhaps even more interesting is the Find Compatible. The Find Compatible operation allows us to ask the tool to give us all the pixel formats from a particular family that match another format. In this case, we're asking the tool to return all the pixel formats from the SDL2 family that are compatible with that particular Vulcan format. The tool replies that for this format, this format is compatible on all systems with this and is compatible on big-endian system with the other pixel format. By default, the Find Compatible operation does a very strict matching between pixel formats and other components, but it's often useful to relax the matching a bit sometimes, so we have a few flags. One is the Treat X as A, which treats unused parts of the value of a pixel format, also known as X, in Quilt A. Then we have a flag which allows us to treat sRGB data type the same as Unorm, and then we have an ignored data type flag that allows us to ignore the data type completely and just do bit matching basically for the components. These are useful in different scenarios. Sometimes the formats we want to match don't even have, for example, sRGB variants, so the best we can do is Unorm or they don't have X variants, so we are OK matching with the A. The PFG tool we have been talking about is really just a front-end to the PFG library, which is a Python module or library, which is where all the magic happened. It exports a set of operation, which you will see in this case match exactly what the front-end exposes as a command line. So get list formats, list families, document, find compatible with all the flags, and describe. The nice thing about exposing all the functionality as a library is that we can easily go beyond what the tool provides and provide custom solutions. So, for example, especially in integration layers between graphics libraries or project programs and libraries, we need to convert one pixel format name to another in a compatible way, of course. And this is where, for example, PFG as a library could become very useful. So this PFG, this Python script does exactly that for the SDL2 to Vulkan. I will go quickly through them, through the script. So we import the PFG library, we set up the switch statement, and then for each format we know about in the SDL2 family, we find all compatible Vulkan formats. Then depending on whether we actually found something and whether this match that we found was compatible everywhere or only on little end-end systems or only on big end-end systems, then we can produce case statements. And running this would get an output like this automatically and without errors which we would have if we would like to do this manually. Of course, you can do whatever you want with the library. It's not switch statement, it's just one of the operations, your custom operations you can do. You can produce tables or whatever for your application. So the PFG tool and the library consists of a core that provides the operations and then of a set of sub-modules that provide all the information we have about a particular family and these are disconnected. So it's very easy to add a new family. First of all, to do so, we should add some documentation under docs-family-md and then start adding the code to implement the other logic. So because I like using test-driven approach in my coding, I usually start by adding some tests in test-family and adding code in pfgfamily.py to implement to ensure the tests pass and implement that functionality. So family-py in pfg should provide the following three functions. First of all, describe format string. That returns a format description which is a Python class you can see in the code base with all the information about the particular pixel format. You also need to implement the formats function which returns a list of all the format strings that this family supports and finally, the document operation that returns a documentation from basically what we have, sorry, that's not py, that's docs-family.md because the operations we need to do internally are mostly similar between families. There are some utility functions in pfgutil.py that you can use to make your life easier. And that's the basic, these are the basic steps. You can find more details in the section of the code. So first of all, what's the current state? The current state is that we support 12 families, if I'm not mistaken, and over 450 pixel formats which we can use to describe or match things. Of course, we want more pixel formats and this is where all you all can come in please, if you have a favorite pixel format, please consider contributing to this. It's on GitHub, it's very easy to contribute. And for the future, the plan is to add compressed and multi-plane formats. I want to add descriptions for multi-plane formats for compressed, not so much, but I want to be able to tag them somehow so I can use them for matching but we know that this pixel format name from Valkan, for example, represents the same compressed format in whatever DRM family want to match. And I guess that's it. Thank you very much for being here and questions. That was fast. Any questions? Right, so I'm dealing mostly with the pixel format itself and not image formats. Stride would be more relevant I guess to whole image formats. So I'm not dealing with this in this project. I will probably change if you do. Yeah, it will probably change. Any other questions? Sorry? Yes, if you go to the FOSDEM page for this talk, there's a link to the project and also to some blog posts that describe the project in more detail. Thank you.