 Okay, good morning everyone. I'm going to talk about the Linux Fund Rendering stack. And that is what I learned the last four and a half years. I worked at the city administration of Munich for the introduction to fund rendering. I can say writing is the most used interface between humans and the machines. So all of you probably use it every day on your computer. Screen text is replacing text on paper more and more. And this is still an ongoing process. And the way from a string in the computer to display text is surprisingly complex. And writing in itself is quite profound. For example, it's influenced by history. So probably all or at least most writing systems are derived from some pictographic writing systems. This is also true for Roman letters. And also the Roman letters themselves, they changed a lot. Since 2000 years ago at the Roman Empire, they used some different letters that we use now. So the goal of this talk is to raise awareness of fund rendering and computing with the focus on the Linux desktop. Now to history. And then I talk about typography. Text display in early computing. In early computing it was just a way to display results. And before that there were only flashing lights where you can read the results. And the output device receives a page or a character stream. And then the presentation is the job of the output device. So for example a typewriter or a printer or a terminal. And this is a VT100. You can see on the right side. So this is a real terminal. But going on in history there's the home computer era. In the home computer era you have fixed character sets. And most of the time due to system limitations, memory limitations, you have graphics mode and the text mode. And in the graphics mode you can basically set the pixels more or less as you want. And in the text mode you can use fonts. And the screen configuration at that time is more or less defined by the state of technology. So every screen at that time has more or less the same resolution. And that's something that the engineers of that time relied on and also software engineers relied on. And the text rendering was just using bitmap fonts. That means you have a fixed size per character and you just define it pixel wise. So it's some kind of handcrafted characters. And you have a given raster and then you really have displayed as hard defined characters. So then I need to introduce a few typography terms. So first there's a glue for a character. And well, that's, well, you can say a symbol. So I think most people can understand what that means. Then a font, there's a set of glyphs. And then we have a collection of fonts that is called typeface or font family. Those are closely related fonts with the same visual appearance and they differ only in slope or width or weight, for example. So usually in typesetting, with typesetting with moving letters in the printing press, you really had for different weight, for example, you had different pieces, really letters, hardware. And then the point size or typographic unit, this is a measure of the size and derives directly from the printing press with moving letters. And there are different definitions. And what we probably all use as point size is the DTP point, desktop publishing point, where one point is defined as 1 over 72 inch, which is 0.3527 period millimeters. And it is interesting to know that this has nothing to do with pixels. Please use the microphone if you have a question. 3, 5, 2, 7, what's the bar over the 7 me? That means that the 7 goes on and on and on forever. So this is a typography of Latin alphabet terminology. There's something called kerning. You can see an example on the right side of kerning. So in the printing press, you have letters, and they usually have a gap to improve the readability and make all the texts appear more uniform. The types that reduce the gap with moving letters, they have special letter pairs. And if you go on with this, there are Liga tools, which are replacement glues for close standing characters. An example often is double F or Fi or FL. So you can see also on the slides here that the double F is such a Liga tour. It's one connected glue. And then there's a specialty. So if you reverse all the process, then there's the term non-proportional font. So you have a font where every character, where every groove has the same width. This is also called monospace font. And sometimes you distinguish half-width and full-width letters. And this is something you'd also see on a typewriter, for example. And early bitmap fonts are usually monospace fonts. Okay, let's go to the text rendering stack. Text rendering today, in all computing nowadays, typography is introduced. So you won't find much devices on the market which don't use accurate typography. And glues are now represented as size-independent outlines. And you can see on the right side there's a font editor, font forge, this is the way such fonts are created. So you can see here in the picture the outline of the S. And it's a size-independent outline and you can just use that for every kind of display. And then you do something that is called rasterization or sampling or rendering for displaying it at the actual display. And in the history this allowed new applications for computers. First it allowed word processing and computer-based type setting. Before that type setting was a very hard job. And it allowed internationalization and it allows universal graphical interface which goes hand-in-hand with internationalization and come to think of it, not so long ago it was not so common that you could buy devices where you easily can switch the language to any language you would like. This is just a recent development. So now I will give you an overview of the formats that we see on Linux systems. First I talk about the bitmap formats. So today we have bitmap formats in the PC screen font format, PSF or PSF-U. These are the fonts that are used for the Linux VT also known as the console. And usually you can store in such a font 256 or 512 glyphs and some of them contain a Unicode translation table. And on the other hand there are X-window system bitmap fonts and there are three formats. There is server normal format SNF, Gluf bitmap distribution format, BDF and portable compiled format PCF and nowadays only the letter you can find on a Linux system if so and the first two are deprecated and also the X-window bitmap fonts themselves are not very common these days. So to the outline font formats, there are post-script formats and post-script formats are different, I would call versions, it's called types. So type one is the one that is still relevant to a certain degree and those are the ones you can probably find in standard Debian installation for example. It uses cubic bezier curves and the file format is divided into different files. So for every font you have a printer font ASCII or a printer font binary file and you have a print font metric or a Adobe font metric file and then you have the true type file format. This is more common, I guess everyone heard of that already. It uses quadratic bezier curves and it can contain optional code for the true type printing virtual machine and that means that I will explain later. And there's a third format, the open type format which has two possible glue formats. One is true type and one is compact font format which is based on post-script type two. So you can say, well, what is new in this format and what's the difference? Well, it supports so-called smart fonts and that means you can have language-specific legal tools or character substitutions or for example, cunning classes which means you have a class of characters like the A and different variations. So these are A with different diacritic symbols and you just can use one cunning class for all of those characters and don't have to invent the wheel for every A with diacritic symbols in you. So now to the font rendering techniques that are used today. Well, this only applies to the outline fonts because rendering bitmap fonts is obviously quite easy. You just paint the pixels that are in the bitmap so restoration is all about using outline fonts and the one library that is used all over the free and open source world is free type. It is used on Linux system, BSD desktop, Android and ReactOS and there are also some others. For example, it's containing ghost script and therefore in most of the printers and it's also on iOS. So the naive restoration algorithm would be you just lay a pixel raster over the outline font, over the character you want to display and if the center of the pixel is inside the outline then you set the color to black and the problem is the so-called erasing effect and this is what you can see in the picture below. So those are the same words rendered at different resolutions. The left one has ten times the resolution of the right one and you can see that for example the W is quite deformed or the curl of the G there's even the part missing so you have details of the fonts that are lost and you have artifacts and somehow, especially in the early computing days you wanted to reduce those and therefore you used a technique called hinting and this is to avoid such artifacts and improve the readability at lower resolutions and therefore the outline is adjusted to fit the pixel raster at the rasterization and some of the fonts can contain instructions, the so-called hints and that is where the name comes from hinting and you can see the example here in the picture above that it's quite a good result so this is much more readable than the word above however, with the use of hinting there are also characteristics of the fonts lost which is obvious because if you change the outline then obviously characteristics get lost and especially important for word processing what you see is what you get word processing is but the tracking has changed of the font so tracking means the width of single characters and the picture below this is a picture I took from LibreOffice and if you have a close look you can see that the gap between those eye characters is not always the same but if you do word processing you actually want to have a result which looks the same as on the printout so I would recommend not to hint fonts what you see is what you get word processor and if you have a look beyond the backyard on macOS there exists no hinting and in the Windows world hinting can be turned off so this is hard wired in the front rendering of the Windows system another approach to improve readability is anti-aliasing which applies multi-sampling so for every pixel in different spots you take samples and then you compute from that a grey value which is a measure of how much of the area of the pixel is covered by the glue and on the picture you can see again the simple rest of the word and then in the middle you see the word with anti-aliasing and below you can see a combination of hinting and anti-aliasing and suddenly the text becomes very readable also on low resolutions and this is what was usually used in computing in the 90s so there is another approach to improve and therefore I have to talk a bit about LCD this place in the picture on the left above you can see tube monitor pixels and on the below you can see LCD monitor pixels and all of them are all of the pixels consist of so-called sub-pixels with different colors and the mixture of the three different colors gives them a wide range of different colors and with LCD display with the tube monitors this is not used but with the LCD display depending on the configuration of the sub-pixel this can be used to improve the resolution in one direction and therefore you have to know how the configuration of the display is so usually one pixel which you can see in the picture below right these are the usual computer monitors they consist of one pixel consists of a red, green and blue sub-pixel in this order but you have to keep in mind especially with tablets or smartphones you can rotate the screen so you have to keep that in mind and also there are other sub-pixel configurations for example depending on the technology there can be additional red or green or even white sub-pixels and so there are also multiple possibilities of the configuration so we can use this configuration as I said to improve the resolution and in the usual case it's horizontally and you can see on the picture on the left side first there is the naive, simple rendered character then the character just using anti-alising and then there is the sub-pixel rendered character and this one is the most readable and most sharp character well not sharp but it's most correctly rendered according to the outline font and well depending on the technique of the display you can see a colour haze around the characters and so this happens when the software and the display technology don't match each other so this is what you can see in the picture on the right okay let's talk about the front-running software parts in the Linux desktop so there's the so-called server-side text rendering so in the X-server there is the core font subsystem and with that X11 clients can request the server to display a text by sending a string via libx font and using the so-called X logical font description XLFD and here's an example for Adobe career font and then the X-server has to render the text using the font and if you imagine a terminal server set up with SYNC clients then well the SYNC client runs the X-server and the terminal server runs the X-client and every SYNC client then has to handle the font rendering so I don't know if this is the only reason but nowadays server-side font rendering is not so common probably not used anymore so now I have to talk first about font management to go on in Linux systems there is a software called config which manages install fonts on the system and it configures for example how to substitute fonts for example if there is in a document there's a font to render a text and the font is not available on the system then in the font config system there can be rules to replace the font with a similar font and this is heavily used in Linux systems and there are also rules what font to use if the current selected font doesn't contain a character you want to display and you can see on the picture on the right that this is also used well I have to admit I had to take this picture on a Debian 5.0 system in LibreOffice because nowadays it works a bit different but this is a good example where you can see that those characters come from different fonts actually and there's also a command line tool which is a quite nice FCmatch command line tool so for example if you want to see what is the replacement font for let's say Arial then you get an output okay it's NimbleSuns and you can also set rendering options which means you can set anti-aliasing or you can turn off hinting and usually these configurations are stored at the fonts and there's also per-user-configuration.config and it's in XML file format snippet so here in the picture you can see an example this is the replacement rule for Calito and Calibri which Calito is a replacement font for nowadays often used Calibri font from Microsoft and so this configuration says the one way Calito is the same as Calibri and the other way around so this is just an example you can have a look at the fonts and there are a lot of such snippets and it defines how fonts are displayed on the system another thing and this is the reason why I had to take the screenshot on an older system there's a piece of software called HalfBus HalfBus is a version for OpenType so this software relies on the OpenType font format and you can see on the right side the HalfBus logo this actually says HalfBus in Persian and before I talked about legal tours in some languages legal tours are required to render fonts correctly and this is an example on the left, Devangari which is an index script so if you have the first two characters combined then this is the rule how to replace those characters by a third glue and there were early implementations by Qt and Pangol and those were integrated in HalfBus and those parts are now known as HalfBus old and the current HalfBus is rewrite and nowadays it's also used for a so-called simple script meaning especially a Latin script and it's integrated in Qt, GTK, LibreOffice, Firefox, Android and Xetech this is what I use for the slides here so all the slides here are also rendered using HalfBus and of course the whole font rendering stack and HalfBus also has fancy features like variable width or weight with only one font you can define in the font characters with variable width and without stepping you can change it using HalfBus and those techniques are used in client-side rendering and one library often was, until recent, often used XFT and this is the applications meaning the X clients rendered the text and based on free type and font config and then the X server only displays the results well there is some caching involved and it requires an extension to the protocol and this was also in the widget libraries this was also used a long time but now for example Qt has its own code for font rendering based on HalfBus, free type and font config and there is the combination of pangu and kyro which is used in the GTK environment and pangu means device from greek and japanese and pangu means all and go means languages so this is the background to write in all languages and this also used HalfBus, free type and font config and if you look at wayline clients they only do client-side rendering okay to sum it up we have a variety of techniques for text display some have more historical and practical value nowadays and the modern font rendering stack is quite complex but writing is one of the main interface to computers so developers should be aware of the complexity and keep that in mind thank you, are there any questions? more requests than a question I find the topic very interesting I'd like to check it again I know your video was not recorded I think it was not recorded it was I think it's recorded never mind but still if you are able to share your presentation I would be very happy to look at it, thank you what's the problem due to this complexity from the software stack in your opinion the problem of what? what's the problem what's the problem it came from this complex software stack in your opinion there's no problem with that or there's some problem with this a lot of layers, a lot of components to render their font okay I try to rephrase the question what is the reason there are so many layers in the font rendering stack today is that correct and it caused some problem do you know any problem with current font rendering system a colored font rendering system current now the question is whether the complexity means that there are difficulties for developers is it too complex does that cause problems the complexity is hidden behind the font rendering stack so it's a huge collection of software usually there's half bus and font config and free type heavily used nowadays but it's all encapsulated in these libraries and it's used all over the free software world and it makes it easy to write applications which are easily translatable yes but for example the font config is not so flexible for example when we're using the mainly Japanese and the other display as a language then the font config chooses the non-Japanese font like the Chinese one and I know that some distribution like Ubuntu make languages specific the config file for each language to deal with we deviant doesn't have such a mechanism so in my opinion the font config is a bit terrible so is there any replacement or any hack for it the font config system comes from the time where XML was popular as you can see and the problem here is actually not so much the software stack more than the Unicode system so in the Unicode system there are same code points for Japanese and Chinese characters although they have different appearance and in the font config system there are replacement rules for characters and it's just a list and usually Chinese fonts are listed first so that is the reason why in Japanese language Chinese characters instead of Japanese characters and I also think there is room to improve the software stack thank you ok, thank you