 Hello and welcome to this presentation on the STM32 ecosystem in the Graphical User Interface context. First, we will review the processes involved in graphics rendering and how the STM32 family is prepared to handle them on a hardware point of view first. Then, we will consider the software parts of the graphics processing through the Graphic Library. The third part will present the full set of tools proposed by ST to develop Graphical User Interfaces on the STM32 products. And finally, we will introduce one of the Graphical Libraries solution that takes full advantage of the STM32, which is called STMWin. What does graphics rendering mean on embedded platforms? The frame buffer is the image that will be sent directly to the display, but before sending it, this frame buffer must be created. This process is called frame drawing, and it's a very CPU demanding operation. It involves processing each pixel several times per second, and for a SVGA definition, it means almost 500,000 pixels. The drawing in itself involves repetitive operation, for example, for pixel color conversions. It's only numerical computing operations, not logical ones, and it involves a lot of memory accesses, as are reading and writing. These steps are very suitable for a dedicated IP, or 2D engine, instead of the general purpose cortex that can be then used for other tasks. What are the most common operations when drawing a Graphical User Interface? Considering the smartphone UI shown on the left side of the screen, you may expect some 3D rendering operation, but in fact, there are only 2D rendering operations involved, such as alpha blending, color filling with transparency, simple color filling, or alpha bitmap glyphs. We will describe what glyphs are, but basically it's the way of displaying text in graphics world. So all these screens only involve 2D operation. That's why ST introduced the Chromart Accelerator in his STM32 family. It is more efficient than Cortex. It has a direct DMA access to memory and is dedicated to pixel processing. The main advantage is to offload the Cortex during graphical rendering. It's much smaller and cheaper silicone than a GPU based approach. A GPU is a graphical processing unit and generally involves a strong software framework and also a lot of dedicated memory. The Chromart covers 90% of nice graphical human-machine interface use cases. Let's have a look at the Chromart pixel pipeline. The Chromart has two inputs, first a background layer and a foreground layer. Here we have a simple image as a background layer and the foreground layer is composed of an ST logo with alpha information as well as what is called a glyph. A glyph is a bitmap representation of a font, of a character. This glyph is associated with a color which is orange. First, foreground and background are put in their dedicated FIFO in the Chromart and then transferred to the pixel format converter, which allows to align foreground and background color depths to mix them correctly. In this case, the background layer is probably ARGB 888 format, while the glyph has a more compact lookup table format, and the ST logo may also be maybe RGB 565. So to combine the foreground and the background, the pixel format must be the same. That's why there are input pixel format converter. Then the converted images are sent to the blender and then sent to an output pixel format converter, depending on the desired output color depths, then to the output FIFO and finally to the screen. The main advantage of the Chromart is to be able to do some in-place processing. It means that no extra memory is needed to store the intermediate results. The ST logo and the glyph are directly blended in the background layer. The Chromart accelerator contains a DMA2D, which allows memory-to-memory DMR transfer with programmable rectangle area. It contains an alpha blender that is able to perform per-object alpha, but also per-pixel alpha. So here you can see on the ST logo that there is a global alpha associated, which makes it partly transparent. But on the glyph, only the part that is outside the character is transparent, fully transparent. In fact, the pixels that compose the character itself are fully opaque. And the pixel format converter supports many formats in input output and some compact format, lookup table format, but in input only. You cannot output color lookup table format from the Chromart accelerator. Concerning the rendering process split, the Chromart is designed for most of the rendering operations. For direct data copy, data copy res blending, field rectangle res or resat transparency, and pixel format conversion. Some of the STM42 products also include an hardware GPEG decoder for real-world image. But the other operations are handled by the Cortex. Other operations include drawing primitives such as lines, circle, polygons, geometric transformations for complex animations, advanced color effect, and the smart rendering algorithm management that we will see in the graphical library part. But even this optimized algorithm is based on Cortex M instruction and floating point unit. So even if it's done by the Cortex itself, it doesn't mean that it's not efficient. The graphic subsystem can be summarized like this. Here is a basic STM42 representation with the Cortex, the Chromart accelerator, a way to access the flash memory, an external one, and a way to access also an external RAM memory. We will not use the internal SRAM in this case, nor the internal flash. The bus matrix is shared by the Cortex, the Chromart, and also the TFT controller, which is called the LTDC. We have a set of input images for our graphical user interface that are stored in flash. And here is the empty frame buffer stored in the SD RAM. In this scenario, we have only one frame buffer. So basically, the Chromart accelerator takes as input the input bitmap stored in flash and filled the frame buffer with these images. Then the LTDC sends this frame buffer to the display. So the green line is only the update of the frame buffer by the Chromart. It doesn't have to be done 30, 60 times per second, while the TFT update must be done at 60 Hz. So that's why usually, at least for a complex graphical user interface, several frame buffers are used. One is sent 60 times per second all the time by the LTDC. The other one is filled by the DM2D. When a frame buffer is ready, these two frame buffers are switched so that the new frame is displayed on the screen. Regarding the bus matrix dimensioning, let's consider an STM32F4X9 running at 180 MHz. It can support the update of the TFT display without controller on a parallel RGB interface using the LTDC. Plus, in parallel, a continuous Chromart re-dried accesses to the external SDRAM. And plus, in parallel, a regular CPU access to the external SDRAM, with a maximum pixel clock of around 30 MHz in a 24-bit per pixel and 45 MHz in 16-bit per pixel. For example, the achievable definitions with a 60 Hz refresh rate are SVGA at 24-bit per pixel, or larger SGA, but at only 16-bit per pixel. And in terms of memory bandwidth, it means that on a parallel RGB interface, the bandwidth used for the refresh of the display, not even the rendering of the image, is around 90 Mbps. And this is handled by the LTDC. So this is a kind of worst case where you have a display without controller, and you must send 60 times per second a full frame to the display to get a proper image. But for displays that have an integrated controller, the 90 Mbps is no more relevant, because you only send once a frame, and then the internal controller of the display refresh the display with this frame without using the bus matrix. We will see that in the display interface's presentation. The graphic library features. First, a graphic library proposes graphic user interface elements rendering. It means fast drawing primitive for basic geometric shapes such as circle, lines and polygons, anti-aliasing support, geometric Leonardo transforms, alpha-blending supports, for example for fading effects, smart frame buffer management, both for seagull and double buffering, and an optimized graphics engine, which supports occlusion culling and dirty areas. As a reminder, occlusion culling means that we do not render objects that are behind other objects, and dirty areas are the areas that must be redrawn because something moved in this area or something changed in this area. Besides proposing graphical user interface elements, the graphic library must also handle their internal logic. It means event-driven object updates, events can be touch events, keyboard events, timers, sensors, and also other graphical user interface elements. Graphical library also supports touch gesture recognition such as press, drag, rotation, and zoom. And the graphical user elements states are event-driven, so the graphical user interface application rendering is based on a three-step loop that periodically checks new events, on touch, timers, etc. Executes the GUI elements logic upon these events that determines the list of the objects to be redrawn, and starts the competition process through the library's graphics engine. Another feature of graphic library is the support of the text. Another feature of the graphic library is the text management. Glyphs are a bitmap representation of characters, so that the graphic library has some tools to convert true type fonts into glyphs, so into alpha bitmap. The alpha bitmap, as you can see on the right side of the screen, and as we already mentioned previously, is a bitmap representation of text. So each selected characters are converted into a glyph, so it's an alpha-only bitmap. The graphic library also handles the font kerning, which is the space between the letters. As you can see with our result kerning at the bottom right side of this screen. And also support dynamic multi-language. Finally, the graphic library generally offers some widgets. A widget is a window gadget, and it's a graphic element with an embedded logic. The most usual one is a button. A button, when clicked, has a different appearance as its release state. So it has an internal logic to change the bitmap of the button when the button is clicked. Then you have several other widgets, such as indicators or menus that are listed in this slide. So you have some radio buttons, some sliders, some checkboxes, and some more advanced ones such as menus, treeview, drop-down menus, scrollable list wheel, and even virtual keyboard. So a graphic library generally offers their own proprietary widgets, but also a way to define your own widgets, to customize the widgets. Let's now have a look at the STM32 graphic ecosystem. This is an overview of the STM32 ecosystem. It is composed first of hardware tools. The hardware tools are all the evaluation boards, the nuclear boards, the discovery ones, or the eval ones. But also the ST link that is included in all the evaluation boards. Regarding the embedded software, the ST ecosystem proposes the drivers, of course, at a very high level, such as HAL, but also the low-level ones, the LL interface. And the embedded software offer also proposes the real-time operating system solution of VR tools, but also the stacks and application bricks, such as, for example, the STM win that we will present in a few slides. The ecosystem also provides some software tools. First, configuration tools such as the Qubemix. The Qubemix allows to configure all the peripherals of a given board and automatically generates the source code associated to this configuration and initialization. It supports many integrated development environments, also called IDEs, such as system workbench based on Eclipse, but also IR or KIL or ATOLIC. And finally, the ecosystem includes information, sharing, and support through either the online system tool on the website, but also through the communities dedicated to the STM32 product. So this ecosystem is a combination of ST design tools, open source ones, but also partner's ones. Some of the partners of ST are dedicated to graphics, so they fulfill the ecosystem with graphics functionalities. These are the three solutions that take full advantage of the integrated hardware accelerator of the STM32 family. First, the STMWin is a free solution provided by ST. It comes from a partnership with a secure company, but with a special contract that allows ST to provide for free binary version of this library. Other solutions, such as the JFix and embedded wizard, are also taking full advantage of the Chromart accelerator, but also the LTDC. In this session, we will focus on the STMWin free solution. These are the ST products that support graphic features. We will, in this session, focus on the F469, which offers the DSI interface and TFT controller, which is the LTDC, FMC, and Quad SPI, and of course the Chromart. But there are other platforms, as you can see, that can support graphic features. The STMWin graphic library. The STMWin solution is proposed in partnership with the SIGIR company from Germany. Their fields are embedded middleware. They propose some IP stacks, some USB stacks also, but also propose some JTAG embedded tools for debugging and profiling. Actually, the STMWin is the ST integration of the EMWin library proposed by SIGIR. And the STMWin, of course, is fully optimized for the use of the dedicated hardware accelerator included in the STM32. It is delivered as binary with the STM32Cube firmware packages. The library in itself is programmed in an object-oriented C-frame rock. And the development environment includes a drag-and-drop, what you see is what you get, screen editor, dedicated to the graphical user interface appearance only. The gray logic must be done in an external editor. It proposes a PC simulation tool based on Microsoft Visual Studio Express. And it also gives some true-types fonts, image, and icons converters to C files. We mentioned previously the Cubemix tool called Generator that allows to configure all the clocks and peripherals of a given STM32 platform. It now also supports the STMWin library configuration and initialization, as well as the Friartos configuration and initialization. The Cubemix tool supports multiple IDE for building, flashing, and debugging. The IDE, as also mentioned, is a system workbench, IR, Keele, and Atolic. And the CUBE firmware also includes a set of ready-to-use examples, or demonstrator projects, for each relevant STM32, which includes, of course, the graphical user interface example. So developing an application with the STMWin and the STM32 ecosystem would, for example, mean first configuring all the peripherals of the chosen board using the Cubemix tool, then set up the graphical user interface, then generate the code, for example, this is for the IR IDE, and finally, flash directly on the board using the STLink included on the board, or an external one for a dedicated custom board. For more information on the STMWin, you can refer to the application note 4323, but you have also some interesting application notes on the LTDC rendering and on DSI also. And for other information or issues, you can directly contact the STSupport on myst.com. This is the end of this presentation. Thank you very much for your attention.