 Okay, so I'm starting now. Thank you for joining. So I'm Neil Armstrong, and I work for Belibre. It's a small French company, French and American company, and we're specialized in embedded software engineering and embedded Linux, open source Linux, abstract Linux, custom firmware, SOC support, and product development. So first of all, we'll talk about the initial state of the art of the AMlogic mainline support when we started working on it. So first of all, a small presentation of AMlogic SOC families and AMlogic in general. So this is a American Chinese company who designed some multimedia SOCs for a set of box tablets and TV and projectors. And in the last year, they made a 64-bit ARM-based family, and the codenames are GXBB, GXL, GXM that are commonly used in some well-known development boards. And they also have some TV versions for TVs and projectors. And the older 32-bit families are presented here, but are less supported. So briefly, how the most recent AMlogic SOC family are organized. So they had some Cortex A9 processors, and they made the well-known S805, which was with Cortex A5. This generation could decode the 4K HT4. And last year, they made the S905, which is a 64-bit A53 processor, and can decode 4K HT5. And at the end of the year, they made two different SOCs. So the X version, which is the same as the 905, and the 9012, which is 8 Cortex A453, and has a better Mali core. And this generation uses better DDR, and can decode 4K, VP9, and 10-bit. That's quite huge. So there are a lot of products used on these three SOCs. Mainly, you can find more information and the difference on the SOCs on Wikipedia. So we concentrated on the last generation of SOCs, which is the 64-bit SOCs. So they have one or two clusters of Cortex A33, up to 1.5 GHz, or up on the 905. The 905 and 905 X has a Mali 450, and the last one has a Mali T820. I'll come back later on this situation. They have HDMI 2 with 4K HDR display. They can handle HT4, HT5, VP9, plus 10-bit decoding for VP9 and HT5 hardware decoding. And you can do hardware encoding in HTC4 and in the later versions HTC5. They have 10-bit and 10-bit internet USB 2 Austin device. That's quite common. Those well-known and well-bought products are here. You have the WebTech board here and here, which is really small. You have some next-box boards and a lot of generic boards, which are really cheap and really functional. And you have at least three community boards. The well-known Art Carnell-Ojoid, the C1 with the previous generation 805 SOC, and the C2, which is a well-loved development board, which has the 905 version. And the fresh one is the CadazVim, which uses the S905X version, which is a smaller and similar version. So what is the Linux support point? AMlogic uses Linux for all the SOCs, all of three, and based on the Android 3.10, 3.14 kernel, and heavily changed. But hopefully, not like other SOC vendors, they made all their changes in one directory. So it's far simpler to find stuff, actually. This is a good point. For F1 support, it was started in Linux 4.1. And some independent hackers made the initial support with the minimal booting, UART, the minimal DoF3. And only all the SOCs and very, very early, ARM-CC4 were supported. So they created a website and a lot of people are idling on the ERC. And we have a main list for Linux project. This is the state of the art of when we started the work in 4.5. So what was achieved since June last year? So this is a graph of all the work was done since 4.1. This is the small commits for the UART and the minimal stuff, the pin control and the basics. And since 4.8, we started pushing more advanced drivers and more support for all the elements of the SOC. So the last revision, 4.10 is the highest. We pushed nearly 60 commits and nearly 6,000 lines change, which is a good work. So what they believe does in this project? So we maintain and develop support for the SOC. Kevin Hillman is a well-known ARM maintainer and does the ARM SOC maintaining and development. Along with an endless mobile employee, Carl Lucaron, they use also the AMlogic SOC for their product. And I got the mentorship of the DRM display driver. What is great with the DRM subsystem is that they added recently an experimental white git access to the DRM MISC Git repository. When the patches are hacked by the maintainer, I can directly push the patches in the shell repository. So it's kind of great. It's a great experiment to directly push your Git commit in something that will be merged automatically. So it's a really great experiment and I'm happy to experiment this with the AMlogic work. So like I said, we concentrate our work on the 64-bit SOCs primarily, even if the hardware is really similar on the previous generation, so it will still work. So it evolves and the best, today the best release is the 410. Like we did a demonstration yesterday when we made the Quake 3 Arena running on the last test kernel revision. So mainly we pushed MMC-SCPI for DVFS, the display driver and the USB for one of the SOCs. So globally, the general area where work was done, we worked with the clock support in the clock framework because the number of gates is huge, the number of PLL is huge, like any modern multimedia SOC. We pushed the random generator, the watchdog driver, we modified the infrared decoder, we pushed the PWM, which is a key element for Wi-Fi, actually. We modified the ASO-SC and the SPI controller. This is the basics and we were quite easy to do, but still need some testing and some work. So one of the first big points was to make work the SCPI for DVFS. So DVFS is you change the voltage of the cortex cores with the frequency changing. So nowadays with the recent ARM cortex, ARM pushes to have a standard interface to changes and they use it in the Juno reference implementation and they ask all the licenses to use a similar implementation. So globally, you have your Cortex AS53 cluster, which has its own power domain and it communicates with Cortex M3 firmware, which has its own power domain. Nearly it's always on power domain. And you have a charred memory and a link which is used, which is provided by M2 with a mailbox interface with only a single word to synchronize communication. So it seems a really good idea to have a common and stable firmware except they gave early implementation of the SCPI firmware to the partners like AMlogic or Workchip. And they copied the implementation and they did the work. But in the meantime ARM reworked the protocol, changed the ordering of the command numbers and pushed it a spin. So we had a problem. The protocols are different and we needed to support a legacy protocol of SCPI along the modern SCPI. So hopefully Workchip did the same error. So it was a good point to push the legacy support. So globally we use a mailbox to synchronize. So ARM provides documentation of what the mailbox should support. So they have their own implementation, which is an AMBA device, so their own IP. So the licensee must design their own IP, which should be similar as their IP. So it's not very great because every vendor will develop their own IP. So it's not great because they don't support. Workchip used their own mailbox, which doesn't work the same as ARM implementation. So it will be a headache for them. Sorry guys. So AMlogic did a very similar support. But I need to work the ARM driver, but I tried to integrate in the same driver the AMBA support and the platform support. And it was a mess, absolute mess. And the maintainer told me, oh yes, you should try to put the two, but you need to duplicate only half of the driver. It was awful. So hopefully I made a clone of the driver, one for a platform device, one for AMBA, and it's stupid, but it works. And it's ready for new vendors for their implementation. So this was merged in 4.9. So finally, we had SCPI support for the GXBB for the S905 SoC. And because it's generic, this is a good point. Support for the two next generation of SoCs was really simple. Only adding the correct node, adding the correct indices for the WS3 and up, you had SCPI and WFS. So this is a good point of having a standard firmware interface, actually. So the next good, very important work was the EMC-SDIOSD driver. They changed the hardware implementation between the S805 and the S905. And we needed a modern driver to handle the, because the EMC framework has done a lot since 3.14. So we'd make a fresh driver based on the implementation. And the hardware supports a lot of modern features for EMC to achieve a very high speed SD card and SDIO. So the first version was really the simplest possible, one-bit or four-bit for high speed and only up to 50 MHz. But work is coming to have a better bandwidth. So like I said, at the end of the year, AMlogic published the next generation of products, so the GXL, GXM, which are really a variant of the first generation, the previous generation, and have only decoding enhancements. So things are very close. We did a great work of the device 3, with a new hierarchical DT structure, which is possible with device 3. You can have a common DTSI and include it in a sub-architecture DSI. And the GXM is even closest to GXL, so you can even have a sub DTSI, which includes GXL. So device 3 can add a lot of simplicity to describe the very similar hardware, even at a good point. So the last big feature we needed to support was the display driver. So our target was to have a minimal basic driver, so the simplest you have is the CVBS composite output, which is simple but works. So here Laurent Pichard did a great DRM structure graphic, where you can... DRM is the direct rendering manager in NX, which handles all the graphic stuff. Now it's the future and we need to use it, so we use all the modern features to handle the display graphic. So globally the DRM needs... it's separated in four or five concepts. You have the plane, the CRTC, which is the scan-out of the pixel, the encoder which transforms the pixel data in timings, and the connector which handles the physical connector on the board. So globally what we did, we mapped this with the internal hardware blocks. So in the Amlogic implementation, the whole stuff is the VPU, the video processing unit. For the plane management they have the VAU, the video input unit. So they can have two OSDs and two video inputs. They have a post-blend and a pre-blend to be able to put video on top and underneath the OSDs. So we use only a single plane here. And they have different encoders, the interlass, the progressive, the LCD encoder, and so on. So this is the simplest implementation we could push to have the minimal DRM driver. And with this DRM driver implementation, you have access to anything when XORs natively weigh-down anything. It was a good first start. And afterwards, for the GXBB905 SoC, only had a gigabit internet interface with external FI, and the GXL and GXM now embeds the 100 internet FI. So a small work was to support this FI. Hopefully the network APIs are really great and it's really easy to support such a custom FI. So this is the work we achieved until the end of the year for the 4.0.10 release. And this is what is in working products actually to support more features of the Amlogic SOCs. So obviously we need some performance optimization because the EMC and SDGuard are quite slow today. So we need to support the double data rate mode. So we can get data on all the data on the double data rate. We can support the high speed 200 and 400 with calibration. So you go very fast. You can go up to 400 megabits on EMC quite very fast. And they made the end version of the DMA with a scatter-gather compatible DMA. You can push multiple buffers in a chain. And a hobbyist made the optimization and it achieved up to 140 megabytes on EMC, which is really fast because now it's like 10 times less. So thanks, Heiner, for your work. So it's why also what you're pushing upstream is important because you can have some work done by a random guy and it's really valuable because we don't have a lot of time to work on this feature. It's not urgent for us. The other point was ARM, Mali integration and the way to use it to have a decent 3D acceleration. So globally, here is how it's architectured in hardware. So globally, you have, this is the IP that ARM licenses. You can configure it. This is the 450 mp3. So you have three fragment processors, a vector processor. You have a PMU that handled its own memory power domain and MMI had two cases. So all this is a big block. You don't need to know how it's integrated. And the GPL driver implement all this in the driver. So they reinvented the EMMU. They reinvented the cache operation and they reinvented the PMU management. So how we integrate in this system? So you have two physically separated IPs. So you have the SOC video processing unit. It's the same for other SOC vendors. You have the homemade unit. And you have the Mali driver that access the DDR memory. So on the camel space, you have the DRM framework that handle all the graphics. This is OK. It's open source integrated. You have the Mali GPL driver that handle the hardware. It's only an interface with the user space close source library. And for the other way, you have anything open source because the DRM interface is open source standard and in development, so it's OK. So the problem is it's complex to explain. The Mali GL implementation need to be configured and built by the SOC vendor. The ARM does not give a generic GL library. So you need to ask your SOC vendor to build a version based on the last release they give, which is buggy, obviously. So it's a complex situation because you need a different share library for Xorg, Wayland, M-frame buffer and Android. So they need to target at least four different share libraries and debug the four of them in every Xorg version, Wayland version and every frame buffer situation, which is complex and wide. So the complex situation is very complex and ARM doesn't help at all, actually. So they provide the GPL driver, like I said. So it's GPL and Incarnal. So it's not Incarnal, but it's built quite easily. So I had no issue to build it actually. And thanks to Fridtron Maxim, I made a clean platform support for the Mali driver because it's rather complex to use. The DTB links are not very good. So thanks to Maxim also, he pushed some upstream DTB links. So thank you. So I pushed a version of the driver for MLG.com and GitHub. And for the last generation of IFC, the GXM, they use the T820 Mali IP, which is really powerful. They have shaders, they support more than APIs, but still with the same concept of shared library. So and even worse, some S3 vendors even have access to the Xorg Wayland libraries. They only have the Android libraries. They can't provide the Xorg library at all. So it's kind of a shame from them because they have a very powerful graphic acceleration IP and only Android can use it actually. So there is some hope, but half hope, to hack it using LibIbris, which is used because Android GL layer is rather generic and you can wrap it around a library called LibIbris and use it on Wayland to have a proper acceleration. But it's still a shame to have a hack for this actually. So I say you need to configure it for your hardware, which is stupid. They don't want to integrate in Mesa and to see the secrets that everybody knows how the hardware works, but... So for this video display, we need some work to have a better experience and to achieve video decoding actually. So globally, the work is ongoing now and will be pushed for the 4.12 Linux with HDMI controller and free support. Next, we need to accelerate the cursor with the cursor plane. We need to support the overlay planes to push video on top underneath the planes. We need to have OSD scaling and overlay scaling to scale the video and the primary plane. So this is the final hopefully DRM architecture of the IMLIC driver with the secondary OSD that can act as a cursor display and the two video inputs overlay. So this is the plane where you can push your video frames and put it on top underneath each OSD plane. So support for the pre-blend. So you can blend the two video planes before blending it on the two OSD planes. So it is quite common and every SOC vendor does similar stuff. For the output, we need to support extended encoder for progressive video to support HDMI transceiver. So it's a synopsis transceiver and support HDMI connector with HPD and added a SQC port. So like a lot of vendors, they use the synopsis DZNWARE HDMI IP which is very common but absolutely not supported by synopsis. Nobody has documentation and only the SOC vendors. So it's really common. Friskal uses it. So it's a hard way to manage a driver without knowing what are the widgets and what they do. So hopefully it works like a charm on the AMlogic SOCs but like genetic IPs, you can integrate it like you want. So you have the controller which has videos that come in one point and outputs in nearly TMDS. So you need a file, a file that converts it in actual TMDS signals to go on the cable. But the file, you can be custom. You don't need to buy the synopsis file if you want. So the hardware is generic but there's no way, there's no software, no documentation from synopsis. No, they could provide a simplified documentation to know what signals must come in, go out and send the minimum registers. So we need to work some part of the DWV-HGME driver which is here, which is upstream right now to support more different integration because other SOC vendors does this actually. So HPD direction is also custom and it's normal because the IP lets do this. And the CEC is ended by custom hardware that can be supported because now the CEC framework has been moved out of staking. So it's the next good work. So patches are ready and we target 4.12 for this part. For the DRM planes, all the current situations, we only have a single primary plane. So when you move the cursor on Xorg, you need to recompose every frame so it's not the perfect situation. Don't have overlay so we cannot show video actually, even non-accelerated and no cursor plane. So we plan to push the scaling so you can use the primary planes and 1080p and the real output in 4K so it lowers your bandwidth. So this is in planned. Add overlay planes to support the classic YUV and NV plane support which is outputted by the video decoder. So the overlay plane scaling is necessary because we can change the video scale and add the cursor plane. So this is in work, but maybe after 4.12. On the audio part, the SoC supports SPDIF, I2S and PCM input output. So quite common here, though. So the basic I2S support has been written and we work on supporting the other input and output interface. And like you see, so yesterday at the demo, we have I2S working over HDMI so it's a good point. Everything works. It was like it was a worse situation but it's quite easy. So all this support is planned for maybe 1 or 2 release. One of the greatest work we need to achieve is video decoding and encoding acceleration. So it's one of the best features of these SoCs. They have a powerful hardware decoder and hopefully we'll need to really manage to have a proper video for Linux driver to support encoding and decoding and hopefully support VP9 10-bit decoding upstream. It's a dream, maybe it will happen. And I know the Codi and LibreLeg guys really want us to support this. It's one of the priority work for the next two releases. So finally, I will talk about the community around the upstream Linux because it's important. We didn't make this alone. We have had a lot of help from developers. The Odger with CDE is a well-liked board and people prefer this board instead of the Raspberry Pi. It's started because it's more powerful. It has better output, a better GPU. And ArtCanals provides nearly usable desktop. So it's well known. And because this is open source, a lot of projects that were modified for Raspberry Pi work very well on this platform because they all share the common pitfalls of ARM embedded platforms. So for the hacker community, we have a few hackers who push regularly and test the main end patches. It is really fundamental for your work because you can test everything alone. And for them, a USB support was made by Martin on this part-time and pushed as well the ADC which is small but really useful for this kind of SOC. So they keep pushing some great test features and we thank them for this. So I finished my presentation of the current situation of AMlogic SOC support. So if you have questions, don't hesitate to ask me. Please repeat. So is there an estimate for when you might have Mali820 support in the kernel? As a kernel module? Or anything. I started working on the kernel module actually. So the code to support the 450 and the T820 was nearly similar, but I need testing. But I don't have the user space library to test so it's quite limited. I will see the hardware initializing but nothing to use it actually. So Sunday hopefully. Can you provide a brief description how we can update to a newer version of Mali driver? Let's say you have an old driver. You want to update to the latest or newer driver. So what parts are involved? What companies are involved? So what parts are involved in it? Okay, so for the Mali support you need to have the kernel driver and the user space library that has the same version. It's very important. So for this you need to have the SOC vendor to build the last version and so you can take the last test GPL driver. So you depend on the SOC vendor to actually take the new version from ARM change it for the hardware and build it and distribute it. So it depends on what is the status on the SOC vendor actually on Mali. So this is going to involve three parts. The first one is kernel driver. That's from ARM, isn't it? It's from ARM but you have a platform integration support a small C file to actually adapt to the hardware. So this must be imported from the previous version actually. The second part is Mali.SO. Where does that come from? From the SOC vendor. Okay, then there's a third part that is the for example it's SORG, like framebuffer.library. Yeah, it's not really tight too. It's kind of generic. But yes, for Mali, for XORG acceleration you need a special driver. So you have FB turbo on all-winner SOCs. You have the ARM SOC driver used on plenty of ARM SOC because you have the same situation with power of year actually. So it's yet three parts but you need to find the proper XORG driver that works well. You need to find the last test, SOC vendor distributed library and make work the driver, the GPR driver. The XORG part is generic. Will it be enough or has it been specific from an SOC provider or similar party? Sorry? For the XORG part, will that be open source generic? Yeah. Code will be enough? Yeah, yeah. ARM distributes their own XORG driver called Mali driver for XORG but it depends on the UMP driver which is not always built in the char library and the GPR driver in the kernel. So this situation is very complex. Plenty of different parts that's not really fitted together. So it depends how the S3 vendor builds the char library which is unknown. So you don't have any question more. I will stop here. Thank you very much for attending.