 Okay. Good morning, everyone. My name is Hans Drude. And I am a software engineer working for Red Hat. Among other things, I'm working for the X-server. So, of course, we couldn't get the beamer to work. That makes a lot of sense. So, this kind gentleman in the front borrowed me his laptop. Thank you. In my spare time, I work on Uboot and kernel for all-winner SOCs, all-winner ARM SOCs, which are used in budget ARM tablets mostly. And that's what my talk today is about. Specifically, it's about hardware auto-configuration. And then, specifically, hardware auto-configuration on Q8 tablets. I'll quickly introduce what Q8 tablet is and what problems are with these devices, why we cannot just have a single device tree to use them all. Then I'll look a bit in hardware auto-configuration in general. And after that, a conclusion and then there will be a lot of time for questions and answers because my presentation isn't that long and I hope that this is a topic which will lead to some interesting discussion. So, Q8 tablets are very cheap seven-inch tablets like this one. This one is currently running GNOME 3, but it's using software rendering, so it's sort of working, but not really. But these devices can be had for $40. They contain for $40 nowadays if you buy the current generation. They contain a quad-core processor. And the sheet says half a gig of RAM, but they actually have a gig of RAM now, which is good because half a gig of RAM can be a bit constraining. So, the latest models have a gig of RAM if you look carefully and for $40 quad-core gig of RAM, LCD display actually, battery, Wi-Fi, things like that. It's a really nice device for $40, but you get what you pay for, right? So, you have to keep that in mind. The problem is that since these devices are made for only one reason and that is to be cheap, and if you buy a tablet and two weeks later you buy the same tablet, the internals are going to be different. There will be a different batch of accelerometers in there, different touchscreen controller, different Wi-Fi chip, things like that. So, the question is, since I want end users to be able to run plain Linux on these devices, I've been working on that for the last couple of years, which I want to discuss with you today is, is it possible to get these tablets? I have 16 of them at home. I have one with me. 16 different models. Is it possible to get these tablets to run upstream Linux kernel using a single device tree for each SOC family? So, the three SOC families, the single core, dual core, quad core, those of course need a different device tree because it's an entirely different SOC, but is it possible if the user picks the right SOC because he knows which SOC he has to auto-detect the other things, the accelerometer, the touchscreen mostly? So, on to the touchscreen. The touchscreen actually is the biggest problem. There are four models of touchscreen, which I have seen in the 16 models I have. Two revisions of the CYLEED GSL1680 controller. Those are the nasty ones because those are in almost all the tablets, but they're not wired up the same in all the tablets to make things interesting. And then there's the ELEN and the ZYTECH, which are only found in one tablet I have. So, they are sort of easy because if they're there, then I know which firmware found to load and things like that. So, detecting which of the four models is present is actually easy. You just bit bang the I2C bus and if something is listening at an address, then you can read an ID register on most models. And if that matches, then you're probably talking to that touchscreen, probably, hopefully. And the GSL1680 actually has a revision register, so I also know which revision I have. So, those parts are covered and are easy. But the hard parts are, as I already sort of said, that the CYLEED GSL1680 is a very flexible little chip. It's found in 14 of the 16 tablets I have access to. And it has configurable pin boxing. It doesn't just have pins like X1, X2, X3, X4 for the horizontal detection and then for the vertical detection pins numbered I1. It just has a number of input pins. And they can wire them up whatever is convenient when they're routing the PCB and then fix it up in the firmware, the routing. Which means that even if you have the exact same digitizer, you can swap the digitizer in these tablets. The digitizer is like $3. So I have bought a number of them secondhand and just put in a new digitizer because the screen was cracked. The digitizers are all the same. The chip is the same everywhere. But still, there is a lot of variance between tablets because of the pin boxing thing. So I need different firmware files because the pin boxing is part of the firmware and we have no clue what's inside the firmware. It's a block. Even if you have the firmware file right, some firmware files have the XI coordinates swapped on some models. So then they actually have hooked up all the pins the same but they have numbered them mirrored. And another thing is that the resolution which the device reports is part of the firmware setup. It does some scaling in firmware too. For some reason, a lot of these devices on Android actually try to make the touchscreen coordinates match one-on-one with the display resolution. Even though there's no technical reason for it, maybe Android at one point required that or for some reason they're scaling whatever native resolution they have to the display resolution. So that's all things which means that on each tablet, the touchscreen node in the device tree pretty much needs to look different. Which also means that to jump ahead a bit that the device tree overlays are sort of out of the picture because you get such an ex-plosion of combinations with the X inversion, the I inversion, the firmware file, the resolution, XI access swapping also happening. So yeah, we got a lot of combinations so using device tree overlays not really an answer here. So the touchscreen resolution is that actually I've just been trying firmware files from one tablet on other tablets. I can bring the variation in firmware files down to two per revision. Unfortunately not one so I still need to figure out which firmware file I need to load on a single revision. And so far on the 16 models I have access to, I have a little heuristics where I look at the accelerometer and the Wi-Fi chip which is present and then I pick a firmware file and if I need to swap or not and of course there is no correlation between or no causation between the two, right? They can easily put another accelerometer on there and my heuristics will no longer work. But for now I have everything running on all 16 models without needing any override on the command line. I do lock all the settings I detect and all the choices I make and there are kernel command line options to override them for users. But so far I'm getting away at least on 16 models with not needing to use those command line options by using heuristics based on other bits of the tablet, other completely unrelated bits which happen to have some correlation. The accelerometer. Accelerometer is interesting in that I almost have 16 different accelerometers in 16 different tablets I have. Almost. I ended up writing three IIO accelerometer drivers for this. They are being upstreamed currently. Again detected by just poking the I2C bus. So just while you know the addresses they listen at, some can listen at multiple addresses depending on some pins which are pulled up or pulled down. So you need to try them at multiple addresses. A problem with the accelerometer chips is orientation because they can either be on the top side of the PCB or the bottom side and then they can be rotated. So far it seems that most of them but that's also because I only have a few which are used in more than one tablet are used with a fixed orientation. I still need to add support for that. There is actually already a device tree standard for IIO devices to tell how the sensor is oriented versus how it would normally be if it was mounted exactly the right way. And yeah so we can just basically for almost all tablets I have I can just use a fixed orientation. There is one sensor I think the last one the MXC6225 which is mounted different on two tablets I have but there I can sort of do the reverse. I can use the touchscreen to determine the accelerometer orientation. So yeah this was a fun little project. And the end result is this on one tablet. Not sure if it's really we can zoom in a bit more I guess. So this is the Dmash on one of these tablets and this is my Q8 hardware manager which is said I basically says I'm looking for a touchscreen without a regulator. Oh that's also a thing. Some of the touchscreens are powered by a separate regulator. So if I'm probing the touchscreen I'm doing two rounds. First I'm looking for a touchscreen without powering up the regulator and then if I'm not finding anything I'm powering up the regulator and then doing a second round. Well it basically here you see just that it's pretty chatty but that's because well it might be wrong. So it's trying to give the end user info and some hints like try using these comment line kernel comment line options to fix things. And while this code is pretty much ready for upstreaming now I just haven't got around to submit it upstream yet. Although I have submitted an RFC version and I had to change the device tree binding a bit but a little bit more on that later. Now I need to zoom out again. So hopefully more interesting for you is what does this toying around with Chinese tablets I have been doing mean for hardware auto configuration in general. Well for one it's possible. If I can do it without any control of the hardware then if you have some control of the hardware you can talk to your hardware division internally hopefully then it's definitely possible to deal with multiple board revisions with a single device tree file. And I also think that that needs to be the way forward because nowadays you see a lot of second sourcing especially if products are made in bigger series and things like that. So you really need to stop thinking about a single static device tree for a single board because with board revisions or even the same revision where a chip has been swapped for a second source you need to adjust to that. And that's also what I hope we can have some discussion about. So as I said before I probably won't be filling my time. General hardware auto configuration I hope that you have some control over the hardware that you have are working with a product which is designed in-house and I strongly advise you to start talking to your hardware division as early as possible and really tell them we need some sane way not what I'm doing on a QA tablet some sane way to uniquely identify the hardware and if chips have been swapped for second sources which chips have been swapped etc. So probably there will need to be a small EEPROM on there and someone is going to complain it that brings up your bill of materials that's going to cost money but you just really need that to do this properly without crazy stuff. If you have a lot of GPIOs free you can do something with GPIOs. Something like that. But an EEPROM gives you a lot more flexibility. So I actually said it's or so tips are make sure you can identify the hardware but not just that you can identify the hardware but also that you can get the revision of the board. And again if it's possible to swap chips on a revision also that you can get that kind of info. If there are add-ons so if you look at the Beaglebone which Pantele has been working on and his work has been a great help for me I pretty much the device three side was done. I just needed to use his code. Then you really also want a way to enumerate the add-ons and again you want to be able to get a revision and things like that. You just we in the embedded world we in the ARM world device world need to stop thinking static. I think that that's a mistake that everyone is still thinking the hardware static hardware is not static. If you look at how people are using development boards like Raspberry Pi what they're top they're piling on and the chip also has its own extension bus and things like that. We really should stop thinking as device three as a static thing. It's not anymore not in reality. So well if you're not in control of the hardware then good luck. I'd hopefully you could do something creative as I did. So applying the device three changes I already talked a bit about this if you have add-ons right so you have like a development board and you have certain boards which you can stack on top then usually you can have a device three overlay per board right so you have some device three overlay manager which identifies I have this add-on or stacked on top via some detection mechanism and then you can just apply that. So device three overlays are good for that but device overlays are very static. So it's also possible and I did that because of all the different touch screen things I needed to deal with. It's also possible to actually just generate sort of a patch to apply to the device three in C code which is actually really nice and it's it's easiest to just show that in code. So here's a small snippet of code. Basically you declare a struct which is called an open firmware change set. Well in this case I also declare a device node pointer because I need to apply the changes on top of something. Then in my case I use find node by name. That's actually interesting because I started with adding a specific compatible to say like this is a special Q8 hardware managed node and then the device three maintainers one of the device three maintainers Rob basically said like no no no because you're not describing hardware you're describing software just so I ended up using find node by name normally that's actually sort of frowned upon but in this case it feels like the right thing to do. So I'm just going by the board compatible and if the board compatible says it's a Q8 tablets then my hardware manager will initialize and it will try to find I'm using template nodes. So I have a template device three node for the touch screen which is basically empty. The most important rule of it is that it's below an ice for a sea bus. So it will tell me which ice for a sea bus I need to use for probing because luckily at least they always use the same ice for a sea bus for the touch screen and the same ice for a sea bus for the accelerometer that that bit is fixed. I don't need to walk all the ice for a sea buses. And then you just get you in it to change that I add a proper integer property and string property. In this case I also change a string property. I'm updating the status property so that's the node actually gets enabled and then this code which the content has done a lot of work on I assume under the hood does a shitload of stuff and it dynamically up generates the new platform structure platform device and better strategy to put the platform bus and everything or ice for a sea bus in my case and everything just happens. Which is great. And of course don't forget to free your resources. So this is basically the non overlay site. I don't know how the overlay API in kernel actually looks because I haven't used it but I'm sure it's shouldn't be that hard to use either. Yeah. Yeah. But you're you're you're not if you're doing an overlay manager in the kernel you're not directly calling these functions right you're just loading the overlay and it's generating a change set from the overlay and then you apply it. So conclusion. As I said I won't be talking long. I'm hoping for some discussion. Well I started with the question is it possible to dynamically adjust a single device tree file per sock family for this tablet so far. Yes it seems to be possible. It's a bit heckish but it works. Do's and don'ts learned. Do start thinking about this while you're designing your hardware. And then I'm repeating myself. Do not leave it till after the hardware is complete which is saying the same thing again because it's important. Really something which if you want me to take anything home from this talk it's you're probably going to have to deal with hardware revisions during the lifetime of a product and you really want to be able to detect that any changes you want to be able to detect them in the same way. So yeah do make sure your hardware revisions variants are easily uniquely identifiable. And one thing which I learned is don't add a note to the device tree to represent your hardware manager because the device tree managers don't device tree maintainers don't like it because the device tree is meant to describe hardware. So what you can do what I did is that you you trigger the activation of the manager based on the board string or the board compatible or the model string. And that's pretty much it. So yeah only 20 minutes. Sorry. And as I said I hope we'll have some good discussions. Yeah go because device tree if you have a simple bus instance it's platform devices. That's how it works. I could interior create my I squared C client except that my drivers have a device tree binding so they expect to find things like which firmware file to load with resolution etc in the device tree. I don't think they're respinning their current I think they're respinning their entire firmware image. I mean this is just they just they go to the Schengen market they see which real width accelerometer chips they can buy. If you actually if you look at the PCBs there are like three or four foot pins for the accelerometer. So they can decide the moment they start pick and place in which accelerometer they're going to throw on there. Sort of yes. Sorry what. The device tree maintainers are sort of accepting that I'm doing this but they're not really happy they would like to see some sort of generic hardware manager. So maybe we will get a generic binding that if you're smart enough to put an EEPROM in there you can you we also have some mechanism called device tree quirks. Not sure what the upstream status of that is. So what could be is that you start using quirks and then we get a generic mechanism and you can just declare an EEPROM as this contains your quirk info or whatever and then. No no it's it's a very interesting discussion but I don't want to see bytecodes in the device tree. Yeah it's especially for me since I'm near the yaw in a new boot maintainer so I can just just sneak it in. Yeah that's the that's all we're themselves are doing but for for running upstream stuff on it like like the fedora I have running here it's it's running regular you boots and we could definitely do this in new boot the interesting thing is when I was discussing this with Maxine repart who does the the kernel maintenance of the all-winner bits he said I think this is better in the kernel and one of the reasons is we have some more infrastructure in the kernel for example the kernel needs to probe the Wi-Fi chip already anyways right because we want to have the Wi-Fi chip when running so and we might want to make decisions depending on if the Wi-Fi chip is found and things like that and I have a feeling that if I move this to you boots the code is going to become significantly larger basically generated device to overlay that that that feels weird because this this works fine you mean you mean do it in user space but how's it looking is it going to get accepted if your route there are not ways to break a device anyway so not in this case because in other cases it will definitely be I think that the bootloader versus kernel space question is more interesting because also of course if you really have a case where you need to patch up device three to find your root file system then of course kernel could still be just in time but if we're talking root file system that you can of course imagine some sequence where you need to have picks up your device three before the kernel boots far enough to do this in the kernel yes but I don't want to see this as a redhead or even a generic distro specific problem right it's it's why better than that for the embedded case you usually don't use in a different file system well each day Yeah. I did that for this place. I put in all the displays you can detect. And I call it like it's running in cats, so you don't know which one, the determinant or not. But it's OK from that note, which is unordered. And it was next, because it's a maintainer stuff like it's actually in the system in the device. Which maintainer next to that, Linus? It's a maintainer, probably, like, what is it? Frame where? It's a frame buffer maintainer. OK, so it was in the device, which is a maintainer, it's a frame buffer. Yeah, but it was on the list. OK, that's just curious. But actually, for our process, you have a similar thing. You can either use, for instance, for PCI Express, you can either configure it as PCI Express 5.4, or just use the lanes. So you've always some NL and some digital properties. And where's the difference? Where to draw the line? Yeah, but that's not something which, hopefully, not something which you're changing on board provision, right? So that's just fixed for a specific board. We're starting the enable and disable discussion. So it's just, if you have that display, you can't have the other one. So it's the same thing. If you think about the lanes, PCI Express, you can only have X4 or you have the same lanes. So if you think, OK, you have this display and you can't have the others. So it's the same. Maybe another number of combinations, right? So PCI lanes, they're only, it's a natural thing. You have only a few. But this place, it could be, like, it's totally independent of the actual board. That's... Basically, I use it for the same. So it has also enabled and disabled bits. And the biases are very generic, basically, and for instance, support a number of CPUs and then it goes into this. But it's for very specific hardware parts. Yeah, that's to answer your question specifically why I didn't choose to do that for these tablets is that I have this combination or explosion going on with the touchscreen, right? I have the firmware file and then I have... YX inverted is XX inverted. So right now I have 16 tablets, but each time I buy a new tablet, I get a new combination. So it's basically the same reason why I'm not using device tree overlays because then I would need to generate device tree overlays for all possible combinations of is the XX inverted, is the YX inverted, are the X's swapped. So you get this combination or explosion and then just doing it dynamically is so much easier. No, but the problem is I have, like, say, let's say I make a single node for the GSL1680 revision A, then in that node, I need to set a firmware file name and I'm actually allowing to set a random firmware file name on the kernel command line because if a new tablet with new firmware files up, the user can extract it and can drop it and can just override the firmware file and on the command line. So that's already sort of makes device tree overlays and fixed nodes not work because the user can pick any random ASCII string, so I need to generate pre-entries for all possible ASCII strings, which are a lot. And we have the same thing for resolution. If a new firmware shows up, it may have another resolution and I support, so I would need to sort of make a list of all possible resolutions and then get multiplied, right? If I had fixed nodes which are only enabled or disabled, I get number of possible firmware names times number of multiple resolutions times X inversions, which is two times. So the combination of explosion is just not manageable. Sure. Would have added the eProm you requested and handling all the variants. How would you do that? Probably still the same because the eProm could then contain a device tree overlay in an ideal world. Assuming that device tree really is the perfect ABI we all wanted to be, but which it really isn't. But if you're already picking and compiling, then why first compile it into a device tree overlay and then have the overlay manager apply it? Why not just directly apply the changes to the device tree? You're only adding complexity and code and you need some temporary storage somewhere and... I know I still have the code, but the code now generates a device tree file on the device tree overlay. It could unless I need to do this before I have access to user space for some reason, right? Yeah, so... I think the more interesting question would be why not do it in your boot, but it also depends on how much control you have over your bootloader. Yes. We don't care about other things. You might not. It's open source that can use the same methods as we do. It's open source that can use the same methods. We create an ill-defined set of methods because they get DT and now our method as described by us is look at making that full-stream to identify the board and then figure it out. That's a really ill-defined... Yeah, but that's a... If you then do it in your boot and then what about the people that don't run your boot? I mean... Yeah, that's an interesting one. So I have a feeling that one of the biggest topics here is I think we all agree that we need to do something where we adjust the device tree. I'm hearing some people say do it in user space, do it in the kernel, do it in the bootloader. Oh, okay. Yeah. Yes. So I have the feeling that if I submit this patch you're going to naked. I know that I need to move it to U-boot basically. No, no, no. It's... Well, then you would also need to say if it has this value and if it masks with this mask if it then contains this value. Because for one, not all iSquad C chips have an identification register. It's something which vendors decide to put in there. It's not part of any standard. Actually not all iSquad C devices have the same protocol to get to a register, right? iSquad C just has either your writing data and it's a write stream or it's a read stream. So normally to get to a register you need to first write an address and then start a read stream. And even that part is not standardized. How you get to a register. That can be vendor specific. Yeah, you know, you... Where does it seem like a mechanism for, you know, discovery at all? You have to say on this board I know that I can identify by reading this IGC address on this class in this format and here's what the data looks like and then you have a way of mapping out the data over there or something. Yeah. Yeah, I'm fine with that. You need to look at him. Yeah. But why don't we apply him in the bootloader then? If the bootloader is going to look at that DTP yeah, most of the people mess with that DTP. Bootloaders are pretty, pretty diverse you don't know when it puts out short bits of something really stupid. I think what he said is you tag it on to the end of the DTP so the bootloader would know it has to be different. Nothing. It puts in the memory size but it's not very smart. We're talking about I would have to take a look I'm afraid now. They will but only the base DTP. You don't know what they're going to bunch. If you have to pen something to the base DTP you don't know that the bootloader is not kind of when they munch it over right. Well, you can put them over there. You can actually get all the bootloader queues to agree with each other. If you do that, then I'm expecting it to be easier for us. There's one thing there's a lot of the drivers like I squared C that touch me it's a very basic information and if they don't find anything they can deal with, they should just get out the trouble is that when you've got overlapping devices that are much easier to get out. There are devices that would do stuff when you needed it but if you know that there's a set of devices that you've had you could actually say these are set of devices and they are at the point where they would be accessible in the same way that's not common is if you have devices that you've had to walk by and you've got to work and you're but you're also hitting the problem there for example there's one of those accelerometers it actually has an I squared C on an echo and if you do anything do that I squared C echo then the bus is dead you're underestimating how bad hardware is well it's not a solved problem but it's a known problem if you're talking about the CPU itself you have the same thing I don't want to deal with this massive comb and turtle mess and they don't necessarily know on the baseboard why does the CPU sound to be added it's just the same problem as the generic expansion connector for that board problem you don't even need to see the expansion order, you could use it but you need to separate it when it's sold without I think it's a big problem Any more questions? Discussion items? The device tree the information that the whole boot process is not really a big thing it's not a one good thing but we have to say you don't have a second but yes you could use that so we for a lot of our EVMs to run into kind of a similar problem especially for different board revs and I guess the problem that we have is now you're kind of coupling when you start doing a lot of this dynamic editing of the device tree now you're really just coupling it into kernel where you do it in a boot loader you're really coupling in the code with the actual version of the DTP so how do you handle having old versions or new versions of the DTP that you have to go in and dynamically modify for them? Yeah right oh yeah that's that's the official answer the unofficial answer is that at least for all winner devices we luckily don't have although there are plans any devices yet which actually come with the DTP burned into the firmware so we always load the DTP from the same medium as the kernel and they get updated in sync basically so although we try to keep the ABI stable if we accidentally break it it doesn't really matter because it will basically we have a version DTP directory and then a version kernel image and they get matched in the bootloader compact with that kernel which is actually also a good reason why to do it in the kernel by the way the whole this is actually an example of so this was one thing where the the rack changed the reflux actually in the empty area but that was one thing so usually in the in the DTP there are a lot of new versions of the DTP and there are a lot of new versions of the DTP but that was one thing so usually having this actually a good example because I can do the latest kernel with an ancient DTP that was created three years ago and it burns and yes those same happen I'm just wondering if you had some sort of vision or question information in the just in case you do need to put something that would be there if somebody switch to a row and making impactful changes they're not going to remember probably the version string anyway that's a good one yes I think you actually my decision structure should take the CFC of the tree but the tree changes too frequently you can't even you can't do that now this is something you need to sign but what do you really mean why doesn't work signed DT and not what the point in that sense which something's got wrong that's not how it works it's a it signs the the entire tree yeah that means you have stuff which is not very dynamic yes oh yeah that was by the way another way to do the hardware management in the kernel because people tend to burn the bootloader once and then be happy the board boots and you want to update this for newer revisions of boards and things like that it's a reason to do it post bootloader yeah so user space would still be an option so you get your bootloader to boot something which does the hardware auto detect and then modifies the ddb and then boots the real kernel so we have one more question here at the front