 Eight device tree talks this week and know all about it and don't need to hear any introductory material. Is there anyone who doesn't know anything about device tree and needs an explanation from A or, okay, and so this is a pretty small audience and there's no point in my just yacking on and on if people don't know what I'm talking about. So I don't see any reason for people not to interrupt with questions. Obviously I'll cut them short if they're going on for too long. Particularly if I'm using acronyms that people are not familiar with, which I have a tendency to do. So there has been a lot of discussion about the device tree. There was a session at the Kernel Summit, which I didn't attend, but which we heard a little summary of from Mark Rutland. In his device tree, The Disaster So Far presentation yesterday, which I greatly enjoyed. I'm going to take a little bit more optimistic approach and try to talk about the bright future of the device tree. There's also an excellent introduction, I guess maybe in the same room by Thomas Pedazzoni and will largely not repeat what Thomas said unless someone really is puzzled by what I'm saying. So I'm coming to device tree from the world of automotive. In the world of a lot of consumer electronics, devices are kept by the user for a year or two years maybe and traded in. And carriers are not always providing kernel updates, even if they're available. And phones and tablets now are almost considered consumables by the people that purchase them. A car is not in that situation. A car is a major purchase by somebody. People expect parts to be available for 10 or 15 years. And I think they will be expecting security patches even though they won't call them that, probably for that amount of time as well. And the whole concept of updating software and cars as part of regular automotive maintenance is new to the auto industry. And the software production and maintenance model for automotive and other products that are going to be installed for a long time, like maybe security systems or ATMs. All the different traditional uses of embedded is quite a different model than the tablets and phones that have dominated the thoughts of a lot of people and embedded the last few years. So this is just the beginning of some ideas about how if we have a product we expect to be in the field for 15 years, we might approach the problem of updates and maintenance when we introduce the device tree into the picture. So, updates of fielded devices are already pretty hard. I don't think I have to tell anybody and embedded that. I think device tree makes it, maybe from some point of view, optimistically a little bit better and maybe from another point of view, harder. Maybe we can make our life a little bit easier by thinking in advance of what are good practices to make the update task tractable. I'm going to talk a little bit about tools that are imminent and tools that are wanted and a little script I made to try to help and I'll try to summarize best practices. So this block thing in the upper left-hand corner is some text that KDE and Debian have decided put on my screen that I can't make go away. So just ignore that. The rest of this is a diagram that's no particular rocket science. It's just kind of a state machine for deciding if an update you're trying to push is complete and coherent and makes sense. There's nothing particularly brilliant or novel about this. The point is just that the state machine involved in updating a root file system or the kernel or both in a field of device and making sure that they have a correct checksum and perhaps they're signed properly is already kind of complicated and painful and now potentially with the device tree we're adding another file that has to be kept in sync with the kernel or maybe we can update the kernel without the device tree. That's something that's been discussed a lot this week already. And so as I mentioned I'm thinking about this from the point of view of automotive and providing long term maintenance and support to customers as part of building a kernel and a root file system for them. And so I do believe that cars will be getting security patches and that they may require new device trees. A lot of people are pushing the idea of device trees as a binary interface between the, it's a contract with user space as with the kernel. The binary interface of the device tree and so maybe we won't have to update the device tree when we update the kernel with security patches. But this black thing is really annoying. But point is is that cars I think will probably want them. And I think that's just practical. I'm not going on a big campaign that we should all look forward to changing out our device trees. But if you're in charge of actually making things work with real products in the field, you find that you do a lot of things that people don't necessarily recommend. So one aspect of cars is they have many processors in them. Just an example that occurs to me where I think people will probably end up updating the device tree as well as the kernel is. If you look at the battery pack in an electric vehicle, it's extremely complicated charging circuit. The batteries are a very complex system with many microprocessors involved in just measuring the state of the battery, cooling them. And so the battery warranty for a brand new Prius in the United States this year is five years. And that battery will certainly be replaced at the end of five years. And I think the technology and the new battery will be quite different. So this is an example of a case where software is going to need, perhaps, to be upgraded. Another point with cars, even from the point of view of the so-called infotainment unit console in the software on that, is that that will control the climate system. And the climate system in an EV runs directly off the battery. I mean, you're not getting heat from the manifold the way you wouldn't in an internally combusted car. So all the driver comfort features are taking the same power that's used around the engine. So besides even security updates, I think we're going to get power updates. And the features of the kernels that go in as far as power and security are always changing. So device trees are supposed to be a hardware description. I think the hardware in cars in the field is actually going to change because of these different dynamics. I think even if people can update the kernel without updating the device tree, their desire to get power and security features will maybe cause them to change the hardware description, even if the ABI is stable. And then another point is, to be perfectly honest, if you look at the device trees we have now, there's a lot of configuration information in them already that is not really a hardware description. I mean, something that I've used, for example, is MTD partition tables. No one can tell me that a partition table is a description of hardware. I mean, a partition table is kind of sort of a description of a real entity on the storage device, but it's not like it's the address of USB or something like that, and you can change it. We have pinmux, resource, and clock stuff in the device tree that we might change. We're selecting boot devices. A lot of this is configuration rather than hardware description. So anyhow, I think we are going to change the device trees. And actually, this is kind of hard. And why is it hard? Well, one aspect that I referred to a little bit before is that cars have a lot of processors in them. They are really rolling data centers. If you haven't kept current with this technology, it's really kind of daunting how complicated it is. The new Mercedes S-Class, according to Mercedes, is going to have 200 microprocessors in it this year. That's astonishing to me. Now, a lot of these are 16-bit microcontrollers that just run a PID loop on the brakes or something like that. And they're not having Linux on them, and they're not going to anytime soon. So they're kind of out of scope for this discussion. But I know for a fact that a lot of the cars, actually, Mercedes is QNX, not Linux historically. But I know that a lot of cars are going to ship with more than one processor of running Linux in them. I wouldn't be surprised if there are cars coming out in 2014 model year that have half a dozen Linux running processors in them. And it may be that only one of them is going to be connected to the internet. So maybe if you're getting that new battery pack, it's getting boards with it, and those boards will have new software on it. But some of these other processors that are part of this data center, this rolling LAN, may need to be updated over the LAN locally by software that's loaded onto the one processor that's accessible from a USB drive or from the internet. So this procedure is really kind of, it's really kind of configuration management of this whole system. I mean, it's almost like enterprise computing in some sense, rather than traditional embedded. Another part about the device tree update that I think a lot of people are aware of, but that people who haven't made heavy use of the device tree won't have thought about a lot, is that the device tree and the kernel configuration have to be kept in sync. And this is kind of obvious, but if you enable a feature in the device tree and you don't build the driver, then the device is obviously not going to work. But this actually can be a little bit tricky. I've worked on something where a multiplexer or a pinmux or clock configuration needs to be changed to enable a particular device in the device tree. And so then in order to use that device, you not only have to build the drivers for that device, you have to build the drivers for the bus it's on. So and if you do update the kernel and the device tree, the names of some of the parameters may change from kernel to kernel. So it's going to be important, certainly, in the future, as a bare minimum to preserve the information about the configuration and the device tree together that you have. And if you have a compilation of the device tree that relies on a whole bunch of include files, you'll have to save all the include files and all the device tree files and the kernel configuration all as kind of sets. And if you have a whole bunch of different products, you're going to have to be very careful that that information is really, really well organized. Speaking for a personal experience, which doesn't involve anything like 10 or 15 years, you have to keep pretty good records of this stuff. So this is just a, people say to show pictures in your talk. So I thought the pictures you'd like to see were screenshots of LKML. But this is just an interesting post from somebody named Jason Gunthorpe, who says that he has basically a script that checks the K config against the device tree. And so that sounds like a good idea. And I guess he hasn't published that script, but that's the kind of example. I'm going to talk a little bit more about tools in a little bit, but that's an example of a kind of tool that would really be useful for the community, although it would require a lot of maintenance. So this says unintentionally under this obscuring black banner. Oh well. Another point about the device tree that was not obvious to me at first is the ease with which one may unintentionally break something that was working. So here, this is just an example that Koon Kui posted to, I guess, the arm mailing list. So it's pretty obvious if you go into the node that corresponds to a particular device and you change the parameters, you're changing the behavior of that driver. I don't think anybody's unclear on that. But what's less obvious is your driver is using resources like the pinmux, its regular voltage, or its clock. And if you make some apparently innocent patch to the device tree to enable another device that's using the same resources, you can break your device without thinking that you're changing it. And this is a little bit of an alarming example that Koon posted where basically he pointed out that he had an unsupported device that no one was using. And when he added support for another device, it could basically blow the one device, the other device up, which sounds pretty alarming. So you can easily imagine that your board has some little chip on it that you don't even have a connector to, that you're not even using. And you're going to give your user a new device tree to turn on some feature that you do support and you're actually going to alarm them by having smoke come out of this other chip. I mean, that actually honestly could happen. I mean, I had another example where I changed the behavior of a device that we didn't change the driver and therefore we didn't test by changing the device tree and rendered it non-functional without even knowing it. So that's just kind of a word of caution is every time you change these clock, pin marks, power supply, and probably some other things, you need to look at the whole device tree to see all what might use those devices. So there is hope about this. As I said, Mark Rutland's talk yesterday was kind of a disaster recovery. But I do think things are getting better. I'll talk a little bit about what we might do to preserve information and do updates with a few more files. One approach that was posted by Joel Fernandez, presented by him at the California ELC last winter, is the flattened image tree. So the flattened image tree is just a little file that describes a kind of a manifest that can record things like the versions of the device tree, the kernel, and any miscellaneous binary blobs, and the cyclic redundancy checks sums. So this is really handy. While I might wish that I did a good job of keeping records, this method of building an archive kind of has it all right there. Obviously, you could also use BitBake recipes or an Android-like repo XML manifest as well. But some type of build system that calls out all the files in one place is great because you know it actually built that way at some point in the past. The other nice thing about the flattened image tree is you have this short little ITS file, which has a syntax much like device tree, that you can compile with the make image usual tool and with the device tree compiler DTC. And it creates a single archive that you can use for an update that has all these files in them. Obviously, there are other ways to do this, but this one is really simple. And UBoot now has a facility for handling these flattened image trees. And thanks to contributions by Chrome OS, who uses this mechanism, there's now a facility for signing the flattened image tree archives. So as far as solving multiple problems at once about security, about doing updates as atomically as possible, and as far as preserving all this information, I think this approach is actually pretty attractive. So hats off to Google and to TI for making this available. It's something that a lot of people may want to look at. As far as I know, there is no ability to use this sign device tree blobs absent this flattened image tree. And someone at Google has told me that's really why in part they've adopted this approach. They want to have everything in the Chromebooks signed. So another as yet unmerged feature in the kernel that seems like it has some appeal for typical embedded problems is the implementation of runtime configurability via overlays. This is a contribution from Pantelus and Tonyu, who's been here this week and who some of you may know. Basically, right now, when you compile your device tree, you're making a binary file, which is in slash boot or on some auxiliary storage, and which is loaded by the boot loader and which is static. And so you can, for example, the flattened image tree have a multi-boot almost in the spirit of grub and load different device trees. But you cannot, at boot time, with U-boot easily, you can't change the device tree. I mean, it's a compiled file just like the kernel. And unlike the kernel, the device tree doesn't have modules that load. It's just static. So Pantelus's proposal is in spirit similar to having modules for the device tree, having fragments that can be loaded. So Pantelus created this facility to support the capes that go with the Beaglebone, which are these daughter cards that can plug in. So it gives a hot plug or kind of power on time configurability of the Beaglebone so you can attach different accessories to it. The Beaglebone recognizes its capes by reading an EE prom when the main controller gets an I2C event corresponding to the attachment of the cape. So clearly, this cape EE prom model is not going to apply to most of embedded. But if you look at the changes, well, if you look at the cape manager that Pantelus has created, which you can find links through his talk and his paper, which are both online. So these are hyperlinks in my presentation, all this highlighted text, by the way. And so you can pull his sources and try out this overlay for yourself. And for most use cases, you'll want to change the signal, which causes the kernel to load the fragment. Basically, you take some patches from Pantelus for the device tree compiler. And you can compile the short little plug-in files that he has several examples of into DTBOs, basically Device Tree Binary Object Files, which this cape manager that's watching for some trigger can then load and basically add to the device tree on a running system. So to me, as far as updates go, you could have a strategy similar to Union FS. If you're familiar with how NAPEX works, you get a writeable file system that the user can employ in a system that they're booting from unwriteable storage using Union FS. You could basically have a device tree binary in some ROM that you never change, that your golden fail-safe device tree binary, and you could just do updates via these overlays. I haven't really thought through whether that's a good strategy, but I would think for some cases, use cases, it might be. And then the third method of adding flexibility to the device tree and the kernel that seems kind of appealing was actually a nice talk yesterday about Zen on ARM. But the whole idea of using hypervisors to have collections of bootable images, and clearly then, if you try to boot one image and it doesn't come up, you can fall back to the old one. There's some ability here also to kind of preserve. You could have a long list of sets of files. It's just kind of a nice way of collecting information. And of course, there's a lot of ability with hypervisors to have a more secure system and to check signing of binary blobs and things like that. So whether or not you find Zen in particular appealing, a lot of people in automotive are going to hypervisors for security reasons. And this is another method maybe of both preserving information and managing the large perfusion of files we now are having. So what are some best practices for making the device tree more maintainable? Well, one that's been discussed a fair amount at the meeting is the device tree schema and using a validator with a device tree. As a matter of fact, Steven Warren of NVIDIA has been hard at work and has just posted last night some code for a validator to the mailing list. So the idea is just to, as you can easily do with web pages, any HTML or XML, you can have a kind of a DTD description that you can check if your device tree is valid. So that could involve, right now, the device tree compiler just kind of checks if your curly braces match or if there are any missing dependencies. It doesn't tell you, as Russell King was just complaining about on the mailing list, whether you've made a typo in the top level board file. The device tree compiler will just, device tree compiler doesn't have the level of checking that you're used to with C files. So there's a pretty large variety of mistakes you can make in it now. Not to mention just bad ideas, but actual mistakes that the compiler won't catch. And so a validator, as proposed by Perant and Kusan, would catch a lot more mistakes. I think we have a lot of opportunity with the validator. I was talking to Mark Rutland about this yesterday. But the validator could use a tree of schemata, I think the plural, as schema is Greek. And so we could have separate schemas, schemata, for different parts of the tree. And this would just be a way to make the burden of the maintainers who are reviewing device trees and the bindings a lot less, because they could just run the validator. It would allow people who are submitting upstream to make sure that they're kind of in keeping with the recommended device tree coding style. And it's an opportunity for the leaders of the various subsystems to express to the people who are creating device trees really what's acceptable in a way that is difficult now. And so if you did have a tree of schemata, then you might want to have the validator compile all of them into one top-level schema and compare that against your top-level device tree source. I was going to make a diagram of this to make it clear what I'm talking about. But basically, the validator would be enforcing the inheritance that's essentially going on in the device tree. There was a brief discussion on the mailing list about using XML as a schema language, which I was kind of surprised about. Lots of people like James Bottomley said good things about it. To me, that doesn't really make any sense, just because when I first looked at device tree, I was struck at how much like JSON it looks like. JavaScript object nucleation for those of you who have not heard of it. As a matter of fact, I thought it was JSON. This is just a little fragment of JSON right here. And you can see it's key value pairs, terminated semicolons, and side curly braces in a list format. Really, there is some question as to whether to describe properties in key value pairs in a tree, we need to create a new domain-specific language for the device tree. So the advantage of taking over JSON is that there's a huge, huge, huge number of visualization, validation tools. The device tree has been around for years now. It's got international language support. There are tools for it and C, C++. And the design of JSON, what its intention was to be a natural representation of data structures for the C family of programming languages. That is the problem we're solving. We are solving the problem for which this very popular, stable, well-supported solution exists. So I suspect that the train has already left the station about creating an entirely new tree structure format for device tree. But certainly, rather than pick XML, I would pick JSON, the fact-free alternative to XML, as its creator, Douglas Crockford, has called it. So just one more thing that's kind of a pet peeve of mine is what I would call premature optimization of the device tree. That is, if you think of the device tree include files as themselves a tree, there's no reason in the bottom-level file to make choices that can potentially break some of the top-level choices that you might make. In other words, if you have OMAP3, then OK. That's the fundamental choice for your platform. And I don't have any problem with saying OK, turning on the OMAP3 in your lowest-level include file. But if you have some peripherals that have resource conflicts with other peripherals and you only have them on some of your boards, please don't enable that peripheral in the bottom-level file. Please enable it in the top-most file where you know that you actually have that device and want to use it. It's just the idea that when you have a nested hierarchy of include files, you want to use that structure wisely and not put choices in the bottom-most files that are liable to create conflicts at the top. In particular, when you have a lot of these files and there's a choice that's bad for you in them, sometimes it takes a while just to figure out where it is. So the OK chosen and config on boot file options, those properties and the device tree nodes, are our configuration. They are choices. And unless there's really a good reason, like we're talking about the processor for the board, I would not put those in the bottom-level of the hierarchy. It's just liable to lead to trouble later on. So this is just a general argument about level separation, essentially. If you're going to have a board DTS file and a CPU DTS file, don't be putting choices about the data storage and the CPU one. And it is better to think of it those files that way. And hopefully, the validator may help enforce some of that good practice in the future. So a lot of the pain about device tree is related to tools in that the DTC device tree compiler is kind of new and not that featureful. It doesn't, for example, have a verbose mode. The problems I alluded to earlier of do my kernel configuration and my device tree enumerations match, there's no real tool to answer that. There's no real tool to do the reverse problem of, gosh, I have this binary that I like and that I want to change one thing in which of my 40 device tree source files do I compile to produce this. So that's why I suggested that we might want to use some of these manifests or recipes to preserve that information. On a booted system, we have no tool that reads slash proc to tell you what's in your device tree in case you wanted to know. Maybe if the device tree is really going to be an ABI, we can have that information in slash boot in a zip file like we have config.jz. There is an option, which is not in the def config for most arches, of exporting the device tree information to slash proc, but that's not on by default. So it would be great if we had visualization tools. And eventually we will need, I think, some type of facility for signing device tree blobs, which are not appended to the kernel, although that is at least a simple solution. So the tool that I've made for myself, because it solves an actual problem I have, is a script called Make DTS. This is just a bash script that runs the Cpre processor and runs the device tree compiler and produces source output just so you can read it and kind of get a sanity check. Given the structure of the device tree, if you make a mistake and your node gets in at the wrong level, usually when you look at the source, it's pretty evident. So that script is, this is just an example of it running. It produces an output file. This is x86, because I don't have the cross compilation tool chain on there. But this is the result of processing maybe sometimes 10 device tree source files and outputting it into one ASCII file that you can then read. So it resolves macros and compiles all the files together. There is a program called FDT, Flatten Device Tree Dump, that's in scripts in the kernel source directory. And that produces output much like that. But in that tool, these strings all come out as hex. And I find reading ASCII characters a little bit easier, especially when things are broken and I'm not sure what's wrong than reading hex. So this script is up at my website, shedevelop.com. My slides are also there. An older version of my slides without all these hyperlinks in it is at the Linux Foundation web page. It's also at slideshare.net, so anyhow. This stuff is all there if somebody wants it. It would be great if I also posted this to the mailing list. So this is a little bit, got a little bit more on it than this. This is a pretty humble thing, but it's useful, I think. It's useful to me for debugging. So I've got about two minutes left. It's really important for long-term maintenance to preserve the set of information of all the different systems, all the different files that go into your build system. And this requires maybe a little bit more thought and care now that we have the device trees, just one more thing. I think this Flattened Image Tree with a single archive for updates is really appealing and its ability to have every component signed and checked some is really a plus. And that's now part of UBoot, at least. The validator and maybe some other simple things like a verbose mode for the device tree compiler would really make life a lot easier. You can make your own life easier by having good practices regarding level separation and soon validation in the device tree. The overlays and maybe a hypervisor are appealing as safe ways to approach updates. And so device tree, according to Mark and many others, has been a disaster so far. And I think it has caused a lot of people pain. And maybe the worst doesn't quite over yet. But on the other hand, the whole concept of separating the detailed description of hardware from the abstract interface definition and the description of the methods that are used on the hardware is a good idea. I mean, that's a generally accepted, widely agreed software engineering practice. And so now we're finally abstracting the definition of the devices away from their configuration. And that's a good idea. So while the beginning of the device tree has been ugly, that's just because adding new features is always hard. And I think in the end, maybe in the imminent future, the device tree will actually end up saving work and making life easier, even though it's promised to do that has not yet been fulfilled. So maybe we have the disaster so far, but the happier future ahead. So with that, I've just about run out my time. If anybody has any questions, please pick up. So they tell me that either you can speak into the microphone or I'll repeat your question. Looks like it'd be easier for me to repeat your question. My idea of this is there's one thing I'd like to challenge because this means the idea of a firmware, whatever it means, put loader, of modifying the device tree in case when the hardware is changing, et cetera, et cetera, on its own. And that's the one thing I'd like to challenge because, hey, the device tree, Latin device tree, is a direct descent, it's a counts from the area of the open firmware architecture, where by definition, the device tree was actually constructed in runtime by firmware based on the real configuration hardware. And the other thing is that it's actually very easy to modify the firmware. And even put loader technology like you are doing this today, just using the EFTT. So it is definitely an option. So in case of your changing, because in one of the businesses, we'll have to update the device tree because the hardware has changed. And I'm perfectly fine with this. The device tree by definition is the description of the hardware. If the hardware changes, obviously you have to change the device tree as well, right? But it doesn't necessarily mean that it has to be somehow detailed by the kernel, or it doesn't necessarily mean that you will have to update the device tree load, which is there. It can be modified by firmware. Moreover, if the hardware by definition can change, maybe firmware should generate the device tree in the business. So the question is, what about some alternate methods of updating the device tree blob perhaps via firmware? There also was a very nice presentation yesterday. I thought by Darren Hart, talking about ACPI, kind of managed to convince me that that may be a viable alternative to device tree. For the most part, we just want things that work and are maintainable and stable. So yeah, I'm not, I confess, I don't really know how the power architecture and the spark have generated. They're using open firmware of their device trees from the actual hardware. I'm aware that they do that. But sure, yeah, I mean, if there's a way of easily extending that to other architectures, maybe that's the way to go. I do think it's pretty clear from the state of these things that we don't have it all figured out yet and that people have to have open minds going forward. So does anyone else have any questions or any violent objections? Or yes? So the remark was that there are no comments in JSON. And so we'd have to deviate from the JSON standard if we wanted to have comments in the device tree, which I think everyone agree is probably a good idea. Yeah, I mean, I suppose the device tree might want to modify JSON if anybody really took a hard look at using JSON as opposed to creating a new domain-specific tree description language. But JSON has a plug-in model and things like that. JSON is already diverse and has all kinds of files on it. So I don't know. The question is, would that be easier and better in the long term? I mean, the great thing about JSON is we'd get all these visualization and validation tools for free if we used it. So that would be nice. Fair enough. I mean, the comment was that we're not getting it for free if we have to change anything. But we're at a moment when we're sort of contemplating changing things anyhow. So that's why it's worth approaching the possibility just because the changes wouldn't be very large. I mean, it really is extremely similar. But yeah, maybe it wouldn't be worth it. Or maybe we want to go. Seems like the idea of using XML hasn't exactly caught fire. But I think there's general agreement that adopting one of these other choices wouldn't be ridiculous. Anybody else have any? Yes. Device wit, I'm sorry? Configuration. Oh, yes. So the question is if device configuration shouldn't go in the device tree, where should it go? And that's something that was discussed a little bit at Mark's talk yesterday. The way I look at this from an embedded perspective is, well, we don't want device configuration in board files. And we don't want it in device tree. It's just clear they don't want it. The people who are real software purists are like, we hate this hardware. It's so messy. Just get rid of it. We'll just write everything on some glorious virtual machine without hardware. What was this thing? Thank you. That Mark showed yesterday, Linux dummy vert. Let's just run everything in Linux dummy vert because we hate all this stupid hardware configuration. Just band it all together, right? So at one point, I think there was a proposal on April Fool's Day to get rid of slash art, right? That's essentially what we're talking about. So yeah, I mean, wherever we put hardware configuration, the grown-ups who are in charge of this whole system get annoyed by it. And I mean, I guess we end up having some header files, which of course are not board files. They're not board files. No, they're not. And they won't be in the directory where the board files are. But yeah, I think everybody basically agrees that somewhere this information, there has to be hardware configuration somewhere. But if the device trees are an EBI, then I guess they can't be in the device trees. So the message we're getting in all seriousness is that the configuration cannot be in a place where user space or other binary interfaces expect it to be stable. It has to be kept internally to the kernel and not exposed in any binary format because otherwise people will depend on it and they will be angry when their files stop working. I guess that's kind of where we are. So that seems sensible enough. But yeah, there is a question now, where does the configuration go? So does anybody have a? Yes. You just have just a separate piece to clear that unlike the device tree, this is configuration. And this is not to be touched by the Google and thus it can be done in the kernel. So the comment is that some of the virtualization schemes have an explicit configuration file which no one is supposed to read unless they're the owner of that file and that people are sternly warned is not being exported to user space and is just the private file of the hypervisor, I guess. So the comment is just that you have to be very careful about interdependencies that you're kind of coding in between the bootloader and the firmware and the kernel because then there's really no update scheme that really is going to be robust at all. So I think I'm basically out of time unless someone has one final quick question. So thanks very much for coming.