 I'm Frank Rowand. I work for Sony now, and I'm one of the device tree maintainers. And as a result of being a device tree maintainer, I feel the pain probably even more than most of you. And so I've been incented to create some tools to help make debugging easier. And today is another talk on that subject. This is the third in a series of talks. I was here last year, and I gave a two hour version of the talk, completely different material than today. Today is just an hour fresh material with a little bit of an update of last year. And I'll provide my my nice new tool for this year. Kind of a few housekeeping things. I always have way too many slides. And some of them are for context. The slides make sense when you go home. Some are just nice things I wish I could include in the talk, but there's just not enough time. So I just marked those with a skip in the corner. And when you read the slides at home, you see that that little skip you'll know that I didn't talk about it. And you might want to spend some more time reading those slides. Also, any questions, feel free to ask me during the talk. I'll be here after the talk. You can catch me anytime in the hallway. And I'll be here through tomorrow morning. The slides are already up on the conference website. They're also up on the Elinix wiki. And there'll be a slide at the very end pointing to those. So I give this talk, I used to have a very long explanation for this, but I think everyone really knows why. It's hard to debug device tree. And hopefully at the end of this talk, my goal is that you will have another tool in your tool belt to help solve device tree problems. Here's my outline, what we're going to talk about today. First, I'm going to update on the things I talked about last year, what's changed, what's better and updated. I'm going to talk about some concepts that you need to understand the last part of the talk. It's a little bit out of order. In theory, I should give those concepts first so you can understand the updates. But I think that's going to be explanatory enough that you'll understand it. And finally, I'm going to talk about my new feature and a tool to use that feature. So talking about last year, just a quick reference, you can look these up at home. These are the slides from last year. There were two major sections, the first hour and the second hour. The first hour, I talked about a tool called DT diff. And this tool now has the name of DTX underscore diff. And it's in the mainline kernel. It arrived in 4.6 RC1. And here's a real quick explanation instead of the hour version of what this tool is useful for. If you know device tree, you know it starts with some source files. They go through a blender. And out the other end, once you have a booted system, you have a device tree on your target system. And here's the more technical slide of what the blender looks like. And this is in last year's presentation. I'm not going to talk about this in detail. Don't try to understand it right now. The basic concepts are you start with source. It gets compotted into a binary. It flows through a boot loader. The boot loader may munch it a little bit. It gets put into the kernel's address space. The kernel then copies that into a data structure. The kernel may munch it some more. And overlays may come along and modify it further. So there are a lot of different stages in the lifecycle of the device tree where you might want to look at what does the device tree look like at this point in time? How does it change from a previous point or a later point? And that's what the DTXDiff tool is all about, is giving you the ability to look at device trees in the various formats that they can be in. It can be, like I said, the source, the compiled binary, or on the target, it lives in the sys file system. There's a nice side effect of DTXDiff that just slowed out of it for almost no work. If you've ever tried to look at a device tree source in the Linux kernel tree, you look in and you have the beginning source file, and then it includes lots of things, and those may include other things. And a definition may not stick. You may describe what a node looks like or a property, and then it includes 20 lines later, it may change that value. And so it's very tricky trying to understand what the end result will look like just by looking at the source. So if you just run one device tree through DTXDiff, it'll do all that preprocessing and out pops the final result, what it's really going to look like on your target system, assuming nobody else modifies it later. The other half of my talk last year, the second hour, included about four slides on one tool called DTXDiff to config, so a very minor part of that second hour, and I worked on that a lot more since last year. Last year was kind of a proof of concept, and now it's actually a usable tool, and I described it in an hour-long talk at LinuxCon Japan this year. So if you want to understand that tool, I'd highly recommend you look at those slides, and you'll get a much better understanding than my real quick explanation I'm going to give in the next slides. It is in mainline now, it just got into 48RC1, so it's out in the 48 kernel, and here's what it's trying to deal with. There is a manual process in the old days where you have a device tree, and it's describing devices, and then you want to generate a kernel that can actually handle those devices, you need the right driver in the kernel, and the driver needs to be able to access that device. So the manual process you went through is you looked in the device tree, and for each device there's this magic string called a compatible, and that magic string, the compatible, somehow matched up with the driver file, so you had to search for a driver file somewhere that had this magical compatible string, and then you had, once you found your driver file, you then have to look in the make file, and in the make file you'll say what is the kernel config option needed to enable this driver, and then finally you look in your kernel configuration, is my configuration option enabled or not. So it's not complicated, but it's a long process, especially if you have several hundred nodes that you need to do this for. So the solution is the new tool, dt2config, and you just pass it the device tree that you want, the kernel config file optionally, and out pops a line for each device, or multiple lines, and this is an example of just one line. Can you guys in the back actually read that? This is really cool, I've never had a screen this big to be able to have small font, so I always blow things up, and you'll have to see my LinuxCon Japan talk to understand this. I'm not going to talk about flags, but you'll see all the things I talked about in the manual process. There's the device node, in this case the coin cell, there's the compatible string which has got to magically match with the driver. This tool found which driver matches it, in this case the qcommcoincell.c, and it looked and found out what config option needed to be enabled, and it checked the kernel config and said, no it's not actually enabled, and people saw this tool and they said, but why doesn't it just fix the kernel config file? So I gave in, I didn't want to do this because it's just too easy to blindly accept the result, so I compromised, and you'll see that same line that was there before, that report line, but it's nicely commented, and there's a comment line saying, what's the current value in the kernel config, and if you want to enable this driver, what do you have to put in the config file? So in theory you could just take that blob attach it to the end of your .config, uncomment this one little configuration option, do a make old config and you're all set. The complexity is for a given node, given device, there might be multiple compatible strings, and for each of those compatible strings there might be multiple drivers that can match that, so you as a human being actually have to pick and choose, this tool can't do everything for you, so it's not just an automated configuration, it gets you most the way there and it tries to make life easy for you, but you really really do have to pay attention and think about what you're doing still, you still need to be involved. There's some other things I talked about in last year's talk, there's still proof of concept, nothing new to talk about on those, so here are the concepts we need to understand today's new tool. Is anybody here not familiar with what device resource looks like? Okay, so if you do understand what device resource looks like, raise your hand. Okay, so it's a mix. I'll go really fast through this. First of all, why have a device tree? The point of a device tree is to describe hardware, you cannot auto probe, and by its name you can guess it's going to be a tree, and the nodes of the tree are the actual devices, and a node can contain other devices, a node can also contain properties, and you can think of a property just conceptually as being a static variable with some value, and that then gets fed into the system. Here's a very simple example of what a device tree looks like, and I'm going to use this just to point out some of the different parts in some vocabulary. So first of all, a node, I've been talking about nodes. We see our tree is rooted at slash, just like a Unix file system, and our node slash actually contains one node called SOC. Our SOC node contains two more nodes, our interrupt controller at some address, and a serial device at some address. It's a very straightforward. You can also notice that there's a path here since it's a tree, so our serial device is actually full path is slash SOC slash serial at this address. The other thing we care about is properties, and what values those properties have. Again, each node can contain properties. These are just examples of an ASCII string. Down here there's another property with two ASCII strings. There's some properties with integer values. There's a Boolean property. This existence simply is saying true. So fairly straightforward. So today, a new feature, a new tool, and hopefully life will be better after this. It's not a mainline yet. I've not even sent a patch to the mailing lists. Shame on me, I guess, but I've been actively working on this, trying to make it clean and functional and working out the warts, and I wasn't quite ready to submit it yet. All the patches for this are on the elinux.org wiki, so they are available, and again the slides at the end of the slides that give the pointers to where those are at. So what was I trying to solve with this new tool? Basically, I wanted to be able to look at how properties were used in the kernel versus how they existed in a device tree in the source. And to be a little bit more specific in some examples of that, I wanted to see when is a property being accessed, for example. Which properties were being accessed? Which properties were not being accessed? Is there some difference between what properties are in the device tree and what properties are being accessed by the kernel code? And does the device tree source actually contain the proper properties? So you can look at this from two different directions. You can look at this from the kernel code side, and you can say, did my code in the kernel actually read a property value that's in the device tree? Did it try to read a value that should have been in the device tree, and it failed to read it, got some sort of error back? Or did it not attempt to read a variable that does exist in the device tree? And if we turn it on, let's have a look at it from the other side, look from the device tree source side, we ask essentially the same questions just looking in the opposite direction. Does the device tree have the necessary properties? Does it have the properties that are needed by the driver? Are there properties that should not be there that are there? Just because a property is legal and optional doesn't necessarily mean you want it in your specific device tree description. So that could be an error if you actually included it by mistake. And finally, do you have properties in your device tree that the kernel did not even attempt to access? That might give you an idea that you have a stale property, or maybe someone forgot to implement that in the kernel code, or in the driver, even though it's defined in the binding. So it's just another clue in insight. So being a typical kernel developer, my very first tool has got to be printk, right? So a very simple tool, incredibly simple. And I put those printk's in the APIs for accessing properties, and that leads to a slight side effect in that some code accesses the device tree structures directly instead of through the APIs. So you have to be a little bit cautious about these results. You can't just look at the result of the report and say, oh, that's broken over there. You have to say, oh, that might be broken over there. Let me go investigate a little bit more and see if my suspicion is correct. So the same issue applies both to drivers and to the general frameworks that exist within device tree. So you can use it as a black box, but just be aware that it may bite you occasionally if you do that. And there are times you really, really do want to be looking at the kernel code in association with this. And there will be some slides later in the talk which I think I skip over, which are, again, cautions and warnings. So it's really easy to turn on. Just turn on a config option. And of course being in the kernel, it's never quite that simple. We have a few more steps. Your printK log buffer is probably too small. So you probably have to increase the size of that. You probably have to put debug on your kernel command line to get the messages out. You're obviously going to have to rebuild your kernel and reboot your kernel. All of this stuff, for the most part, is really happening at boot time. That's when the interesting messages are coming out, except for overlays, which are another whole other interesting area. So you boot your system and then you can collect the data in one of two ways. You can either capture your boot console, for instance, if you have a serial console, or you can just run the de-message program to print it out later. And here's what it looks like just to scare you. It's kind of cryptic. And my first, there's always something, some gotcha in it, all these things they do. Here's the first gotcha. The kernel takes that device tree blob, which is a binary object, and it expands it out into some big data structure. And while it's doing that, it's accessing properties. So if we're reporting all these properties that are just part of this whole expansion process, that's not useful information. So I had to put in a marker to say, here is where the end of that expansion process is. Ignore all the debug messages before that point if you're looking at the console output. So that's just a nice breakpoint that you have to be aware of. And so once you've gotten past that message, and that's just a magic constant string, anything after that is going to be a useful message. And I'll talk about this format, but basically these are OF underscore find messages. And the format is that first there's a PR FMT, and that's part of the whole debug infrastructure, which puts a prefix on your messages. So in this case, it's OF colon. Each file optionally has a PR format which may differ. And you'll find that in the different files I use that prefix actually differs. So it's not always OF colon, it's sometimes something else. But then the next thing is the OF FND. So that's always going to be there, something to search for. Then there's going to be a status, which I'll talk about on the next slide. Then the full node name, so I mentioned how each node has that path essentially from the root all the way down to the node. So that full path will show up in these messages. And then the name of the property being accessed and how large the property is. So here's, well, I'll do this here. Here are some examples of that for nodes slash CPU at zero. First the register properties being accessed, then the same node, the enable method properties being accessed. Then a different node, CPU at one, instead of zero, again accessing its register property. So it's pretty straightforward looking at the messages. The statuses, these are the same statuses that come back from the internal APIs. So if you know the APIs, these totally make sense. The most important one is success. It was able to access the property. You're most likely to see minus 22 E and valve if it doesn't exist. There's some other errors that come back if the data is malformed. So that just gives you some insight if you're really deep into the core of how the APIs are working and what your driver is doing and why it's getting back a weird result. There are two values that I added which are positive numbers. And those refer to the internal use. Marking when are these messages usable or not? So when we're in that unflatten phase, we don't care about those messages. So I bracket with a begin and an end. Same with the overlays. When we have overlays and applying them, there's going to be a range where those messages are going to be garbage. They're not going to be useful information to us. I actually have not added those two messages in for overlay. So I have to add that in to the overlay code. Pretty straightforward. So how do you actually use these messages? Nice ASCII messages coming out on your console. You should be able to just read them like any old print K, right? No big deal. My first system, really tiny system, 18,000 messages. I'm not going to read 18,000 messages. I'm sorry. I don't know about you guys. So that's easy to fix. I'll just use one node and look at one node for our talk to start with. And it's a really simple node. And I chose a node to be deliberately simple. And you'll find that it still provides a good flavor of what's going on. And the way I find that is I just grep. Yay, Unix tools. And you'll notice the sort on the end. So each of these messages may occur multiple times. I wanted to get rid of all the duplication and just simplify this as much as possible. So the sort dash U says give me all the unique instances of these messages. So I'm down to just four messages for this one device, this coin cell driver. So that's pretty cool. I'm used to having a really small screen. So I didn't want to give you all this small font. So as usually, I tried to do things to blow up my fonts. So I'm just going to strip out that whole path here, just adding a said. So now we have our four debug messages in nice big font that I can show on subsequent examples. The thing I just skipped over was there's something that I left out in this very simple grep. Grep is wonderful, but that whole unflattened device tree phase got captured by this grep. I didn't go to my console file and find that range and delete all those lines, which I could have done, but it's just one more manual step that you'd have to do. So here's those four messages, and I put them on the same slide as the actual note in the device tree. So it looks pretty straightforward. Like I said, a rather simple simple node. And we can look at how those messages compared to the node. So we start with the first access of a of a property, the compatible property status was zero. We were able to access that that property. And we look in our device tree node. And yes, we have a property called compatible. So far so good. We have our property size 21, which is our string size, including our terminator. So far so good. Pretty straightforward. Easy tool to use, right? Another catch. Our next property is name. We successfully accessed it. It has a size that's nine bytes long. But do you see a name property down in the node? It doesn't exist. But you'll notice this nice coincidence here. I have something nine characters long with the terminating byte. Matching the size of the property. That might make you think that's what's going on. And indeed it is. Name is a special property. The device tree compiler knows about name. And if you put name into your device resource, the compiler will just strip it out when it compiles. It'll be gone. It won't be there in your binary blob. On the other hand, when the Linux kernel takes your binary blob and expands it, for every node, it's going to create a property called name. And it's going to take the node name and put that in as the value of name. So it magically gets populated for you. Next property status. Again, we found it. And here we have our status in the device resource. And it has the very important value of disable. Which means this node is not enabled. And that's going to be very important in a second here. And then we have, once again, our device type property, which doesn't appear in our node. And that's an artifact of that unpacking since I didn't filter that out. So just to summarize how to approach all this and what I just learned for this node, there are several properties that are in the DTS and the kernel code that access them. There are several properties in the node that the kernel did not attempt to access. These four properties. And that's where this status is going to come into play. This is going to make sense that these properties were not accessed. If the node was not, not enabled, then the driver probe function didn't get called. So that's why the driver code didn't even try and access these properties. So when you put all that together, those pieces, then the picture should make sense. Then finally, what wasn't in the DTS? And I'll be going through those three categories multiple times in different examples. So it's what's in the DTS and actually got accessed by the kernel. What's in the DTS, the kernel failed to access it. And what did the kernel try and access that's actually not in the DTS? So that wasn't too hard. It all fit on one slide, right? No big deal comparing back visually. But imagine if you had an entire device tree. And my example device tree is over 1,000 lines in this case. So I decided it's time to automate this part. So a new tool, DT underscore prop, and that's going to compare that console log file to the device tree. So very easy. You just give it the target log file and a device tree in one of the many possible formats whether it's source or a binary blob or a file system tree from the target. And here's what the output's going to look like. It looks a lot like a diff. We have the console on one side of the diff. We have our device tree on the other side of the diff. Here we have our node and our properties. And we can see our typical standard minus plus or space for our diff. So it looks a lot like DTX diff. So just to lay that framework a little bit stronger and cement that in your minds, I'm going to just give you the algorithm of how DT prop works. And hopefully that will make some of this stuff later make more sense. We start creating a device tree source file called DTS target one. And the way we create that is we scan through that console log, look for all those OF find OF FND debug messages and for each of those messages we add a line to our device tree source which provides that source path to the node and the property that we're accessing at that point. And this is what that looks like. It doesn't look like any device tree source you've ever seen I'm sure. And it's really ugly and that's not going to do you a whole lot of good to examine manually. Our next step is to take that and we run it through DTX diff. And I mentioned earlier that nice side effect of DTX diff you give it one file and it's just going to preprocess it and out pops it a standard normal DTS file. In nice canonical format the nodes are nicely populated with all the properties. The nodes are in alphabetic order. The properties are in alphabetic order. So it's going to look like a normal device tree source file something you're used to looking at. Run that same process through the other the device tree source that you're comparing. In my case I was using a real device resource like I said it could be a binary blob. And again we're getting into that canonical format. And then the final step is comparing those two files that are in canonical format. And now I can do an intelligent diff. I'm sure that if you've dealt with with diff you've all dealt with diff. If you have a block of of tax and you change two lines in the middle of it sometimes you end up with really weird results where you might get a plus in one place and a minus in another place for the same exact line that didn't change. And I didn't want that to occur. I wanted those pluses and minuses to really reflect that the property was either accessed on the target or not. It was in the device tree source or not. So I didn't want any of these silly artifacts coming out of diff. So I had to create an intelligent diff that knew how to deal with this expected format of sorted device tree. So you can start with just the differences. And I'm so undecided about this whether the default should be just show me the differences only lines that have that plus or minus in the first line or should I show all the properties is the default. Even the ones that are in both places both the console and in the device tree. So if anybody has suggestions on that feel free to pipe up. That's what makes more sense. So we're going to go back to I did this before for the the grep example. And this is much easier. I can say what's in my device tree and what was accessed by the kernel. And of course standard diff my space means it's in both places. So it just jumps out. I just run the tool look for the lines that are in both places. Easy. No thought. No fuss. It's right there. What's in my device tree but it wasn't assets by the kernel. Again I just look for my plus sign. What was in my device tree. My plus line. But not actually on the target system. Just jumped out of it. I need to think about it. And finally what wasn't in the device tree that was accessed by the kernel. And in this example there aren't any of those lines. So crummy example sorry. I pointed out before the importance of that status property. And we saw how the status being disabled told us something really important when we're actually looking at the device resource. One side effect of this whole processing is the values of the properties are expunged. So you don't see any value there for status. You don't know whether it's enabled or disabled. And I thought that was such an important thing that comes up so often in debugging is why doesn't my device exist. It's because it's disabled. So I added a feature to specify I want my disabled nodes tagged. And it will actually check and it'll put a comment up warning you that your node is disabled. So in this case it'll make sense. These four properties aren't being accessed because my driver didn't even probe. And it just jumps right out at you. So that was a really simple example. Let's make it a little bit more complicated. We'll change that status from disabled to okay. So now we're going to actually probe the device. And we should see more properties in our list. And here we go back to our graph example. And every line that's read is now a new message a new property that wasn't there before. So it's bigger. And again I'm going to put that on the same slide as the actual device tree node. And I was getting a little bit harder to compare back and forth visually. But it's still humanly possible. So I can look and see what's in my device tree. And also was accessed in the kernel. So I go through my messages. I find my compatible. I look in my node. Yeah, there's compatible. Name. Oops, that's the special thing. The node name. Q com R set OMS. I look in my device tree. Lo and behold, there it is. So I can manually look at that. It's still not too difficult. It's a little bit painful. But it's humanly possible. And conversely, I can say what's in the device tree. But never was accessed on. On the target by the kernel. So Q com charge enables the only property looking through this long list. That the kernel never tried to access. Again, still humanly possible error prone. It's really easy to probably overlook things. And then that third category, what was what did the kernel try to access that turned out was not actually in the device tree. And again, you have to go through the list and then compare it to the node. And hopefully I got all these right. Again, it's humanly possible. So DT prop is a whole lot easier. If I just run that. It just jumps right out all the minuses, the pluses, the spaces. So again, those three categories, what was accessed, all I have to do is look for those spaces in the first column. So those five properties jump right out. What is there in the device tree that didn't get accessed by the kernel? Just look for that plus sign. Jumps right out. Easy peasy. And finally, what did the kernel try to access that was not in the device tree? Just look for that minus sign. Again, they just jump right out. Really, really easy to scan for that. So what should we expect? We saw a list of what the driver or the kernel was accessing. And if I go and look at the driver, these are the APIs. Again, a really nice, simple driver. This is a great example because of this. It's only trying to access four different properties. And that matches what we've been seeing in these slides. And here are the four properties. Trust me, they actually match. Then there's that one property that's in the device tree and the kernel didn't even try and access it. What's up with that? This was a really cool mistake. I didn't do this on purpose. So this is not a set up example. This really is what popped out when I ran the program on my system. It's a real honest to good bug. I've been carrying a patch to patch my device tree to enable this device. And the device tree definition changed from several releases before to the current release. So it used to be the case where you'd say you have to enable the device. So you'd have a property Q-com charge enable. They decided to flip it on its head and say that if you don't want it enabled, then you have to provide a Q-com charger disable property. So this pointed that bug out really, really obviously. Okay, I talked before about how you shouldn't just use it as a black box. It really helps when you're working conjunction with the code. Now that you've seen some examples, you can see that just looking at those diffs, some things do jump out and you probably can use this kind of as a black box. And that last bug was a real good example. And I ignored a lot of the other properties. So here's another example from an entirely different node where there's something that looks suspicious. We have an interrupt property that was accessed by the kernel, our space up front, and it exists in the device tree. We have another property, our interrupt parent. Our interrupt parent is the interrupt controller that's going to handle that interrupt. And the kernel tried to find an interrupt parent property in this node and it couldn't find it. It doesn't exist in the device tree source. So is this a problem? Is this a bug? Someone in the back says no. If I had prizes, I should have brought prizes. You would win this prize. It turns out that if I look in the device tree source and I just look at a small part of it, we see what matches that DT proper report. It makes a lot of sense. We have our slash SMD slash modem interrupts property and it does exist. So, of course, it got accessed. We don't see any interrupt parent property. So everything's still holding together. It makes sense. And if we go and actually go back to the print K messages in the boot log, actually print K does have its place in time. This is a place where you have an exact issue you're looking at. You can go find that spot in the console boot log. And this is going to actually make a lot of sense. And you can see it tried to access SMD modem interrupts property and it got that, status zero, tried to interrupt the interrupt, tried to access the interrupt parent property, couldn't find it. So the next lines in the log, we see it move one level up the tree. It goes to the parent of SMD modem. It looks in slash SMD, looks for interrupt parent. Again, it doesn't find it. Goes one level up, goes up to the root, looks for interrupt parent again and lo and behold, it finds it. So this is an interesting side effect. If you're trying to understand how the device tree code works, this is a good tool to help you explore just by looking at these messages. You can see what's going on pretty quickly. If you try and read the code, you'll find that it's a bit of spaghetti by the time you nest into it. It can be very hard to follow through. And so of course, the device tree now that we know where our interrupt parent is, we can look earlier in our thousand line device tree file and lo and behold, yet it's there as expected. So everything's good as our prize winner pointed out. That was not a problem. I told you I had too many slides. Okay. There are a lot of different caveats. I've just covered a few of them. The usage message from the script gives some more. So I highly recommend not ignoring that help message. So what did I talk about today? First of all, I talked about updates to last year's talk. First of all, the DTX diff tool, it's actually incredibly cool, despite the fact it sounds really simple and stupid. So I recommend looking at last year's talk if you are not familiar with it. Then there's the new tool, which was barely mentioned last year, the DT to config. And then that got fleshed out in my LinuxCon Japan talk. Now a useful tool in both of those in the kernel tree now. And today I talked about my new tool, that wonderful print K, which is incredibly simple, but that simplicity is very powerful in the end. And then DT prop to help analyze the results of that. And you may notice that these tools have been piling on top of each other. So that very first tool, DTX diff, has been a stepping stone, which gets incorporated into DT to config, also gets incorporated into DT prop. And so in the standard Unix tradition, I'm finding that a lot of these tools are working very well together. And hopefully there'll be more coming in the next year. These are just pointers to where to get all those articles. Again, they're in the slides available at the conference website and elinux.org. So thank you. Thank you for coming. Thank you for staying and not going to lunch. And they'll take questions. Question back there. I'm sorry, can you repeat that? Oh, okay. Yeah. Um, I think my DT DT prop help message actually gives some examples of that. There especially a few frameworks that scan through this, this, the structure of the device tree because it's just more efficient and easier for them to process that way. So you won't get any of these OF fine messages when they're accessing the structures directly. Does that make sense? I'm not sure if it will make sense to modify those that code to actually use the APIs, whether it's worth it. If this tool becomes really popular and useful, then maybe it would be worth doing that. Yes. Yes. Why do I strip out all the values? They might be interesting. I was really tempted to put them in. So you will have noticed on those console messages, there were no values. There was the node name, the property name, the size of the property, but not the actual values. And I was just concerned that the volume of the size of those messages was just going to become very, very large if I included those in the debug messages. It's yeah, in theory, you should be able to look in the device tree and see what value should be there, but maybe they're wrong. So yeah, it could be useful. Yeah, maybe a switch. Yeah. Yes. To adopt the which part. So the same sort of messages. Pardon the same sort of not that I know of. I've haven't talked to the bootloader people at all. And this is actually the first that I've really talked to any extent about about the tool. So I haven't even planted the seed in anybody's mind. Yeah, yeah, it would be a great thing to add into you boot also or the other bootloaders. Are there any bootloader people in the room? Actually would like to talk to bootloader people about other stuff. There's question over here. Yeah. No, I didn't consider using trace points. Do you know how early they're available in the boot process? Yeah, that could be a good, good alternative. Yeah. Yeah, one thing that you may have noticed was I used a config option. And I use that config option to enable debug statically for all the files that have these messages. And that was because doing dynamic debug kicked in too late in the boot process. So it's a similar type issue. Yeah, so that that'd be one alternative I could look at. Well, okay. Yeah, trace print K may be a better alternative altogether. More questions. Thanks for coming. Have a great lunch.