 Awesome, all right, welcome to the booth about challenges for low-spec embedded Linux. This is kind of an experiment. I have the feeling that, or we have the feeling that, in general, if you look at ELC in recent years, that the topics are not so much more about the topics that you need for really burning really low-spec devices. And so I was trying to see, maybe it makes sense to set up a forum, collect like topics, what are actually real challenges these days on the low-spec side together. And then we can see if we can, in the next couple of months before we hit the next ELC, maybe make some progress on it or communicate these things that we can find here to the appropriate communities. Or maybe go out of his room and say, no challenges, everything perfect. Let's go for bloat. And the world is fine. So I don't know exactly what to run here and how to run it. So your contributions and your ideas are very much welcome. So I mean, but I thought like for the agenda for today and not try to aim too high is first to look and discuss what is actually these days a low-spec Linux device. It certainly has changed in the last couple of years, right? And let me have a little bit of a discussion about what we would say is a low-spec device for Linux these days. But what I hear is that you kind of agree that it's good to have a discussion about what does low-spec mean here in this both initially and so we can settle on something and say, okay, I mean, whatever, 200 megabytes or one megabyte or. I agree. That's the first thing. Let's have a bit of a discussion. What does low-spec mean or what already contributed to something, but maybe others have opinions as well. The second point I think would be good then once we have understood what it means to be low-spec to look at the potential dimensions where there could be challenges, right? I mean, it could be like the Linux kernel has challenges or the boot loader has challenges or we have challenges on the lip-sees or we have challenges in the distributions that are just getting bloated. Or we have challenges on the tooling that make it hard to rebuild products in the low-spec side where we think bigger, if you have more space, it's a moment far easier, but it doesn't need to be that case. Could be. Let's talk about that in the first point. Then I wanted to basically look at the various levels of the stack and the tools and dimensions. What is there? Where are there, especially maybe we can identify some challenges that we can then take out of this session and carry forward into the individual communities. And then maybe if we are good, discuss what could be done, what are the topics, what are the actions that we could try to communicate out and bring into those various communities. I mean, does anybody want to talk about something different here in this session? I mean, or does everybody think that's a good approach to tackle this? Yeah? All right. So let's talk about what is low-spec. I mean, can you see something that is pretty low-spec? Not sure if it's a low-spec thing, but I don't know. Does anybody know what that is? It's networking. Yes? Does anybody know what that is? It's a router, yes. What do you think is the spec of that roughly? I think that's close. And of what? It makes a flash and memory. I think it's really close. I think at the moment, this guy has 16 megabytes of storage. Does anybody know? Yeah, I think N64 of RAM. Is it low? Probably. Or, I mean, like 20 years ago, it's very much a lot, right? You could? You could not. Yeah? Indeed. I mean, I don't know. And then we have this guy, I mean, is this low-spec? No. Or is it? This is zero, yeah? By zero. It has no EMC in built-in, so you can plug in whatever SD card you want. Storage gig, I think. Is it a memory? But it's cheap. It has 9 euro. I think you can run a desktop. I don't know if I would use it for a desktop. But I think you can certainly use it for some graphical appliances. And so, I mean, it's something on screen display, and you do funky stuff on it. Certainly. Right? Yeah. Exactly. There's scales down that's running main internally, running off of an ACO battery. Do those things run on it? I don't know if they run internally. I know they run main, and I don't know if they run off. Right. I mean, so this is, I think, questionable. Probably not. I guess the other one was pretty more. This is certainly not Low-Spec, right? It's a nuke. But in the end, people are using it for stuff that previously people used on really Low-Spec devices, right? People want... It's a big gap for an animal when it's used. And the problem is the limit kernel is deep, you know. The current limit kernel, the binary, won't fit into the next brain. I think you might not be able to understand that. That's not what I said. I said, if your platform is such that you can run a traditional stack that does top stuff, maybe that doesn't even normal on a regular stack. But you can do that. You can make it run. You can distribute your application as a substitute of some form, in the way that does top the system. You can distribute applications. We don't even think about that as Low-Spec, there's nothing that you can specially have to do about it. It's un-traditional. Since the initial community, I'm talking about... I'm talking about traditional. I'm talking about mainstream, the kind of thing that said, the kind of thing that said, let's put a limit on my desktop, and I'll find it, I'll take a distribution on the system, and I will pick up the... XST... That's my favorite. The point is that you could think that's been anything that you had to have there. You could pretty much run any desktop you wanted. Even smaller ones than that, those high-zero would have enough support power to run XSTD. So we've got stuff that if you wanted to do, you could run XSTD next to it. You could, and then that's about a week you could throw it out. You could do that if you had to. But if you were just... If you were just going to do the D-out, then you would be developing some kind of... That's the purpose of using built-to-box, right? So you wouldn't put a desktop on it. You would put only what you need on there. But if you were building... I mean, that's just exactly the idea. Okay. That's the issue called high-box. Figure out how all these pieces work together. And as I'm doing this, I'm realizing that I can build anything I want. I don't necessarily want to build anything. I just want to build what I need to do the thing I want to do. All these small devices in order to squeeze the performance I'm going to try to get. And then it came very easy. Obviously, I'm going to build a record high by putting it down to a record high-doer. So I'm trying to build a little phone system. Okay. I have to... You cannot responsibly think about the title music. You can't support... Well, you can't support it because the title is a bit more complicated. So the... Look. If I'm going to run packages that come from the open source world and say, oh, I like that. I'll take that. There are so many newisms that are embedded in there, but I don't want to be working with them. I have to run it. GCC are playing. I know. I think the product is running 19k bytes and you know, things like that. But that's not what I'm going to do. How many GCC do you actually build? I made 20 GCC for over three years because I was trying to get it to build everything. And I just... I had to test it out because I didn't know what that was. The reason I've heard it stop through the 90's and the 30's and the C-front was 20 years fail. What? LLVM recently grew a C back end. It produces C code. So there's a new C-front you can compile LLVM with the LLVM to C converter. And then if I fix up the GCC enough, I can redirect LLVM with that. I think... I personally, I like the idea and the approach to say we couldn't define what is a low spec device based on what kind of stuff can you run on it. I mean, surely you cannot run a desktop on a low spec device. I mean, not even XFCE. So I mean, that shouldn't qualify as a low spec device if you can run XFCE on. Right. I think the one that I showed first that has 16 megabit of storage you will not be able to easily put an XFCE on. And now I'm thinking, it doesn't matter. Maybe it's about... Exactly. But I think the graphics is not the real factor here. So I have a feeling when I'm listening to that I was trying to think, I mean, is it about traditional distributions? What does it mean? Is it like the standard distributions that are binary, but then Alpine can also be pretty slow. So I was maybe thinking maybe it's low spec device is something where you wouldn't want to use G-Lib-C on. I'm not saying, I'm saying the opposite. I'm saying display and so on is red herring. So it's a low spec device is something where you wouldn't want to put G-Lib-C on but you rather want to go for something like Musa or U-Lib-C where you can really go small and bedded. And that means, implicitly, it is usually built-root or you need to use Yocto or something, but even Alpine can produce very small stuff. Exactly. And it's small. Exactly. But we also use it on small things like that. Because it's nice, it's binary, it's easy to install stuff, but it's still reasonably small. Very easily there and otherwise you would have to spin up a whole Yocto engine and build everything to do these things. So I mean, that's why I think it's not binary versus source distribution. It's rather you wouldn't want to use G-Lib-C on those low spec devices. Yes. I mean, if I'm saying on low spec devices you wouldn't want to use G-Lib-C, I'm not saying you want to use G-Lib-C on high spec devices. I'm just saying you couldn't fit your G-Lib-C in it. Right. There's a lot of other tips that I can give about how to fit your G-Lib-C before it's on my screen. You don't have to go to the store. I mean, if you have hair, if your device is supposed to be talking to the person without it, then you just need to have a hair out of it. But then if you can fit your hair out because I'm not being talk to you, I just need to do a specific thing. Right. I don't want to have to do it. So that's why I think it's only the most effective device is to have a specific purpose. And as a resource worker, that's the only way you don't have maybe a poor methodology on how to use G-Lib-C. You have very little resources to be a resource worker on the store. That works when you're, when you have a person through the body that's not going to get up to anything that's too much external. Right. Right. Right. Right. But I think the definition of saying devices really designed for a certain purpose and then the resources available on that device tuned to match exactly what you need for that purpose is actually a pretty good definition of potentially, maybe it's not low-spec but there are constraint devices. Purpose. I think about it This is the same thing in the computer state. Microcontrollers outweigh all the big things. And Linux. And Linux. And Linux. And Linux. And Linux. And Linux. And Linux. And Linux. And Linux. And Linux. Can play in that space. There was a. in�만bit.com 2015 that had executable flash with the kernel and the fibresimages flash. And it had to be the only way the But most of these systems aren't running Linux, they're running some kind of public art culture. And there's a sense that I think that the case is that, oh, I want to do school and respect that effort. But if you really look at what makes Linux Linux, it's the unit aspect of it. All right. And so... I think we're going a bit off talking here now. So I mean, I think we have talked a bit about, and I thought it was super valuable to talk a bit about, like, what is low spec? And I think we have some great ideas here. It's not about finding a solution, it's really about getting an idea. And do you want to say something to that, sir? Yeah, I think we do that with a lot of TV. A lot of the stuff that's written through every TV and we're having to do a lot of those things. So once you power off, that's what we want. But you don't have a lot of... We don't have a lot of... Yeah. All right. So I mean, let's maybe... I would like to talk for a few minutes about something else. I mean, or maybe I don't know. You can also talk about it. But I think we've talked a bit about, like, what is low spec? So the idea was now to take a look at the various dimensions of Linux and developing Linux and so on and see if you can maybe spot certain areas where we believe there is some risk of things going off bounds or making it very difficult for people that want to build stuff for Linux to do their job because those guys that are developing it are not behaving, they're adding boost everywhere and I mean, all those kind of things. And I wanted to figure out, I mean, maybe we can identify some hotspots in some areas of those things and I mean, I had basically tried to figure out, like, a segment of those dimensions and the way I pointed out here, which I think is one of the core Linux components or what is the footprint of things like bootloader of Linux and Lipsy and standard libraries and so on. Maybe there are some areas where we say that's actually pretty fine. So what I know is that Linux has been pretty good in maintaining the zero-defcon thing and it has been pretty stable in keeping pretty low footprint and so on. So that sounded like those guys are out of the waters. But maybe there are other things that you have experienced, have a bit of a problem or we could talk about the distributions in general. I mean, are they up to the task? It's like Yachto going through the roof and getting bigger and bigger all the time. Do you have challenges keeping that slow? Sorry? Exactly. So that's the point. I mean, we don't have much time so I wanted to see in which of those you would want to take a look at and we can maybe discuss on those things. Are you more interested in talking about the individual components or the distributions or the tooling, how you work with those things? Let's maybe take a look at the components and see where we feel things are going bad for the people that are really trying to do low-spec devices. So as I said, I think Linux is okay. It can always be better but I think you can compile it very small, I think a couple of hundred kilobytes. Kind of. There's tiny defconfig as well or something, right? I think this was replaced by a tiny defconfig. It's pretty small and constant. I know but I'm not sure about whether the networking is too big for what the size is adding there. Probably. I mean, that's why I think if you talk about it, I can also take it offline and we can measure it and bring it next time back. It might not be. That's why I wanted to talk about it. I feel Linux is pretty good. I think if you come up with a kernel one or almost two megs, two megs are very powerful. Lots of stuff you can stuff in there from the kernel side and you can still put that on a 16 megabit. Exactly. I think Linux, everybody is in vehement agreement that this is not really the big place where we could go and try to say that we can ensure that we get better about it. Where is he adding that to? Yachto? Is it a distribution challenge? Or is it not built into the kernel, I hope? On the runtime side? Okay. Let's look at the other components real quick. Is there anything that is bad where we should try to say that we have enough weapons and tools that are good to go? So is anybody agreeing that system D is not very good for embedded or distributed? Everybody agrees that it's not a choice. You cannot make a deterministic boot order very easily and these kind of things. It's very hard, right? You could tweak, I guess, all the dependencies in a way that you then know how it starts and when it starts, but it's like a big of a... What does it say? Okay. I don't know, Python. But Python is not suitable for embedded for low spec, right? Or we agree that we feel like it's... Okay, good. I agree. I mean, so we have tools there available, I think. This VInit is also available. It's a pretty small thing. Procty and OpenWDT works pretty well. It's kind of like this similar like this LXD thing where you can have the... That's basically what you said, right? Running LXD containers as an in-it process, that's what it's doing. That's basically exactly that. I'm just very small LXD containers to manage your system and do that in a reasonable way. Why not? Yes, we keep all the containers small, right? In the end, I mean, the containers give you abilities to constrain them, how they are running, and you can be managing better from top. Containers are not necessarily... In my opinion, not necessarily conflicting with the low spec paradigm. They have low overhead. What is in the container needs to be again small, and if that is tiny and so on, then there's no problem. Exactly. Exactly. You can run the containerized system and then get the abilities to do some more, a bit more flexible about the software you run, how you control it, and so on, right? Right. Then I think middleware, they always have the feeling that there's always... Whenever you also system D, if you bring it in, you always get D-bars, right? I agree. Even though the Bluetooth stack, I must say, is not that big, so at some point we made a nice small microcontainer with just blue Z in it, and that was compressed with exactly one and a half max, and this could take all the readings of the Bluetooth dongles and send it to the cloud. So that was fine, but I think, I mean, graphics for sure, I don't know, I mean, Wayland, I think... The X is certainly big, right? That doesn't work if you want to just have something nice, a bit more comfortable than frame buffer on your device and those factors. There's no good choice available. Wayland, I think it's not... There's also not nothing there where there's really... But I mean, I think for Squash, it's not like it doesn't make a copy of everything in there just what it needs on demand, right? I mean, it's not... But OpenWT is using it quite a lot, right? They use Squash with us to keep this... I think... I don't know. When you... When you optimize your... True. When you optimize your... You're almost always juggling cases that are different from... That's the honest... ...reform... But that's when you're going to use the... ...unique stuff on your device, right? You're just... You have a lot of flash and you've got a whole bunch of stuff on there, not everything will be read at the same time, right? If you have, like, 3D runs on there, then we'll be good to be running. But it sounds like there we have enough choices available, right? I mean, we have Scottish affairs, we have, like, grammar affairs, and so on, you have what you need in order to do your job, so I think on the middleware, I don't think we have everything we need. To do something nice, I mean, I think no graphics, stack that is more comfortable than doing frame buffers is available. So maybe an action we could take out of it is to really look at the individual, I mean, like, Bluetooth and the D-buses and so on, and see what are the source distributions, how small can they actually be, and then if they figure out there's really issues in source distributions, then we know that something can be done. I mean, I know in binary distributions sometimes they have to make choices and make a generic, so they bring in dependencies that in low spec and embed it you might not need. But I think if those components at the source level, you can build them in a way that they're very constrained and they're not doing stuff with normal configure options without having to patch them and so on, then those should be pretty good, right? I mean, so we could take a look at those options and see how big they are and maybe summarize it afterwards. But for graphics, nobody knows anything, right? I mean, is there anything that is pretty good for doing low spec things on graphics? I don't know if you can act on it. We've seen a lot of moderators. They like to talk with friends, right? They're friends, why are you doing that? So there's no attention right now. Are you doing that? Open the time directly, because I don't worry about the amount of time. 20 years ago, there was a better room that's only like 10 years ago, right? I can't be able to call either. But there's 50 volumes on the opening. One thing that happened is the low level stuff and it's a big drop because the people who are only, oh, you know, four gigs of RAM is tiny. You never use it without the 37 dependency kind of stuff. And it would get you, people like you and me were talking in this way. You are crazy for not expecting the environment to have four gigs of dependencies. It's like, I would like to use it without it. But you and who else? I don't know if anyone else is doing that. That's a good one. All right, all right. But I think that the idea of the UK is running on the framework. You're right. You should stay out of the situation. You know what I mean. Right. You run the flow of the PCK with the PC. That's another thing. Depending on what you want. Yeah. Okay, great. Depending on what you want. There's next console. There's drawing fonts. You can also open the framework. So if you don't need much, you can have the kernel drawing text into specific areas. You can just add to the tables and move the here and draw. And then you can draw boxes around it on the framework. I don't think that's happening. I would like a hot application. I'm not saying it's going to be different. You can make it look like it's going to be well. I mean, keep in mind, does GTK and FrameBuffer have basic window management? Or is it all? Right. But can you run GTK and FrameBuffer with a window manager without X and without Wayland? I mean, how do you run it without X with a window manager? With a window manager. Let's quickly finish those. That was just really correctory where your touchscreen is dampening from same file to same file. We don't actually have a brain in this. We're just, which slide do we want to go on? Yeah, I don't know. I guess. You're right. GTK is one of the things that we want to survive back there. If you're doing that, you don't even need that. You just have a brain while I am. I guess. Alright. Networking. Anything good? Are we happy with whatever we have there on small scale? Just manual commands and WEP if you need it. You can pull those things in manually or do we want... I have the feeling there's something missing in the Linux system as well. We have a network manager that is usually bloated and you have command. But if you go for the primitive, there's nothing available that is more comfortable. Exactly. I'm sure it would work. The WP is the thing that I thought, uh-huh. Well, I can't pull it off. That has to be... So, like, you can't do the other thing when you have to... You can't pull it off. You can't pull it off. You can't pull it off. You can't... You can't pull it off. You can't pull it off. You can't pull it off. All right. I think we are getting kicked out. I see a big stop sign coming in. Anyway, they were nice that they let us through so many times, so... So, thanks, everyone. I think we made half way progress...