 Good morning, everyone my clock and the clock at the wall says it's nine o'clock So maybe I start slowly Hopefully you didn't party too hard And if you really did and need to lay your head down to sleep a little bit. That's okay. I Know how it is sometimes with party So I'm giving the talk what's up in the land of the Linux kernel You might wonder is it guy standing there actually a reliable source and if you look at the kernel commits He's not I actually have I Have actually two two commits in the kernel and the third one is actually on the way to for the part 5.1 And these are actually quite small commit The third one is actually on the way. That's a proper comment So that's the reason why you should run out of here immediately The reason why you should stay is that I'm covering the Linux kernel as a journalist For about 15 years now I'm doing it quite throughout Lee actually. I'm because I'm a journalist for German computer magazine Quite well-known German computer magazines. We still have those there and Actually, my stuff was translated to English On the open for a while, but that closed down a few years ago, but some of you might remember it The other reason why I might be considered a reliable source is because I'm was or I am involved in In the Linux kernel development a little bit. I did regression tracking for the Linux kernel That's so if you had a problem and told it to kernel developer Sometimes they get they fell through the cracks and I tried to make sure they are not forgotten But it did it did that in the spare time and right now I don't find time for it I hope hopefully I find time for it again Maybe soon or later even somebody might pay me a little bit for to do that. We may maybe then I can find more time But anyway, we are here to listen on and learn about what's happening in the Linux kernel a quick orientation That the Linux kernel development is working like a world tuned engine that are actually not my words That's what Jonathan Corbett from LWM net always said I've it's fitted quite well that's why I picked it up and To show that the new Releases of the Linux kernel come out every nine weeks and sometimes if there's some big problems or something then it might take a week longer And even rare occasionally rare occasions It might take even 11 weeks Each of the new kernel brings about 13,500 changes sometimes one or two thousand more sometimes a little bit less and on average The current Linux kernel sources grow by about 300,000 lines each release And I think we are above the 25 million lines of code limit these days You might know this paid from the Linux from kernel.org that shows what's Actually cooking to get get into a little bit more detail is Linux 5.0 is under development Small reminder 5.0 sounds like a big new version, but it's not it's just another main for for thought 21 it's just like if the second number grows too big if Linus He says always if he if he has not enough fingers and toes to count then he switches the first number So 5.0 is an ordinary kernel even if there's a zero in it The release is actually expected at the February 25th, or maybe a week later For the 20 is the latest Linux kernel right now or for that 20.5 to be precise Actually was merged into Linux into Fedora 29 about a week ago as a proper update and Fedora 28 gets is right now. It's in testing so it will get it soon 5.19 is the latest long-term kernel long-term constant normally gets released get supported for about two years The 4.20 is maybe just for me three months about supported, but for that 1942 years Because it's a long-term kernel Actually, it will be used in Debbie and 10 Buster which some of you might might use and because it's used there It's likely that Debbie and maintainer will pick up the support for this kernel and give it Supported till Debbie Debbie and 10 is supported That's basically five to six years about that's at least what happened in the last few times And there are actually a few kernels that Linus kernel developers and themselves Support for six years these days you can find information about that on Kindle.org if you want to know more Well eight which is in beta right now if you're curious the car that's based on Linux 4.18 actually It was already modified like like it's all always the case for a rail Upstream actually this car is not supported anymore But that doesn't matter much to a redhead they support the kernel themselves anyway So what's happening in the Linux kernel work recently as I said well tuned engine, but there were a few hiccups recently One of them that was directly in the kernel. It was more related That was in January and set of us on Linux stopped working the first pre-release from Linux 5.0 and And That's happened due to some housekeeping tasks the Linux kernel developers will remove something set of s on Linux needs something related to using your the floating point unit in your CPUs and That's that happens sometimes Because set of s on Linux is under a different license from the kernel. It can't get merged into the kernel proper and and The kernel developers us don't care what set of s on Linux does and then that's why they remove this It's something set of s on Linux has to deal with Actually, they what they removed there's some some replacement for it, but it's exported on with a special tech so it's only legal to use it for modules that are GPL license or GP compatible to GPL and Yeah, that's why set of s on Linux can't use it work up work around for set of s on Linux is under discussion what It hasn't been much. I haven't checked in the last two or three days yet, but I don't think it was The thing is because you can't use the FPU anymore might have a performance impact because they use the floating point unit for check summing But maybe the performance impact won't be that big But it's a good reminder why we lying on set of s on Linux has grisks So you should think about it a little bit more than it's instead of blindly using it While talking about this GPL exports for the Linux kernel I want to mention Linux 5.0 removes another exports or a few few other Exports that are used for that are Important for heterogeneous memory management. That's basically What a lot of you might heard have heard from AMD with the fusion stuff like all the different chips in your computer That have their own memory to make a big giant giant address space so it's easier to move things around and to From a CPU for example to get data from your GPU memory and And that's what a HMM is for and there was also an export that they could use up till now and that was changed so to make it only available to GPL modules and So that will be important for GPL It got nearly no attention But it might be more important than the set of s on Linux stuff because the NVIDIA stuff is important and HMM is for a high-performance computing Artific artificial age Intelligence and machine learning remains to be seen how that works out and what's even more important and unusual This exports were changed in retrospect because they existed and they would change to GPL exports and That's normally not not the case that normally doesn't happen Normally, that's only if you do do some housekeeping some improvements in the linear problem You can change that but in this case it wasn't like that for for some reasons that I explained an LWN that if you want if you're curious But these GPL exports were actually were back ported So even older kernels like 4.4.14 and 4.19 now get removed remove these exports and Might break some use case and video might have wanted to Use the Linux kernel for actually it doesn't break the graphics driver right now because the HMM stuff is not Finished there yet. That's why it's not built up to now another hiccups that recently happened with the well tuned engine is block corruption Buck that was in 4.19 that actually happened in November and was a hard to trigger Buck in the Linux kernel when you have under under low memory restrictions or something like that There was something that written probably to disk and some corrupt corruption occurred in the beginning It was suspected that it was an X4 buck but it turned out to be something the block layer What I want to say about that is yeah, it's nothing to see here bugs happen get fixed. Please move along. That's some The kernel developers You can't stand still they have to do and make improvements and And sometimes things will get wrong because in this case it was really hard to trigger so it's Think like that happen you have to have to be prepared for that and we all have backups don't we do you For me it was at least a good reminder to take my back up strategy Don't let the scare you off There will always be bugs and if nobody tests this carls throughout Lee Nobody will find that those bugs and the kernel will get worse in any way So just make sure you have back ups and don't get scared and try these modern kernels in production even There was another hiccup recently is also November that was performance problem a big one made the news and was Mentioned a lot of internet websites that happened due to improved specter v2 migrations. You remember this this big performance bugs Security bugs that are in a lot of modern CPUs that were found about one year ago early last year and there was an improvement to this mitigation thoughts that got back ported a little bit quickly and That's made the performance worse. So it was even in 4.19 4.14 Fall fall about one week or something. It got reverted there in the end The story is the kernel developers already had worked on even more improvements in this area So then those patches were revisited quietly turned around and changed quite heavily The performance was restored the mitigation afterwards is even better and the Mitigation patches were even back ported too. So everything is fine What the kernel developers learned from it that they are In cases like this keeping an eye towards performance to make sure something like that doesn't happen again right for line. Yeah The question was about the performance impact of the inspector v2 migration actually we and later in the talk I'm talking about this and But I can talk about this now They are right now in Linux 5.0 to afford to make the specter of v2 mitigation have less overhead Some technique in the networking layer benefits from it and UDP package performance also gets gets bigger and they're actually two more two more improvements that kernel developers are working on to reduce the overhead of the red poline stuff that Introduce the performance problem, but in the end it's there is a bug in your CPU You if you try to work around it by software, it will get slower. That's It's unavoidable unavoidable. You can only try to reduce the impact as much as possible Yeah, let's move on to the next Hiccup that was Linus break and the code of conduct. I think most of you have heard about it That's why I'm not going too much into the details. That was in September October made the news sites throughout the up and down Linus took a break to rethink his behavior towards others because with other developers He was quite blunt sometimes like that that was in August Actually, I found that really really funny But I was raised with Monty Python because that's if you if you see this down here. It's it's a quote from The Holy Grail and if you know that movie you will get that and you know what he's referring to but some people I mean the modern generation. I Would hope they learn about Monty Python, but I don't think they do so it will get people will get that wrong and It's just one example To be precise, I think Linus Thomas improved his behavior in the last few years a lot already And I think if you're making such jokes before among friends, that's not a problem The problem is you're doing it in public and there are lots of websites Yeah, it's it's a complicated topic as you said some people that are not involved come up and and require changes But I get to that. Yeah, wait a moment. I'm getting I'm getting to that Actually, the thing is Linus when when he moved he put a code of conduct in place as you know, they are Pretty popular these days in this conference some for on many countries other conference have one these days and actually left for a while back for a hard man took over and When he did that a lot of bike shedding and concern Came up there were reddit threads that was so big and when the websites news sites reported about this and lots of concern But also the concerns were sometimes from people that are also not involved in the Linux kernel But so people people that wanted that the little kind of implies a code of conduct There were not always members of the current community and there were People that require said they know no code of conduct. It was also not not Part of the kernel community within the kernel community within the core team of the kernel developers is what the Discussing was also heated but not that that much anyway The discussion worked somehow and it was the code of conduct was fine-tuned a little bit and the surroundings were fine-tuned there was an document added to Explain how the code of conduct is work and there's a mediator now if there's something happening and Yeah, that's how it is that was the improvements came added to the kernel to Fordot, oh, what was it 19 19 right before it was finished and Linus came came back as planned Yeah, it didn't Silence the discussion there was still concerns and bike shedding But in the end one is perfect if you would switch to a from one maybe from MIT license to GPL There were discussions always that's always the case and the thing is the Linux kernel developers accord team There was a kernel maintainer so some it recently in October Pretty much said it will stay at it is for now and we see how it works out and in the end it's a signal like we are friendly here and That's important these days because due to the males like the one I showed you earlier Linux kernel has a bad reputation that needs to change because times change and The Linux we are all not like 10 or 15 years ago Code of conduct kind of is important these days and to give an example However, this is also from Monty Brian Monty Titan who knows this I guess That's that's from life of Brian where she says from now and I want you to call me Loretta That's a quite popular scene from that movie and But I guess they wouldn't do that today Because things change back in the 70s when that movie was made you could make fun of people that said Hey, I want you to call me Loretta. You wouldn't do that these days anymore. I mean you can but I don't think you would because it's not not For two days well, it's not appropriate anymore And that's the same with the code of conduct in the Linux kernel and in the end remember if problems of the code of conduct Come up it can always be Revisit because it's still the old wanted to say old Same old trusted people in control. It's and if there's something with some problem with Linus and Gregor Hartman can say Okay, we change this to get to solve a situation Where where the code of conduct turns out? To not work as intended so don't worry too much about it So that were already 10 20 minutes about the limit colon Quick distraction. Why am I here? Few guys might know I Me from the old days like for door like 20 years ago I did a lot of work in for our ex suspect then when we had cool effects on the desktop. That was actually fedora for six and Was the first non-fascot chair not working for it had and did a little bit and apple and rpm fusion bring up And these days I'm doing not that much anymore for for door as I said I'm doing the regression tracking that takes a lot a lot of time. I'm also having a badminton club I have to manage and my regular work also So there's not much time anymore, but I'm still quite interested in redhead and fedora. That's why I'm here The one thing I'm still doing for fedora is I'm building Vanilla kernels for fedora and not doing isn't not doing isn't a couple I'm doing it actually publishing them on people dot fedora project for dog There's actually a page in the wiki that shows how you can can use those kernels and Actually, it's an easy way to simply run more up-to-date kernels Like run the latest mainline colon like 5.0 right now to check if there are bugs In there that the current developers can fix Actually, if you use that the fedora kernel will benefit from it because if it's fixed early the bug won't happen Won't actually will never show up in regular fedora because it will use those kernels later Actually, you can also use those repositories to check if the bug is fixed in a newer version Or check if the bug you're seeing with fedora is present upstream to be able to report the bug upstream in case you want it It looks like this looks a little bit complicated because you There are like four different rep for streams for different for for the current Fedora releases if you want to Get into the details. What's what? look at the wiki and Basically, it's two commands to enable and be an effort will do the lack work later It will simply update the latest kernel and if you google or another use such search And then if you google for fedora Kernel vanilla repositories you will find this page that explains all the details and has an FAQ FAQ that explains why it might be a good idea to use it or why not So that was a short distraction Now I'm getting a little Switching the gears a little bit now. I'm showing you what's happening in the linux kernel right now I have a lot of slides as you already noticed and I Won't make it through all of them about if we can stop at any time It's kind of a worldwide view view from 10,000 feet or even higher to see what's up and to look at some interesting points like where we are now and but As is even there and will be pretty brief sometimes It's basically to show you a few errors where you might want to look a little bit closer If you're interested and I'm trying to look back and looking ahead a bit depending on what it is so networking One of the we're about distractions. I think I need something for my voice so in networking one of them Hot topics right now is wire guard. That's a promising new VPN tunnel solution like open VPN or IPsec to connect different systems and It's I won't go too much into the details. It's stateless That's important if you switch from one network for the other the VPN will will will stay There won't be any Service outage. There's easy configuration deep integration in linux Quick reconnect a lot of benefits and if you want to know a little bit more about this There's an ask technical article about it If you Google for wire guard ask technical you will find it And actually the one interesting thing is line is actually what he normally doesn't is Actually stated on a public mailing list that he likes it and wants to see it merged That was already in August last year. It's still not merged The reason for that it's this wire guard is only I think three or four thousand lines of code and But it relies on new crypto library for the linux call that's called sync and That is way bigger. I don't know how much lines of code it is but It's basically creates a new API in the crypto subsystem in the linux condom and the crypto Maintainers don't like that too much because they don't want to manage two APIs So they have to find a way to integrate those two and that takes time if you're interested in more There's an LWN that article about this. It's from October or November. I think since then didn't much happen, but I think in new version actually is up for Seems to come out for reviews soon. So maybe there will be some things moving ahead soon package filtering or firewalls as everybody says Remember NF tables or NF with the tool NF tape for configuration It was started in 2011 and really worked on since 2013 a lot of people said made Made a lot of people say hey that never really lifted off But it lifts off now. It's slowly and silently especially takes over and One way how that works is IP tables use IP tables tools package these days installs the NF tables and legacy Tools in parallel and actually sets a link So if you're calling IP tables, you in fact might like in this case, it's using alternatives Using the IP tables version that's Using the NF table stuff in the Linux kernel. So you might have switched to NF tables without noticing it and Actually, that's in progress firewall and the 0.6 actually Switched to NF tables already and there were some problems with lip weird and Docker integration That's why open susan tumbleweed and fedora 29 switched back to using the legacy IP tables and tools But well eight Debbie and ten Will actually use the NF table stuff by default and fedora is also looking into it Then I'm not sure about the time frame. I thought it was fedora 30 But might be that it's fedora 31 that will make the switch you as I said You won't notice much of this But if you're running a big firewall, you might want to part your you your rules because if you're porting your rules to NFT you might have additional benefit benefits because NFT and the syntax is a lot more efficient You might think about might have heard about BP filter. That's a new New filtering technique for the Linux kernel So you might think hey, why should I port my my rules from IP table syntax and NFT NFTs syntax these days if there's a new contender on the horizon already thing is BP filter is not much more than a proof of concept right now and So there's still a lot a lot of work to be done before it can be used for firewalling and The important thing for for for the consideration if you want to part your rules. It's changes BP filter changes internals So in the end you can use today's NF tables NFT tools and IP table tools to work with the BP filter So you don't need to worry too much about the BP filter if it's really starting off and and and getting Becoming a shooting star you can use your rules and tools like you do today and you will have the benefits from the NFT syntax with With BP filter as well. That's how it looks right right now as I said, it's Working changes internals and the code is actually Looking at the packages that are coming in it's actually a Customized code that's generated on your on your on your system with a Local compiler that looks at the rules and then creates a program that does exactly what you want that avoids overhead and makes it so quick that's one of the reasons why it's so interesting and That code for example, it doesn't exist yet. That's actually still the plan How that's that's supposed to work. So remains to be seen if that Works out, but it really promises quite quite a lot of performance improvements. I Mentioned this BBF or eBPF Without explaining it really and it's basically a small VM in the linux kernel that was all added over the past few years VM like a Java VM not a GVM VM So it's basically code that you can can run within the linux kernel, but the thing is You can write those programs easier than kernel code You actually can write those programs on demand or by the other tools and that makes things much more flexible And that's what's why it's important for example for the BPPF case because That's compiled with a specific user space helper on demand But the really new thing is this user space helper to generate your firewalling code basically This can be shipped with the linux kernel in a specific module that has kind of a own user space That's a little bit complicated. It's like You have another root partition where only is a compiler for this Code in it and that If an appables or IP tables roots come in this code will generate an eBPF program and then we'll back Upload this in the kernel. That's quite a new concept And remains to be seen how that works out and in the end it's The effect is the boundaries change Linux becomes more flexible and and it Not that traditionally at anymore. It's not a monolithic kernel. It gets even far away from it and they actually even talks about Using file systems in the fuses file to run those in in eBPF because that also promises to be have performance improvements It also allows pushing things out of the kernel on a dedicated error But which is still under control of the linux kernel developers and not under the control of the linux distributors We may have to be seen how that all works out. Some people say it's The linux kernel gets a more micro kernel like Maybe it is maybe maybe it will maybe it won't we can talk about this again in 10 years time would have The important thing from now the eBPF Is used in more and more places. That's something That's if you haven't heard about it, you might want to keep an eye on because it's so important and Used in more and more places. That's why it gets mentioned so often during the talk and to give an example Or how that looks like there was a linux plumber's conference recently That's one of the most important news conferences in the world where developers get together and they were actually Didn't count but it doesn't Talks just about the eBPF and what it can be used to what can be used for tracing Networking and and a lot of other stuff. I'll get through it in the talk if you Google for LPC to 10 2019 and eBPF you will find this page about it's also mentioned on the slide below and Slides go online Anyway, so you can just look there Networking again XDP is also a hot topic That's XDP is a kind of expressed It stands for express data pass and it's a network path path That relies heavily on eBPF it basically allows you for example to write when the Package comes in to your computer on the network card before it's really forwarded to the network stack in the linux card To drop to look at the package and say drop this forward this revise this or send this send it to some application and that's Makes it a lot That can improve the performance a lot To handle specific cases and all the other Packages you don't get on that this level get through the normal and networking stack and sus Get handled there normally That's really quite interesting that network developers do a lot of them Facebook uses this for example a psyllium Catran psyllium is a container Networking technique that's also usually heavily relying on it. It seems They are popping up use cases for XDP all the time these days and you can say see that also on on the linux plumber's conference program there was recently the networking track and As you can say see not every second, but nearly every second talk was about bp Xtp there here are a few examples If you want to these these talks there abstracts There and some of the video recordings are on the web if you want to look into this stuff You can it's a great resource to learn what's happening there the most important thing right now. That's happening. There is Zero copy support for AF Xtp. That's a technique used by That's Technique at on kind of for it that that uses Xtp to get raw packages right through from the car Bypass the network stack and get it into the your application and To speed up processing as you as I said for packages you're interested in That's might sound familiar to some of you. There's a data plane development in DB decay making a lot of Getting a lot of attention in the last few years and together with a Xtp. That's might be possible that This gets more under control of the currently again because DB decay is Kind of strange sometimes because it's a real kernel bypass and the kernel developers don't like that too much storage As in kernels I own Is something where Linux has a bad reputation on in the windows world is quite normal and That's something where the developers are working right now as a polling infrastructure and That's it One developer recently Improved and actually this API is Kind of inspired by one or was used by similar in redhead advanced server 2.1 Who still remembers that was around there one? Yeah, even I I think rel 3 was the first I had contact with So so long ago Yeah, but they're really important change right now is still in work in progress. That's this I owe you ring It's a user space ring that's a structure to exchange data between the kernel and user space program using a ring buffer and That's cooking for like one or two months these days So it's really earlier not much in the kernel, but it looks like we'll make AI or proper first-class citizen in the limbs called because it's quicker easier to use and less quick less quirks and That seems to be really promising and it's also an answer to get to avoid or Bypass technicals is a storage performance start development kid is also kind like the dbdk I mentioned earlier where some app Actually bypasses the kernel And just I owe technique is supposed to to be the answer to that to make sure the kernel Stays in control. That's where we like it or where the kernel developers like it But I said it's still young and work in progress. There's an LW and the article about this simply go there Storage what also changes is You might have heard about CFQ and deadlines some schedule us for your For your storage device they actually vanish with a linux 5.0 And they actually vanish with rel 8 to because the changes that are responsible for that are also in rel 8 and Actually, not only those schedule us vanish They actually old legacy IO path and the block layer got Removed because there was a new block layer infrastructure Created over the past few years. It's called block multi-Q block MQ because That was important for the one NVM E SSDs showed up because you have to use multiple threads and multiple CPUs to get the performance out of these and that's why this Stack was created and that's now used everywhere and Then you schedule us for it. It's called one is called B of Q and the others called MQ deadline They also have new tuning knobs. So if you really into Storage performance, you need to learn new tuning knobs If you just want to use it your old school SSD might benefit from from the Multi-Q architecture and your HDD might benefit from the new scheduler And normally I talk a lot about file systems here. I'm doing this talk nearly every year and for example last year I Mentioned there was an xvf XFS revamp in the works What makes? That adds some cool features from butterf s to XFS That's slowly progressing processing not much talk about right now It's the same with buckers as there's mainly fine tuning these days the file systems space What's interesting right now is word IO FS and it's a local file system exchange between VMs and the host or other VMs Some might know about P9 Which is used for that was it 9p 9p or P9 I kind of mix it up Yeah, 9p. Oh, thanks And whether you FS makes this more efficient that sounds really promising and something also overdue But it's also quite early. I think it was posted early December and That's remains to be seen What also was also what was already added to the car is some diagnostics infrastructure That's called PSI or pressure stall information Nobody can remember this I guess What you want to remember is it's a proc load average on steroids So it's really if you have used load average to see hey my why is my machine slow? Okay, or use too much you it doesn't help you see it's use that much of it You don't know why and the cool thing about PSI is that CPU mem and IO are separated That looks a little bit like this and it's a bit strange in the beginning Here our for example is in the top line is the CPU already It's a CPU just like a load average you have the 10 seconds 60 seconds or 300 seconds And it actually says how much the system was stalled and Didn't get to run the CPUs you the processors he wants to wanted to use So the process had to wait and there you can for example if the CPU was overloaded you will see that this line and Now you can actually see Thanks to PSI if I owe our memory are the problem because this stats are here for for this IO and memory also and The thing is you don't have This some line here, it's called like like some some processes Got got stalled and didn't do we're not able to do their IO and had to wait then it's added to that line and in the full line It's actually if basically nearly all Processes got stalled because there was so much load and if the full line It's really has high numbers here And then the next it's actually Pretty likely that your machine will lock up soon because the con is has so much work and In its queue for IO and can't get it fine find time for it and that It's pretty likely to lock up Yeah, how much time do we have trend? Okay? Yeah What one minute then let that it was planned like this that I can Jump to the end at any time one thing That's also in the works right now is that real-time news are tea patches Few of you might have heard of that's basically at the real-time Capacity to the Linux kernel It looks like those patches actually will get mainline this year They're working on on on those patches for about so 15 years right now to make Linux and real-time operating system and The current developers are not finished yet, but they're also Preparations for the time after already if you want to know more about that Yeah, there's was a talk at the embedded limbs conference Europe and and and October last year Which explains to what what is still missing what the current developers want to add over the year before the next long-term colonel comes out and It's a lot of good Source to to understand what's up there what real-time actually is because real-time doesn't make Small reminder real-time doesn't make your system faster. It makes it predictable when something happens to and actually some things might get slower Due to real-time for example, I all regular I all might get slower to make sure The real-time tasks are running within the deadline So it's kind of you can't have everything at once you have to slow down something to get some Other things as some other benefits in a different place if you take a ways Want to know more about this stuff use the search engine? There's Lots of more details about this actually this presentation has a bit more slides It will go online. You can look up what I wanted to talk about if I had more time and It's pretty self-explanatory But the main important things is networking it's VPN stuff the wire got stuff is really cooking xdp is Quite important changing a lot of things might might be really Something that's pretty normal in two or five years from now that everybody will use In storage, it's asynchronous IO and the block MQ changes that are important Both like just like the xdp stuff. It's our solutions against kernel bypass technic is That the kind of developers don't like that much as I said EBPF is involved in this quite a lot. It changes changes the kernel quite a lot EBPF is also used doing for a lot of tracing stuff I didn't have time to go into that details But also quite interesting there were also talks about this yesterday So look up the videos later on on the on the web from the room yesterday It's quite a lot of things to learn Security is also never-ending story like I mentioned with the spectra stuff as I said improvements in the work to get To to to make sure the performance gets back Yeah, don't fear the code of conduct. It's not as bad as a lot of people might Want to make it sound and one thing to take away is developer development in this kind of does not slow down. It's really Watching millions gone for 15 years and it's really quite happening quite a lot I could talk for maybe three or four hours if I had the time But I then all of you would be boring and sleeping here. That's way too much if you want to keep up I can only suggest Follow LWN nuts at the LWN net as I've grabbed to it It's really a good source to know what's cooking in the Linux kernel and related areas And you might want to follow me on on social media Actually have an account dedicated to kernel stuff. That's a kernel logger account It's actually on Twitter and also on friendica and friendica can be used from mastodon on diaspora as well Actually, that's just one of six accounts. I maintain I'm a bit crazy, but I'm a Google plus refugee and there you had collections to to sort the stuff You're talking about and that way I won't Know you a funny cat pictures in the kernel stuff because I'm having a dedicated account for that Yeah, that's it Please provide feedback. Tell me if you liked it what you didn't like and leave feedback for the organizers They want feedback also Yeah, that's it. That's it By the way, that's what would have been slide 230 if I they hadn't skipped a few in the middle Are there any questions I make I did I make you tired and sleepy So the question there were actually two questions. You actually fear that the kernel is getting too big kind of and So yeah, the current growth that that's true And there are sometimes subsystems that are not that well maintains Maintained, but it long as long as those subsystems don't have security issues they are not a problem and The kernel is a size is also not not not a problem as because I mean if the source go go to 30 million lines or something in the end what matters is that you how you can configure your car to be dedicated to your use case and so that's what what matters and I think I need to leave it at that because that's a complicated topic. You can't get simply explained in one or two months one or two minutes and because it's It has benefit that's everything in the kernel and the set downsides and that coming to your second part of your question Actually, as you mean you ask about stuff. That's getting the boundaries change and ebpf makes things going to user land Linux kernel developers actually want stuff in the Linux kernel because having everything together is important like if you have say USB 4 comes out and Needs some changes in the subsystem how they work if you have all the drivers together you can do these changes you can change the API because you have all the drivers in one place and That's why the kernel developers normally want everything in the kernel And that's also for example why the kernel developers with the BP filter stuff created this this mechanism to have some things and in modules to user space things in kernel modules and to keep us keep them under control because of this distributions and Control them then strings spread out and things start to stop Breaking so the kernel developers are normally interested to get as much stuff under their control as possible Okay, right any more questions No time time stacked How many processes? How much how much time processes got stuck because they were stored because they couldn't do it I owe because something else was doing I owe That's it's not easy to explain I actually had had to look up the documentation also and read a few minutes before I get got a handle of that Actually, it's in the kernel documentation read it up and then there's also an LWN net article about it and Actually was for Maybe it's of interest it was developed at Facebook and they use it to Increase the uptime of the servers because they can actually see when processes Going haywire and Before and kill kill it before it's a cash register system Ask me or I think it was on the all systems open System de-conference the video is on on YouTube when you if you look See you and you need to write me a mail and I can point you to it. Okay. We are out of time. Thank you Enjoy the conference