 There we go. Okay. Hi, I'm Greg and I am the maintainer of USB and running development kernels Causes this to happen and you lost my US. Did you lose my video too? There we go. Okay, video is good. All right. It's good. Okay. Um, let's talk about maintainers and getting your paths is accepted I did this talk As this Title as well sometimes I don't really want your code. I'm I'm gonna talk about why maintainers Don't want your code and what would you can do to make sure that they accept it? The big thing is when a maintainer accepts your code, then they Take responsibility for it. So you have to make it very easy for Somebody to be willing to accept responsibility for work that you've done And I'm gonna talk about what we go through and what we do as far as maintainers and what you can do as developers So this is the size of the kernel As of the last release 5.9, which was about four weeks ago. No seven two months ago And we're at 70,000 files 30 million lines of code Please remember you don't run all of this you run a tiny subset my laptop runs about 2 million lines of code Your phone runs about 3.5 4 million lines of code But the entirety of the kernel all the drivers all the architectures everything is together in one big giant source code One big tree. So this is the size, but this isn't the size of what you run So that's very people get confused about it thinking it's getting huge and bloated. It isn't This is the size overall if you look at the size of what your laptops been running for the past couple years It's been pretty much the same This is the amount of developers we had last year At least 450 companies. I haven't updated my number How many people are actually How many companies are participating I need to go do that but this is about it So we are the lot. We know we are the largest software Development project out there with number of different developers from different companies out there. It's huge This is a giant giant group of people. Nobody works for anybody else We all contribute and we all contribute to make things better for ourselves And that's fine because it works out for everybody that way This is our rate of change. I like showing this. Um, it isn't that bad. It's kind of not Unusual until you look at the units and this is what happens every day 24 hours a day seven days a week This is what goes into the kernel. This is averaged over the whole release cycle, but it's huge We're going very very quickly very very fast But we're used to this. This is what we've been doing for a long time. And in fact We keep going a little bit faster. This is number of changes that are happening per hour This number the links foundation told me not to say scary, but every year. I'm like, there's no way we can go faster than this and we do Go faster than this if you look at the 5.8 release we're up to almost 11 changes an hour This is very quick. This is very fast This is a huge huge development project that's moving very quickly and you need to remember that when you try to Participate with it and you need to remember this if you try and not participate if you want to stay away from it And you want to try and keep your changes outside the kernel if you graph this over time over the number of years We've had it slowly goes up. It's kind of plateaus, but it has been going up every year every time so this means Please always remember that if you take a one-year worth of development cycle from two years ago And then look at one year worth of development this year We did more work So if you kept your code outside the tree you will have more work to try and do to catch up to us It is in your interest to catch up and merge with us and participate with us in the community Otherwise you will get further behind faster, and I don't think you want to do that So let's talk about how we do this really quickly like we said we have 4,500 different developers They make a single change and then they want to contribute it. So what do they do? They turn it into a patch It's a very small text file and they email it We still work with email because of our huge rate and a huge number of people are doing it tools like github tools like Garrett just do not work for us. They don't scale properly And those people they all send the patch or the change off to the owner of that file or the maintainer of that subsystem Every single patch or every single line in the kernel can be traced to who maintains it We have a tool you run the tool on your on your patch And then it says okay, you need to mail it to this person this person in this mailing list They email it off those people review it. They accept it. They reject it They push it back and then those people if they accept it Then that passes off on to the subsystem maintainer now for those drivers and file maintainers We have about 700 or 800. I think now listed in the kernel There's that many different Individual owners of different parts of the kernel and then those people push them off of the subsystem maintainers and these are things for like usb bci tty layer memory management specific architecture specific subsystem that way Networking and these people all have public trees and these public trees They're listed in the maintainers file in the kernel And then they accept the patches and then those public trees are published every day and then We have tools we have automatic testing tools that suck from all these trees. They test everything They give us feedback and um, you can see whether your change has been accepted or not And that's how things work and at the subsystems. We have about 150 200 maybe different subsystem maintainers If that but there's about that many different trees out there and all these trees every single day Get pulled into something called Linux next and Linux next is done by steven rothwell and um in australia and um He takes all these and merges them all together and then reports on complex because we do Modify things in other subsystems a perfect example is some irq handling stuff just got changed So broke a drive driver and one of my subsystems because I didn't have those related changes We get notified of this it tests it it builds it a boots it and it runs on tons of different architectures Andrew morn I kind of put over on the side here because he scoops up files and subsystems that nobody really is maintaining nobody lists Um, he takes all those and those go into linux next as well This happens every single day So the patches again start from the developer go to driver file maintainer go to subsystem maintainer go to linux next And next next is if you want to see what is going to happen to the next kernel Run linux next if you want to get involved in the kernel run linux next See what is happening there if you have build problems if you have warnings are showing up If it doesn't boot properly let us know because as developers and maintainers it all works good for us But if it doesn't work good for you, we don't know to fix it So look there if you were a developer and want to get involved work off of either the subsystem tree Or linux next tree because that is all includes all the work that's already been done You can't work off of linux's tree because it doesn't include all this work that's been done by all the developer community at that point in time Excuse me always work off of the next next Okay after that, um when the release window opens up and I give other talks on how we do releases and such not But basically at the release window all the subsystem maintainers send stuff to linux and we send it all Directly we don't say linux does not pull from next we have to individually call out What we want to send to linux and we do that so that if there are problems linux doesn't get them He doesn't automatically pull everything and there's things that stage in anders tree There's things that stage a number of my trees that we just don't aren't ready to go to linux Sometimes a whole subsystem isn't ready. I had one time that one of the tty layer Some core changes were not working well. They kind of worked. They didn't there were still enough problems the merge window opened up I said i'm going to skip this merge window Send all linux a bunch of bug fixes and we'll take this next On the next cycle in another three months if linux was pulling directly from next he would have gotten all those problems That's a much better. It's a push. We explicitly ask him to do something and it works out much better than automatically happening This is what we do. This is what happens every single day every single release cycle And all the time. This is how we get things done um Are there any questions? I think I can kind of see questions Maybe not No open questions. Awesome No hands raised. Okay. That's easy. Um going on I like showing this for a number of different reasons This is the top developers by quantity not necessarily quality But for last year, these are people who've done a lot of work and I want to call out So christian will chris wilson's um graphics christ christoff helig file systems and block layer morrow video for linux takashi sound And these uh graphics You and colon and geert and gustavo I want to specifically call out because these people are going through the whole kernel tree Finding common problems and fixing their janitorial work their low-level People would think it as um not very glamorous work It is very very important for us. This work is required by a kernel But it's required by a code base Especially a code base that wants to survive that it gets a constant maintenance a constant update Collin goes through and fixes spelling fixes that he updates his spelling fix tool and pushes out a bunch more Geert fixes up tons and tons of um quality of tiny little fixes based on different things gustavo Tons of really good. He finds a security pattern That's pointed out by coberty fixes the whole kernel tree with all those issues. It works out really really well Um, that's a very common thing. It's a project. It shows that the project is mature A project is very active and it is required and we also really appreciate we want that kind of work So if you want to get involved do these janitorial patches do this type of work and we'll gladly take it This is something we really need. This is gives you a sense of what people do in about a year worth of time and also gives you a sense of um, where things are being changed Um, to look at maintainers we talk about maintainers sign off when they review something so um david miller does networking and um Few other subsystems. I'm spark ide Scripps of patches and reviews and I do a lot of staging drivers and other stuff alex Those graphics. I think mark does sound morrow does video for linux Linus and andrew so andrew sends all patches to linux and email form So linux reviews them and applies and applies a lot of other stuff But linux also doesn't apply patches or he takes mergers directly from other people But he doesn't review them all And that's the point I want to try to make here if you look back at our old this old link The people below are once a maintainer accepts a file or accepts a change They are now responsible for this so every point along the way you add your name to it So I take it from a usb serial driver maintainer who takes it from a person who submitted the patch We all have our name on it. So now we are responsible for it And that's signed off by and we need to take those things into consideration So you when you send something to us we have to know that you'll be around to fix them And that's the most important thing I don't necessarily trust that the people underneath me who send me stuff all the time got it right always But I know they'll be there when they get it wrong because we all get it wrong sometimes Just that's the key you need trust. It's a human interaction. It's a relationship thing So when you're starting out, it's hard to build up trust And it's also because of that it's really hard to change core parts of the kernel Because it requires that us to trust you that you got it right Basically, so coding style and other fixes are very easy to take and very easy to accept Core changes in the kernel are harder to get in because we need to trust that you're going to be around to fix these When you get it wrong we've had examples where people have abused our trust Used our trust in networking subsystem Most recently an academic tried to abuse our trust by sending us cleanup fixes for error paths and actually tried to Inject errors and inject code into the kernel. That was bad. That was not good. We caught that But don't abuse our trust because if you abuse our trust, we will not like you We will we will blacklist you and you now are not allowed to work on the kernel So these people maintainers we review this code and then we're now responsible for it So maintainers David Miller gives the great quote. We're like an editor. We Do I'll talk a little bit more about that. We we were developers in the past. We know how to review code We tell other people we say this looks good. This doesn't look good Why don't you go do this instead and by average we accept about one third of all changes that are sent to us So if you look at our rate of change and look at how fast things are accepted Realized two thirds more code is out there being Submitted to us now. Sometimes it's just the same patch that over and over and over But most of times it's just um, it takes like two or three times to get the code accepted But that takes review that takes development That is a lot of effort out there that's happening that you don't see in the accepted amount of code That's being merged. So also don't get um Don't get discouraged when we don't accept code right away because we want to make sure that you're going to be around We want to make sure that things are right and that you can accept change and that you can modify things So these are different subsystems that you'll see kind of common Again yens expo for block martin for skezzy These are kind of areas of the kernel that change a lot. This happens that we do this for the past year So, um, who's funding this work? Um, we didn't do an article. We didn't do a um, john corbett and hyphenlinus weekly news didn't really do a A paper last year, but this is the last years worth of work Amateurs did about 13 of work and these are people that we know they don't work for anybody or we don't know who they work for Um, those people normally do not do more than five changes anything more than five changes We know who you work for it's that simple Um, anything more than five changes you can get a job usually if you do real contributions Intel 11 percent. This is not that many patches. This is about um, 10,000. No, sorry 12 I gotta look it up. Maybe a thousand patches From intel red hat amd huawei google the naro susa consultants some people work for other companies a lot of embedded people Do work for other companies and those are consultants um Melonox renaissance are more called there's nobody really unusual in here linux foundation That's kind of is a little bit unusual that there's only four of us doing this work And we're still in the top 10 or top 20. So it is possible canonical. I will call out. There's really pretty much two canonical employees that are doing kernel work And they get in the top 20 So it is possible to do a lot of good work and have your company in the top 20 list It's not unusual for that um, yeah Um, any questions about this so far? It was pretty fast Anything open questions comments Hey, okay, that's easy um Talk about the development process. So I talk about the development process in a way that is common to hardware people Hardware people create design. They emulate it. They simulate it. They take out power on the integrated Then you finally bring up the operating system when you ship the device um This is very common for making a chip making a device. Um, things like that Um, when you're creating a software product, it's also can be stages along this way before you find to make something public This is a very common thing Now if you submit the code to us and send stuff to us It takes us a while to review it takes us a while to get back to you You have to revise it and submit it again Then our release cycles every three months get emerged. It takes a little while to do that um That's how It takes a while to get things merged So ideally you want to submit your code really really early in the process If you submit your code really early in the process Your code can be accepted before your actually chip is spit out there and as public And some companies know how to do this really really well other companies do not know how to do this really well And they are very late to the game They wait until their chip actually ships and that they're using an older kernel They have to go back and take the time to catch up Stuff get stuff out to the community and it wastes time This is the ideal situation a number of companies. I will call it intel. I'll call IBM know how to do this really really well. Sony's gotten really good at this as well Um, they intel and both IBM have had code in the kernel before chips are even hit Takeout um Intel is known for having code in the kernel before chips even ship And they have canceled projects where we've had to remove code from the kernel for something that never actually shipped We don't like to ever remove anything from the kernel if they was using it But if nothing is actually shipped we don't we'll gladly remove it. Um, this is the key This is a proper way to do it is also faster It saves you time saves you money and it's documented this way both IBM and intel have publicly said if you work upstream will save you time and money So this is the fastest way for you as a company to get involved and do this work It makes sense. Um It takes a while to get there, but it will save you time money. It's a business case to work with the community Again, like I said, if you Look at how much work we do in the past year the year before that we did less So we are actually if you stay out of the tree for longer You have more work to catch up with and it's harder to do that. So it's a difficult task Please work with us work with us as developers or want your stuff We also want your changes and we want your stuff because um, everybody contributes to the kernel in a selfish way You're solving your own problems and that's fine because it turns out everybody has the same problems Um, we had numerous examples of this over the years where somebody said no This is only special to our market and turns out no it works for everybody Um contribute to the kernel in your selfish way will gladly accept it because it turns out everybody Is unique and special just like everybody else That's the old quote goes so please we gladly take your work So how do you do this? Again, Dave Miller said we are editors like in public industry. We used to be writers We used to be programmers. So we know how things work. We know how to review stuff We tell you what to do we try and guide you on the way to get your code in This is our job as a maintainer our job is to help you out We want to help you out, but don't make it hard for us to help you out. Um, that's the key thing um I went through my patch inbox a number of years ago I actually know this one. I went through it a couple weeks ago and in the two weeks I received 1400 patches to review Um, that's a lot. That's a lot of patches to review. This is my normal workload So realize when you submit patches as people that they're not going to respond to you instantly Give them two weeks to respond. Um, then you can ask them nicely if they haven't But realize that people are working on other things at the same time I have other projects to work on and there's lots of other patches out there The best way to help a patch process go is to review other people's code Please help. I would love to help with reviewing these patches. Um, I have some sub maintainers I have some co-maintenors. We all do work on this and we help out But having developers review other people's patches is the key to do this stuff Um, but here's in that two week process period I had a number of things that were just really really wrong and here's an example of things I got So I got one patch that said subject. It was patch 48 out of 48 There was no other 47 patches anywhere. We had no idea what was going on. This actually is a very very common error It makes it very easy to do this. Um, please watch out. Don't do this Look at your email before you send it out because I'm going to be say Searching around trying to find your 47 other patches and get very confused because they're not there Um, best one. I also got 15 patches in a series We want patches in a series to show you your work in a specific order and break things down into one little logical change for thing That's wonderful. That's great. But there was no order given. So I had no idea what to apply in what order Um, I you can't rely on email date and timestamps. It will fail. It just did not work. Um, please order them properly Um, I got patches number one and number three through 10. I had no idea where the other ones were That's not as good as well. Don't if you're going to copy me on a patch series I want to see them all because you want me to review them all you can't ask me to review things out of order Um, I had to sign up by in the signature, which is down at the bottom that was automatically stripped out That's not going to just work at all. That's obvious and our tools should catch that for you We had the best these have been like weekly. Um, if your email signature says that the email is confidential I have to insomely delete it because Colonel patches are not confidential. You want us to accept this in the public and to accept this under the license that we have It can't be confidential. It's that simple Fix your email client fix your corporate email to do that You'll see this is why a number of companies have an email box in the corner and they send out email through that Linux server IBM intel microsoft are infamous for having their little Linux dot domain For the email out there red hat people work around the red hat systems as well You can't send me email to set this confidential Um tabs convert to spaces. That's a very white space. That's a very common thing If you ever try and send a patch through a web client, it just doesn't work. Please do not do that That will not work. Um, leading spaces remove. This is also a common thing for web clients If you cut and paste with a mouse leading spaces are removed patches don't apply white space actually matters Um, there's a different a non-unified format, which is really interesting It's kind of hard to do that these days. Um, this is from the 1990s style. Um, it's very hard to read for those of us Please don't do that. That didn't work. Um, patch was created in a subdirectory way down the kernel tree That didn't work as well. You have to create it at the root doesn't work Um, this was funny. Uh, people are still using a really old kernel somebody It looked like it was and they're running it says root. Um, I don't know how they even created this patch, but they did And send it out. Um, please that didn't work. That's a kernel from about 10 years ago Um, made against a different tree. That's a very common thing You make it against a tree that I don't maintain and then I can't take your patches because they just all apply That just does not work at all. You have to at least send patches to the person who events their tree that they Can take it just is that simple in the kernel? We have some Tools that'll help you out with that and I'll talk a little bit more about that again minute Wrong coding style We have a coding style. You might not like our coding style, but the The main reason for having a coding style is to be consistent and you want a consistent coding style So that we all can read it easily and we can all modify your code. You want me to be able to read your code I want you to reveal read mine. This is the key. We have tools that catch this stuff Run them and use it. It's that simple Um, the rest was I somebody admitted they had a wrong coding style and they acknowledged it And they thought that their little tiny driver was more important than the 30 million lines of the rest of the kernel code That's not true. I just isn't going to work out. Please don't do that ever again Wouldn't compile I see this a lot people send me patches that if I try and build it, it just fails Um, that's really rude kind of shows that you didn't even test build this stuff alone test run it some Lot of driver changes stuff. You can't run because you don't have the hardware, but at least build the code Um, because if you break the build no patch can ever break the build and we don't like that it won't work um it broke the build on patch three out of six and Try to keep on going and then fix it in the last one. Well, that just doesn't work You have to every single patch that you submit to us has to stand alone on its own in order is fine But you can't break the build and then fix it up later We have tools that every single patch that's committed to the Linux kernel all 10 of those an hour They all build the whole thing nothing breaks. It all works. That's our requirement It's a good engineering process process and it works really well. So you can't do this Um, the best was you somebody broke the build on patch five of eight of a different patch series and then They said that they'll fix send me the patch later to fix it up. Uh, it doesn't matter. Nobody's going to use this That's not okay. You can't you cannot do that stuff at all Um, a lot of people send me patches. I've nothing to do with me. Um, that's nice. I can't really do anything with them But um, I appreciate it. I don't think it's spam, but I'm not going to be able to do much with them Send them to the right people. We have tools for this stuff. It works out Um, some people send one giant patch and that's really really hard for this isn't actually that big I've seen much bigger than this But reading 5,040 4,500 lines of code at once to try and see what's going on does not work The goal of the kernel is to um, follow your old math professors rules of show your work break things down one tiny logical step at a time Please do that. That's a requirement. Don't send me giant things all at once Sometimes you can send a whole driver and that's fine a whole file for a driver. That's one thing But in this case, it was not that it was just one giant change of all this different stuff. That's not okay We have kernel doc formats to do the documentation something somebody sent it was obviously wrong That was just really weird. Um, I understand people like using other document formats But we have one don't try and use java doc or anything else Use ours. Wait, it's document and if you run the build of testing it, it'll actually tell you errors and you have to fix it Um, and like I say, this was actually a column two weeks This was nothing really major going on. This is what me as a maintainer have to see all the time Um, so that gives you a sense of what being a reviewer of the kernel Gets and this is why we have a lot of stuff. This is why we have rules We have processes in place and all these processes are documented. We document them really really well These slides are going to be available online so you can see the link To these but these are in our kernel tree. Um, they're published on the web in our documentation We have a checklist go down the checklist say did I do this this this this this? Yes, great You can send it off We have submitting drivers and we have submitting patches. These are documents that you should read It explains everything that you want and if you do something wrong that's not in this list and these files That's fine. We'll tell you and if you do what's here, we're very very happy All those things I mentioned earlier this way earlier that happened wrong this past two weeks We're listed in here as things you should not do or things that how to do it right So we have this stuff documented. Please please follow this stuff So, um Any questions so far? I've been going pretty fast If our questions, um, should the bill be tested against all architectures before submitting the patch? Um, no building insurers. Um, usually it's pretty obvious if it will or will not break We have tools when you submit patches to the mailing list that will build on all architectures But I don't expect you to always get it right. I don't do it. I build it for x86 and arm and that's about it So I wouldn't worry too much about that Um, and yes, the links in the test cannot be clicked. Um, because the presentation has it So don't worry about that. Um, I have a link to the presentation in the beginning of this and it'll be on the website for this as well Um, any other questions can ask for life cycle the life cycle of what? I don't understand Colonel life cycle of kernel releases. Um, I have a whole presentation on how kernel releases are done What we do in that. Um, I'd recommend going seeing that. Um, it's on my website. It's on my github account kernel Development I gave one I give this talk about every couple months. Um, please go see that. It's it's a little bit outside this Scope of this talk. Okay. Um, thank you and anything else All right, I got more time. I got more stuff Um first time contributor also. Yes, we have lots of stuff places to start. I have a presentation on where to start Colonel newbies. Um, there's a whole directory in the kernel driver staging that has to do files Look at those files Read what's in there spin a patch She was written the whole presentation a whole course on how to submit your first kernel patch Go through that go through that. That's a really really good one. It's on the links foundation site. I'm somewhere Um, that's the best thing of how to get involved and how to make your first patch how to get accepted Works out really well. There's also a link on the kernel newbies site to Um, documentation on how to do it there as well. So I recommend those Okay, so like I said before, um It's in my self interest to ignore your patch because if I send if you send me stuff then I have to maintain it I have to accept it. I am now responsible for it. If you disappear. I have to um Take it. It's not it doesn't really work. So I have to be I have to know That you're gonna either get it right or be there to fix it when I get it wrong Um, so if I do accept your patch You have to give me no excuses. I have to be obviously foolish not to accept your patch So if you go through all the things and follow all the rules that we have it's foolish I mean not to accept your patch because you did everything right you fixed a problem You made something work for you that um, you need to have solved that you want to have working So I have to have no excuse to reject your patch. Give me no excuse and it's really easy not to do that um Just follow those rules and then when you do that you've given me code. That's properly formatted submit it in the correct way Build properly. Um, I have to take a patch. I I I have no excuses and I will Um, if I do take a patch or if I I do if you do send me stuff I will promise to do this for you because it's a two-way street as a maintainer I am responsible for things like this. I try and review patches within one to two weeks Not all maintainers do this. We have jobs. We have other things we have to do Um, some corporations most corporations do not allow their maintainers to do Maintainership work on company time, which is a nice so these people do it on their own time So realize that for the majority like huge huge majority of all maintainers. This is not their paid job This is what they do on their own because they they feel responsible for the kernel Their company almost always pays them to do new kernel work not to maintain old stuff It's slowly slowly changing But if you look in the maintainers file, you'll see a list of who says actually paid and who isn't paid to do that There's some huge companies that it lists their maintainers as not being paid to do this work But I try and do it within one to two weeks because I am kind of paid to do this for a links foundation Um, except during the merge window during the merge window You'll get a response from me saying I got away till the after the merge window is over But um, then I'll get to it and I catch up within three weeks It takes a little time um and submit something if I haven't um done anything within two weeks I have no problem with you email me and say hey What's up with this patch and it doesn't don't feel bad about this don't feel upset just do it I don't feel bad about it. It's like, oh, I'll feel I feel upset that I haven't reviewed that time. Please do that I also will if you send me patches offer semi constructive criticism I will not just reject things without telling you what I'll tell you what to do How to make this things better how to make it acceptable or I'll just take your patch But I will offer constructive criticism on that stuff I will also let you know the status of your patch because people want to know what's going on A lot of people run patchwork and their subsystems run on that USB is one that patchwork runs automatically So when I accept patches that automatically shows that the patch has been accepted and where it is Um, you also get email notifications from some maintainers. Um, andremorden first write did this I've copied his scripts a number of other maintainers to copy them So when your patch is accepted, you'll get an email saying where it is where you can find it what the next steps are going to be Once the maintainer accepts it, usually your hands off. You don't have to worry about it unless something really goes wrong Um, but I will let you know the status and you can always email me and I'll point you at where it is And you can follow along where it's going through the process through the public trees. You can see where it is um And throughout all of this, um The goal of us working in public is because it creates a better product Jason who did wire guard for the kernel said this to me in text many many years ago um The reason we do this work and the reason you you do this is um In public is so terrifying to submit things that is going to be reviewed by somebody else But it makes you do the best possible work that you can possibly do because your name is on it Your name is a public record now that you've submitted this stuff and you're not anonymous You're not hidden behind a corporate firewall. You're not hidden buried in some corporate database or some internal server This is your name in public doing this work and that's great And because of that we get higher quality work You do higher caliber work and it works. This is our development process This is the process that we do and because it's in public and because it's all out there for everybody to see It generates a better thing And that was it That was fast Any other questions? Yes, I do I see questions. Okay. Um, are there any device in qmu? This can serve standard by kernel musty tested. Um The best thing to do is test if it works on your machine if you're working for drivers that are not on your machine Then um, it's a little bit hard to write new code for that. Um, do a build that make sure that doesn't break the build all mod config Just make sure that everything works everything succeeds in building and you can boot your laptop You can boot your desktop and they can work on the hardware that you are used to if you break somebody else's hardware That's usually pretty rare. But you know, you're going to do that when you when you make those changes Um, accidentally breaking other people's stuff is not a problem accidents happen We take it and we fix it and we move on not a big deal. Don't worry about that That's not something to be too concerned about at least test build your code though. You should always test build your code Don't send stuff that you obviously haven't at least built it on your desktop or arm or something simple like that Um Not relevant to spinning patches kernel read write safely your plate. No, I'm not going to get into that Um, don't read or write kernel files files from within the kernel I'll say that right now. If you want to discuss that over email. I'll be glad to but never try and read or write a file from within the kernel Um, there's also the next week in news article talking about where that's been going and stuff like that. Um What about new tools for kernel development? I do not know about those tools. Um There's lots of people lots of developers like we had said we have 4,500 developers We all use different tools to create things. So it doesn't really matter what you use to create it We take patches through email the least common denominator. It works well for everywhere Email is actually really really good and people don't realize it. Um, because it's not Interrupt driven Gives you a chance to see something to mold over to translate it to your native language as someone who lives in a country Where I don't speak the native language. I very am very aware of it Nice to think slow things down and be able to translate things to a native language Be able to respond and then respond back Email works very very well for that. You don't have to instantly respond on irc. You don't have to instantly respond in Meeting it works really well. Email is a very very good tool for this text is very good for discussing technical issues And technical things and that's the key So there are other people trying to do other types of tools We all maintainers have tools that we use and scripts that we use and we copy them and modify them and share them around For how we manage patches. Please use whatever you want for that Um, if you send a patch for example to the bluetooth, should you use a bluetooth next tree? Yes, you should use the bluetooth next tree or linux next usually either one works out as well Um, they're usually not an overlap between the two if you're modifying something in bluetooth There shouldn't be another subsystem also modifying bluetooth that shows up in linux next if there is We'll catch it really easily, but just want to work on the tree that is for that subsystem So for usb. I have a usb tree bluetooth as a bluetooth tree work on those trees and submit to there very simple Um How and when people transmit from enroll the maintainer from being a developer? Um, sometimes it is as simple as oh you touched this file last. Do you wish to maintain it? As part of the mentor process I had some mentor or mentees this past summer One of them was modifying and fixing some security bugs in one of the subsystems Turned out that subsystem was unmaintained the maintainer of it did not want to do it anymore And they said hey, do you wish to maintain it and the mentee was like, okay, I will maintain it That's how things happen. You can maintain code if you write it from scratch If you send me a new driver now, you're the maintainer tiger it Um, if you maintain to maintain a subsystem, it's usually an organic process You're helping out with a subsystem the maintainer says I want to do something else. Hey, do you want to do this instead? And they say yes or no Some people do not want to be a maintainer because it does take a lot of non-development time I'm not writing a lot of new patches. David Miller is not writing a lot of new patches We are switched from a developer to an editor mode Some I have some developers and some of my subsystems that could be a great maintainer But they don't want to be so I'll do that work for them instead because I'll handle the patches They just want development dolp stuff. They want to fix bugs and push them out And I have to worry about the patch management stuff. It's a very organic process I wouldn't worry too much about it. Um, but again, if you want to write a whole new subsystem and submit it You will be the maintainer for that subsystem automatically by virtue of that happening Um Somebody wants a suggestion for writing script up pearl. I do not understand that. I'm sorry All the kernel is in C. There are some pearl scripts for check patch and things like that But um, if you will if you know pearl, please contribute to those. I think we had a maintainer process There was one project that was taking patches to check patch, which is our tool for managing Patches to see if they follow our rules Um patch accepted by the LTS and these were backporting who's responsible for the backporting the maintainers or the original patch author um That's a good question. So when you submit a patch and the developer says this should be backported to older kernels You can say how far back you wish for that to be backported to Um, me as a maintainer of the stable kernels and the LTS kernels the long term stable kernels Um, some of them go back many many years Sometimes patches do not apply if they apply easily and I can see that oh how to modify them and get them Apply I'll do that my work myself If not, and you said this needs to go to a really old kernel You'll get an email from my system that says please if you really want this to go to those old kernels Send me a patch and that kind of shows that if you do care about those older kernels and you do one of their You will submit the work. I do have there's a number of developers who do get those Acknowledgements and do do the backporting work for us. Um, you'll see them on a stable mailing list There's a number of really good developers that help out with that and I'm really happy to see that But it's not your responsibility as developer if you don't want to do that work It's part of the stable process I never want to add more work for you to do or more work for a maintainer then they want to do if it's important And it's a big security issue. Sometimes I'll do the work. Um, otherwise if nobody really wants to do the work It really probably wasn't that important. So it doesn't get backported. It's that simple um Simplest subsystem for initial contribution on the staging contribute the drive directory so drivers slash staging I have a whole talk about this stuff. Um, and again to do there's a big list of things to do in there Submit a coding style fix for in there and away you go That part of the kernel is there for new people to get involved That part of the kernel does not have all the coding style fixes done by us We could have done it instantly But we want you to get involved learn the process fix your email client submit it again Learn how things work and that's the best place to get involved Do not try and make coding style changes to parts of the kernel outside of there because that gets in the way of maintainers It gets some mix and bother gets things Includes on other development that's happening for other subsystems Just do it in the staging directory and then you can progress from there. That's the best place to get involved to start with Backport patches in the same cycle and or different. There is a different life cycle for stable kernels I have a talk different talk about that. Um stable patches all have to be merged into linus's tree before they can go into a stable tree Um, they happen semi automatically. I talk about how that does but um get things into linus's tree first That's the requirement. That's the rule. We cannot take things into a stable tree that are not in linus's tree first So that's the way that works Um Get involved. Um, again center set of optimization I do not Very rarely that while I reject a patch of it hasn't been optimized enough Very rarely in the kernel. Are there any areas that performance really really matters? Because most of the time we're dealing with hardware. We're dealing with long latencies. Um, it just doesn't matter Performance is if you can measure it wonderful. We'll take it. Um, first off you get it working first If it works, then we can optimize it because we can see where the hot spots are and go from there We're very good tools to do performance analysis on the kernel Um, they're built into the kernel. They work really well work on it them Um, if you slow things down, we will know every patch that's submitted to the kernel Run through a ton of benchmarks a ton of tests and we got automatic emails saying hey your patch sped up This benchmark over here by x percent or it's loaded down unless we get one of those We just take the patch. Don't worry too much about optimizations. Worry about getting it right first If I see something obviously Unoptimized, I'll tell you it's that simple adult. I would not worry too much about that Um, persistent test is a good way to persist changes with the test framework. We have unit tests We shua here on the call. Um, she is the maintainer of the kernel sub the test subsystem Okay, self-test we have tests two types of tests. We have tests where we modify or we Test the functionality of the kernel from outside the kernel from user space And those are in the kernel self-test directory And we have tests that we test from within the kernel and those use the k unit subsist or framework She was the maintainer of all that stuff. We have the ability to do both We have some we have the code in the kernel to do both Um, please write your tests for either one that makes sense Um, when we're adding a new system call, it's really good and it's not a hard requirement But it's a soft requirement for you to take Um, write a test for that. So like I wrote a new test a new sub a new system call called refile I wrote a whole bunch of tests for that and that's exercising that system call from user space if you write a new functionality for some Um, crypto algorithm, we have a whole crypto algorithm self-test subsystem in the kernel that tests it from within the kernel Those are there. So we have those look at what area the kernel you're working on They will usually have tests for them already go from that if not as the maintainer where they want the test to go We'll be glad to answer that How do we handle security patches? Um The there's a really good document on the kernel We have a security document that just says how we the kernel security team will If you notify us of a security problem, we'll fix it as soon as possible The kernel treats every patch or every bug reported to us as a bug a bug is a bug as a bug Whether it's security issue or not because of most of the time We don't know if it is security related or not when we fix it There's lots of instances of famous patches where I've fixed things in code that I've broken And it turns out three years later. Somebody said oh that would fix the security bug So we fix them we move push it out and we move on we do this every week We fix security bugs and we fix known security bugs and unknown security bugs every week The stable patches are taking about 30 to 35 patches a day Um, those are in the stable kernels. Those are patches that we know fix problems. Take this Um, when you submit patches to us that we know that they fix a bug We will tag those to go to stable kernels They'll get back boarded and they'll get in the pipeline in the way they go That's how we handle a security patch We have a document if you have to submit us stuff that you want us to have under embargo Talk to us the security team on the kernel But we really don't like embargoes at all longer than a week after we have a fix made Um, take sometimes it takes us longer than a week to create that fix But we don't like sitting on things for like very long for obvious reasons. Just doesn't work Um Julia says she can boot kernels Did not disappeared Julia's Sorry, I was answering that question typing answer Okay, sorry. I said that I usually report it to the mailing list any problem if I don't have time to debug and So it should be in the answered list. So you can you can oh fast Oh nice. Okay. Great Um Scroll down some more. Um, what happens if I receive multiple patches solving the same bug? Um, I will show we have multiple patches fixing the same coding style issue Um, I take the first one that was sent to me It's that it's simple Sometimes if you're fixing a problem and multiple people submit a patch for the same problem It's usually done a little different way. We might argue which one does it better or not This is actually really rare. It's very rare that this happens and usually it's first come first serve because that makes That's the only fair way to do that. Sometimes somebody like an obvious build warning Happens and everybody sends a patch the same day. I will combine them all and they're signed off and Tested by by all those people on the patch at the same time. This is rare. I wouldn't worry too much about it But try and fix it if you have the same if you find a bug fix it as soon as possible um Backording LTS. Yes. I said before it's very complicated and I will give up on it Even if it's a security patch, but I will ask you I will ask the author about it and we will work together on it Some if it is a security issue that I know about I will usually do the work for you Given there's I mean if you look at even spectrum meltdown, we have people that cared about those old kernels We did the work to get those merged Um, we will do that. Don't worry about that If you're an author of these patches and it is a security issue and I tell you I didn't backport it And it needs to be backported. Tell me and I will do that because I need to know that um I lost my list. Okay list. Um Uh, you lived a challenge. I do not know anything about the you lived a challenge. Um, so Email that site if you have questions about it. Sorry um Linux over quantum computers quantum computers has a different processing model. Um, a quantum computer is hooked up to Linux That's how what usually drives the quantum computer So think of a quantum computer as a driver for a type of device and Linux can control that very nicely If you see what they're doing with quantum computers today, that's how they run them Um If a contributor sends this patch for a specific board or chip will the maintainer test it on the same platform? Sometimes not. I don't care if you send me something that works for your platform I will assume that you tested it on that platform and if it doesn't break anybody else I will take it because I have to trust you if you sent something that obviously breaks it Then you're responsible and you have to fix it or I will revert it Um, I don't have all the hardware for all the devices everywhere So we have to just read the code and know it take it from there and it usually works Um, we do know this type of stuff and it can is pretty obvious of what is happening So don't worry too much about it. Um, we have to trust you if you get wrong You got something wrong in public, but that's fine. It was your device Um, do I think the tenure plus kernels support likes cip makes sense? No, I don't I can go into that if people want me to but I do not think that that makes very much sense And this is speaking of somebody who maintains a kernel that's six years old. I do not think that makes sense either People have different business models. So it makes sense to those people with those business models and that's wonderful I don't have to agree with it because I'm not doing the work for them Linux is used in lots and lots of different places I don't might not agree with all those places it's used But my goal is to give a tool that people can solve their problems with it If a 10 year old kernel is solving a real problem for somebody Wonderful, I'm happy for them. Best of luck. Um, anything else? I think I kind of burned through those I think I'm almost out of time Um Anything else? Okay going once Going twice here Well, thank you so much greg. Um, thanks for your time today and thank you to everyone who participated and and asked questions This recording will be on youtube leader today and we hope you join us for future mentorship sessions Thanks everybody Have a good day