 Hey, good morning That's a crowd and I think we have the two oldest people on the stage yet So it's it's good to be back in Prague. We were actually here I think six years ago and it's wonderful to be back. I Wished our trip had been slightly less eventful. We got our bags this morning Yeah, which wasn't exactly when we arrived, but hey, we got the bag. So that's that's definitely a pro So I'm Dirk Hondl. I'm the chief open source officer for VMware and you are again I am Linus and I Hate doing public speaking. So for the last probably 15 years by now This has been the only format I basically do because this way I don't have to prepare slides and I don't have to think about what the audience wants to hear About that's his job. He asks the questions and we try to make this interesting and I don't Know the questions beforehand. So we'll see what it be. So it's not been 15 years. It only feels that loneliness We started doing this six years ago. Actually, so for 14 RC six is out and You said in the announcement that you might have to do an RC eight So right I always ask myself is this good or is this bad or where does it come from? I have to say we have no real Rules about how many RC's we do it's just that over we So in the current community we use the same process now for over a decade Where for two weeks we have the merge window. I'm assuming most of you are familiar with this and then we have a number of weeks of weekly RC's and And I'm mostly really guided by how it feels having done this for a long time It's not like I Have a checklist of this needs to be done before I make a release. It's really a Looking at what the patterns of patches coming in is it does it feel like it's slowing down and And we usually end up with seven RC's and then then the final release So it takes us about two and a half months With the merge window This particular merge window was more painful than usual probably because Greg who's in the room somewhere Announced that it's going to be a long-term support thing And that always means that all the companies who care about the long-term support kernels they say hey We need to get our features in this release. So there's more hurrying going on Going on than they're normally is one of the nice things about our normal release schedule is We do releases every two and a half months So people don't feel pressure to hit a particular release if if the code is not ready. It's not ready It's fine. I'll wait until the next merge window except when long-term releases happen so we've the last one was four point nine and it was the largest kernel ever and current one is going to be for 14 and It's slightly more painful than some of the earlier kernels were but Right now it feels like I'll probably extend it by one release Candidate, but we'll see what happens. So you're talking about painful and obviously here in proc is the Linux kernel Maintain a summit. So which subsystems cause you the most trouble or if you want to be positive Which are the role models or both? I Am going to name names, but I'm going to name names in private tomorrow during the kernel summit I Guess I can speak about one of the shining stars Is it which is arm which I used to hate and now it's like one of the shining stars of good behavior And I never have any issues with them anymore the the the problems Problems what's tend to move around a bit although there are a few recurring ones that I will we will be discussing tomorrow. Oh So words will be spoken. That's interesting So one thing that I found interesting when when Jim introduced our first speakers was his focus In talking about the kernel report on how many first-time developers we have and and how this community Keeps getting younger yet looking at the two of us I've known you for more than 25 years and looking at the at the kernel community in general and Specifically at the maintainer community. I think we are seeing not a lot of influx of new Maintainers and part of that I think is the insane complexity of of what we do. So what do you think about? Do we do enough to get young? Younger blood into the maintainer community. I don't know. I mean that is probably one of our biggest issues is that it's Being a maintainer is kind of painful. It's not that it necessarily is hard Once you get used to it, but you do need to have a lot of experience you need to have experience for two reasons one is just In order to handle all the flow You just need to have done it for long enough that it's not overwhelming It's like learning a new language It's hard in the beginning and and if you get a thousand emails and you have to walk through each one and Decode it and think a lot about it. It's a lot of work compared to somebody who's been doing it for 15 years or 25 years and and by now I see patches In my sleep. I see a patch and I know what it does and I don't I don't spend any time thinking about it because it's all automatic So that's one reason Maintainerships tend to be with people who has been around for a long time. The other reason is purely external You need to not only have Drown the capacity to to handle the inflow of patches you need to also have shown to the community That you're reliable and you're around and you're around every day of the year or you are part of a group that that Is responsible and somebody's around pretty much every day of the year so getting new maintainers has Being one of our biggest problems we we have people who For one reason or another decide that maintainership is They've been doing it as their main job for a decade and they decide That they are moving on to doing something else having a life having a life or Maybe they're moving on to a different area because they moved companies or something like that and and then finding new people to to pick up the slack can be a big problem and It's not something that a new person can just come in and do But the good news is if you do want to be a maintainer trust me. We want maintainers. I love maintainers I it may not always appear that way in my emails don't get me wrong But but people it's a rough kind of love. It is a it is a tough love But but people who who do the work and are reliable are absolutely I mean the most important people in the community and and we are looking for people and If you can show that you are reliable and you are there and you are responsive You don't have to be perfect gonna get get me wrong Everybody makes mistakes everybody sends in bad stuff, but if if you then stand up as I said, sorry my bad And here's the fix that Makes people trust you and you do need to do it for a few years to show that you really are you Not only are trustworthy, but you have the capacity to stick to something But if you want to be a maintainer, it's not that hard in the end because we want you So over the last few years you have pushed many of the major subsystems to have maintainer groups So two three four people is that a way to mentor new maintainers to add them to an established group and have them Grow in that rotation. It probably could be we have done it mainly for different reason. We have done it because Being there every day every week is tough and and one single person will eventually burn out and people take vacations and I'm I'm in a good position where I can usually time my vacations Not always, but I try very hard to time my vacations so that they're in the end of the RC series when my workload is much lower because At that point, I know I don't have to do 20 pull requests a day I have two or three or Tuesdays, I usually have none for some reason. It's Fridays and Saturdays that I get most pull requests But but if you're a subsystem maintainer and you get these Occasional patches from driver developers or somebody else you need to be there day in and day out and having instead of having one person maintain a subsystem having three or four people maintain one has turned out to be a good way to Allow people to to not have to be there all the time But it probably would be a good way to bring new people in too. So maybe a Maintain a mentoring program is something you should discuss at the maintainer summit Probably I mean you don't come in as a maintainer from nowhere you you come in as a maintainer because you used to be a developer and Mostly the maintainership hierarchy it it sometimes looks like we actually design this thing and that is not the case We don't have a committee that says now you are the maintainer. It is very organic. It happens fairly automatically If you have been a developer and active for a while, it might be more of a taggy or it then Committee saying hey, we have a new maintainer. It's more like you've been doing the work Now I'll take all my patches from you So I'll try a fun question Who in the audience was born? 1991 or later? So all these people Linux has been around for all of their lives if you have to imagine that when when we started with Linux in 1991 to us Unix felt like this old OS that had been around forever and all these old gray beards were at the usenix conferences at that point Unix was About 20 years old Was started in 71 we were starting in 91 So we have now more history that Unix had back then does that frighten you? no, I actually When you put it that way it sounds odd It's It's not how I really think I have I have the memory of a goldfish I remember about five minutes back in time the 25 years ago is kind of hazy by the way. Hi. I'm Dirk. Yeah. Hi good He knows I'm particularly bad with faces and names So if you feel like I should know you and I did not say hi to you outside It's because I have no idea who you are On that happy note Let's let's switch to one of your favorite topics security. Oh, yeah So those of us who live in the US or who have any business dealings in the US Of course are super excited about aquifax my my favorite company right now um, if you if you see the impact that Lap system security have on all of our lives has that in any way changed your I don't want to say Attitude, but maybe that is the right term your approach to security and thinking about security people and and You know, this is a PG audience, so please Security people don't tend to be my favorite kind of people and that's don't get me wrong That's not because I don't like security. It's because a lot of the security Community is about this whole Look at me. I found a bug. I'm I want to spread the word about how great I am Because that's how they bring in business if you're in a security company being this kind of Self-serving person is often a way to be do marketing and and I realized that the good security people In the audience now should probably feel like that's not me But sadly the good security people are the ones that you don't see because they don't do this thing, right? So security people often annoy me That said I Mean there's no question. It's important, and I'm actually very happy. We we've had a lot of What I find very healthy the the random testing I'm a huge believer in fussy and and part of that is Because I'm a huge believer in testing but and not so much reading code I think it's way too easy to Miss very obvious things when you read code because the code looks obvious And there's an obvious bug there that you missed because it looks so obvious So testing is the only way to find most things But most security bugs and some end up being in in these code paths that no sane person ever tests and fussing tends to be a great way to try to Automate that insane behavior, and I think The kernel community has are particularly a few big companies You guys know who you are have been very active in creating fussing tools for areas that we didn't necessarily Do that well on before all the device Driver testing by fussing bogus devices has been really good. It's found a number of bugs Most of them those bugs are pretty hard to exploit It's often things where you have to have a specially crafted USB stick That lies to the operating system and if you happen to use that USB stick and and put it in your laptop It takes over your laptop if the if the kernel does something wrong But this is the kind of testing that that I think is actually very effective. So we're doing better I mean, we're I'm not claiming we're perfect in any in any way, but but I do think we're doing better I think we have made tremendous progress the last five six years And I also think that our attitude has changed Ours as the whole kernel community. There is a lot more Attention to this and a lot more a lot more willingness to listen to people who come with new ideas and to bring up concerns We also I'd like to call out not just a fussing. There's a lot of Tooling that I have to admit five six years ago. I was not that That optimistic about We've had tooling that helped us find things like locking bugs lock depth has been Incredibly powerful for us to do good locking and fine when we do locking wrong But now we're having a lot of these compiler tools that Automates other classes of bug finding and case and and and other things have been very powerful in in finding bugs that No human would ever find So let's switch gear a little bit and not talk about Linux for a moment Looking at community projects, and I truly mean community project. So not things like Android um Why are some areas in which open source communities pop up very very successful the kernel being of course the poster child and Others aren't what what makes it so unique that some Open source projects take over the world But the vast majority of them never really take off I Still actually think so I used to say that nobody would ever do an open source database because databases are horribly boring And I still think databases horribly boring, but I was clearly wrong because people do do open source databases But I I do think that the open source projects that tend to be most successful Are the ones that are good at finding commonalities a lot of it is about infrastructure Which turns out databases Actually fit in that thing, but a lot of it ends up. I think being about things where there's fairly Easy to find agreement. So a kernel is something everybody needs and there are Not that many arguments about what the needs are When you go over to some more user-visible things UIs and things like that suddenly people disagree a lot more About what's the right thing to do So I think one of the reasons that that technical particularly technical projects tend to flourish very very well in open source is that That when you have a very technical area, you also you can show real numbers You can say this is the way the right way and and it's much easier to find a Community and in agreement around that than there necessarily is in some other areas but You also need to have a big enough community. I mean we've seen that in dive log software It turns out dive log software is not a huge community. So it's hard to grow a big open source Projects around dive logging while everybody needs a kernel. Everybody needs source control Not everybody needs to log their scuba diving Sounds weird to me. Yeah But let's talk about this. What does make a project successful or actually how would you measure whether a project is successful? I actually don't like thinking in those terms because that's not why I do Any of my work, I mean who cares if something is successful or not. That's not the big issue the big issue is whether it's interesting and whether it does what you want it to do and In the end it is successful if it does something you wanted to do whether it has five users or 50 million users who cares in the end, right? But that then gets us straight to the question. What makes a good developer or what is it that that drives a good developer? I really don't think there's one answer The open source community when I started was much more homogeneous than it is today There were The geeky White male with often a beard right and and we all did things that no sane person should be interested in And I think that has changed. There's a lot more. I mean, it's not like it's diverse, but but You have people who come into the community because they're interested in Actual graphical design or documentation or all these things that did not exist when I started I think at least I never saw them And and I don't think there is any particular one thing I mean I can I can guess at what makes Somebody a good programmer if you started programming when you were 10 or 11, you're probably a good programmer, right? That's That's what it takes. It takes a decade or more of Doing something and that's true of pretty much everything. I don't think that's really Relimited programming, but I actually think from that you can derive a little bit of what I consider key criteria There needs to be something that you care about and that you're willing to invest in long time and 25 years of doing the kernel and Also, it has to be fun for you If you as a developer find developing software miserable, I don't think you will ever turn into a good developer I agree. I mean definitely I at the same time. I really don't think that has anything to do with Programming or doing projects. I mean if you want to be an athlete the same thing has to be true, right? I'm not one And I never will be because I don't find it find are interesting. I accept my Extra baggage But but it's true probably in marketing too. You probably need to actually enjoy talking to people Okay, so Speaking of developers There is a topic that I know you love to talk about and and so I'll ask my next question very specifically Explain your recent love for C++ because I got C++ patches from you just recently Oh, no, I Suspect If I start swearing on no don't Jim goes like this now no, I do it's all PG. It's all PG No, no, it's a lovely language That's let's leave it at that. I Did wonder how anybody could ever develop anything in in that when it takes so long to just build stuff I mean a small file takes a couple of seconds to build I would never be able to live that way Yeah, we were we were sitting in the hotel room and he made a change to a C++ file and rebuild subsurface and said my kernel compiles faster than this Okay, and a very related question Do you like providing end-user support because that you also do which I find fascinating given your otherwise public persona Oh No, I actually don't do end-user support if you're talking about in my family. They'd like to disagree with you I'm talking about you Responding to a lot of requests we get on the subsurface made in this for random users who have very simple problems And Lina's is our top supporter at this point. I know it's It turns out even though I'm not actually a people person I'll be the first to say that I don't recognize people and I don't like talking to you guys over email It's different. One of the reasons I got into Linux or why I continued getting into Linux Let's put it that way is I actually like Interacting with people over technical matters and I do not mind bug triage and things like that at all And it turns out in the kernel. I no longer ever do that. There is I mean by the time Something hits my inbox on kernel issues It's all already gone through bug triage through three levels of people and it's probably something really nasty And there's no way I want to interact with this IT person at a company who sees odd behavior. No, but but sometimes it's nice to have a small project not like a subsurface but also get was a when I started get it's Over 10 years ago now, but it was actually relaxing and interesting to do something small with Small problems, right? So I still enjoy that part in the on the kernel side. I don't do any programming. I I end up spending all my time just reading email and pulling patches from people and and honestly It wouldn't really make much sense for me to do anything else but but it It is kind of nice to also have a different project where you actually do look at code and work I'm no longer one of the main maintainers, but I I still am one of the people who knows how pretty much everything works But that brings me to my next question So why do you do what you do and ignore the fact that gym pays you? But what is it that makes you go back every morning and go through hundreds of emails and Dozens of pull requests and and keep doing this for 25 years. It is a very long time I Still like the technology I still actually like the people again Despite sometimes my negative emails being the ones that are showcased. I Really enjoy interacting with people I like doing something that is meaningful The kernel matters to a lot of people being in charge of a project that is Something that people care about is something that gives your life meaning, right? It's not it's not necessarily the thing that Gets me up in the morning, but I would be really really bored if I didn't do kernel development So I end up going away for a week at a time usually doing scuba diving, but I Do end up even when I'm on vacation I? Read email at least three times a day just because if I don't do that I just fall behind too much and then occasionally very occasionally I Tell Greg and a couple of other people that hey guys I'm actually going to be completely in communicado for a week or two and And that's an odd time for me I mean that's usually when I'm somewhere where there is no internet at all and after a week I'm so ready to get back because That's my life doing doing Linux in the end the hamster will keep spinning So I know how much you like to do predictions So I have to have a prediction question and I try to come up with one that I haven't asked you before and given that We have done this a few times. It's becoming harder. So what is the most surprising thing that will happen in the next in the next year? That is not how I really work To me every day is a new day and maybe it's the incipient Alzheimer's but more of it is that I don't really plan ahead I Kind of know what's coming in a very big view or just because I do end up talking to both hardware companies and obviously to a developer developers who plan what and talk about what they're working on but at the same time Most of my life ends up being reacting not so much Predicting what's coming up So I'm very comfortable in having the the flow that we have and not trying to To make the next big thing happen but to make sure that we're doing the best we can every day and And I don't know what what we will be doing in one year But I don't think we're changing that much fundamentally anymore. I have been surprised before though So then I'll ask you about something that you actually fully control and most likely can mathematically project So when are we going to get Linux 5.0? Oh Well, I have I Have been known to start losing track of numbers when they Go into the 20s and 30s fingers and toes. Yeah. No, I yeah When I had to take off my shoes to count kernel releases No, it's it's psychological for me It turns out 13 and 14. They're easy to remember but when you're looking back and you can't recall the difference within 23 and 24 or the The numbers smaller numbers stand out more. So I suspect that will hit 4.19 and and instead of 4.20 I might decide it's time to 5 5.0 The numbers don't mean anything at all. They have not meant anything in a long time It was actually very stressful when back when they did It was a horrible Model for development. I remember I mean even back in the 1.0 days just trying to like draw the line in the water in the sand of This is 1.0 and this is when the our internet code is actually working Was really tough and very stressful and then you Never end up doing the right decision anyway, so Jet is telling that entirely and saying no the numbers don't mean anything They do keep Incrementing but we just try to make them sound like easier to remember and then any other kind of meaningful Has made my life much easier and I think pretty much everybody enjoys it now But then there are still the few people when I make 5.0 and they expect lots of new features Disappointed, yes, so if I get my math right that means next summer ish probably I also did some numerology and decided that every two million get objects is When when I need to do a new major release But since our development is accelerating That actually might be sooner than next summer, so it really depends on I It's a random number and I want people to be aware that it's a random number and has no meaning Well, then you could jump to 6.0 and just confuse everyone. Oh, that would that is actually a great idea And on that our time is up. Thank you very much for your attention. Thank you Well done