 Thanks everybody for joining me today. I'm going to talk about which balancing product needs with open source objectives. A lot of us find ourselves balancing product needs with open source project objectives to be challenging. And if you do like me, if you do, you're not alone. A lot of us at times when we are working for companies that we have a product time to market needs and then we are looking to see we also want to contribute and make our product part of the any open source project. I'm going to be speaking with the Linux kernel because that's the open source project I'm most familiar with. And then I'm going to share my experiences having to balance product needs in my previous roles at a couple of other companies and with the Linux kernel project objectives. Now we're getting open source and corporate cultural differences and then objectives and goals as well. These roles have different goals and different objectives. That's what makes it challenging in a lot of different ways. It's a delicate balance very often. It is easier to work exclusively on either side of this divide. Like I do currently, I don't work for an organization that has a product. So my focus currently can be easier in some ways being just upstream focus. And then fast when I worked for companies that have products based on Linux kernel. At that point I had to learn to balance balance the needs of the product that I am working on, and as well as what would be beneficial to the company. So that's kind of what we are going to talk about a lot of the lot of what I'm going to share is coming from my experience doing this. I require a lot of delicate finesse and skill to know which hat you want to wear when you are you representing at any point in time. Are you representing the product teams, or are you representing the upstream. And also keeping in mind that you are looking to be productive in your both roles and bring value to both sides. So today, I'm going to walk through upstream mindset and the importance to do your homework before you go to upstream with an idea. And also balancing. What does it really mean to do the balancing act, and then how do you, the best practices and how can you bridge between two worlds. Some of you might know upstream releases are time based, not feature based. So that I find is the big difference between the two worlds that causes a lot of the challenges or leads to a lot of challenges. When I say upstream is releases are time based and not feature based meaning what do I mean by that. So Linux kernel releases come out once every eight to 10 weeks and each release has a two week merge window for new features. The Linux kernel development is a continuous development model where development is happening while integration is happening, meaning developers are getting ready for the next release, while we are integrating the features that came into them through the two weeks merge window, right, the two weeks merge window. There's a six to eight weeks of fixing bugs and regressions and integrating all these new features with existing kernel base code base. So all of while all of that is happening, people are planning for future releases. So that is a continuous development model. Like maybe in a product world that you might go through a period of design and development and then and then you add those features and then integrate. So, while a separate group of people might be working on older releases. And then at the same and also, when in a product world when you are thinking about next release you are deciding the features first we are saying okay. Our next release will have this this feature this feature a feature be featured see and you're working towards that, whereas upstream releases are time based if a feature is not ready to make the cut for the merge window, then that feature will be will be in the next release which will come three months from now. So that's the big difference. So another thing to keep in mind is all fixes must go into mainline first right. So we'll talk a little bit about the stable releases and on the next slide. So you can find a lot of the information I'm about working with the kernel development community in the kernel documentation. I'm leaving you with a link here to read. So the development process that we use in upstream kind of also leads to the mindset of upstream developers. We are more focused on whether a feature is not ready or not. The time for merge window calls. So we know that if the feature is not ready, it can go into the next release, and we'll have more time to to make changes to it. So that's the mindset we are working with. And we're also working with the mindset that fixes going into the main line. Get applied to stable release. So we are thinking about, well, if we fix, we have a fix do we can we fix is okay fixes can go in at any time into the mainline right, but they have to go into the mainline first. And then some of the we select fixes for selection of fixes for stables is automated, automated, and then they get selected automatically. We attempt to apply these fixes automatically first. And if the auto apply fails, porting is requested. And at times though, the auto automated tools do find a good chunk of fixes. However, manual selection is also necessary to have the fixes going to status and can be marked for fixes can also be marked for stable releases with the stable tax. You can see all about everything you want to know about in this document document on the in the kernel get everything you ever wanted to know about less stable releases. You can check that and see how that works. So continuing on the upstream mindset. We some stable releases are all tagged for long term as well. So long term updates. stable releases get fixes and security updates and so on. Still keeping in mind all the fixes first go into the mainline, which is when I say mainline. This is the Linux's master get upstream. The point to keep in mind is, if you are a product, if you are trying to balance product needs with upstream needs, you want to keep in mind that upstream developers community might not know about your product needs and might not care about your product needs. Your deadlines or time to market needs or features even upstream cares about features that add value. And they upstream community will be reluctant to take features that aren't ready, meaning you might have a deadline saying I need to get this in by this June 2021 for example, but however, if it's not ready, it won't make it into, if the upstream community doesn't think it's ready, it won't make it into the release. So we have to keep that in mind definitely as you are doing this balancing act of upstream and product. So a few things at this, any questions. Do we have any questions. I don't see any so far but just a reminder you can add them here in the Q&A box at the bottom of your screen or feel free to raise your hand and we can unmute you if you'd like to ask a question live. Great. So what does it mean when you are thinking about product and you're thinking okay I have this design. I want to have these features in my product by this time. So do your homework. That's really important. Research related upstream features. So this, all of the things that I'm talking about aren't necessarily applicable, just don't just apply to product development balancing. It's these are true for anybody that's wanting to contribute. It's just that if you are also responsible for a product that adds more challenges. So anytime you have an idea, research related upstream features, and that is a very important first step, you might find that there is a feature that you can leverage, or there is the end and also enhance that existing feature as opposed to writing and reinventing the wheel, right. Don't want to reinvent the wheel. Leveraging and enhancing existing features benefits everybody that is dependent on that feature or using that feature currently. When a new feature, when you think of an idea for a new feature, make sure that that is works well within the existing framework as well. It is important to design new features to work with the existing framework. For example, you might have you might be adding a different feature potentially, then you will have to work within the framework to see how best you can make that architecture dependent feature fit into the rest of the kernel infrastructure, because the kernel kernel supports 33 plus architectures. So the kernel has generic areas that it is very well partitioned in terms of architecture dependent code and architecture independent code. And when you're adding a new feature that might have might support a new hardware, you'll have to think about how well it works on all architectures. Especially drivers, does the driver is, does it have architectural specific components, or a subsystem might architecture independent. You have to kind of look at that and then see how well it fits into the kernel existing kernel infrastructure and framework. So more on doing your homework. Ask yourself very often, why are you making this change. And this is important to even just explain to the upstream community, what you're doing. And this is important. Also because of the remote nature of the development. You have to explain that in your patch documentation that goes along with the patch. And when you are explaining what a single patch does what and then also if it involves more than one patch what the patch series does if you are proposing changes to the existing feature. You have to make sure that you are not in any way, regressing that feature. If you are proposing a new feature, like I said, earlier, you have to think about how it fits into the rest of the kernel existing features and how it interacts with other features. The challenge. If you are an independent contributor are focusing just on upstream, you don't as much need to worry about time constraints right. So you are kind of, you are focused on. Okay, I want to make this right. I want to contribute something that everybody agrees upon and then, and you have the luxury to take your time. However, when you are working, working to get a release out product release out. You, you have other time constraints. If you are planning, you have to think about it and keep the releases kernel releases in mind, be in sync with those to see how best you can plan. And if you are, you also have to plan based on all the questions you already asked yourself. If you are proposing a changes to an existing feature that might be easier time wise, it might take less time for that to be accepted by that stream community. If it's a new feature it might take longer. If it is a new driver that is independent of. If you have interactions or impact on other modules. It might be easier again. So you have that all requires planning. Think about that and then plan. Also keeping keeping in mind at the same time that benefits of upstreaming your code comes in later in the form of collaboration, you. You are recruiting the rest of the community to help with making enhancements and then also testing, you're leveraging a lot of the extended talent out there. So you have to keep that in mind as well. What, what's in it for you from that angle. So when, when you are thinking about contributing also keeping your extreme upstream hat on, you look at is the solution and feature generic enough for upstream. How does your work help upstream? Does it add new functionality? Does it improve performance or security? Does it harden the code base? And, and so on. Is there a bad, is there a important question to ask is, is there a better way to do what you are setting up today? Would you take this solution? Another important question to ask is, would you take the solution as an upstream developer if somebody else proposes it? Any questions? This is another time I can take some questions. No questions. Yes. Okay. So, what does it mean. So when you have your product hat on. There's some one of the reasons you would reach out to make changes to the upstream set up to make changes to the upstream. You would make changes because upstream doesn't work on your platform. Hardware firmware deviates from a standard say you have to do things a little bit differently. You're, I mean your product hardware. New feature or enhancements on an existing one anchors your product. That means it's very important to your product. So, another, with all of this keeping all of these in mind, can you also decide also have to think about, can you talk about your hardware firmware differences? Can you share? What can you share freely what you cannot share? Can I share or discuss? It's also, it's very important to, from both sides, you don't want to share something that you don't want to share. You want to, it is important also to share so that your solution is accepted. You have to walk the fine line of sharing what's necessary and and not giving the story away. You have to find a way to discuss the details in a generic way. So now comes. So, and all of this brings us to the balancing act. So, a couple of things to keep in mind that upstream. Don't if you expect upstream to take your solution as is, then you are setting yourself up for disappointment. It's not a guarantee at all. It's very often even a small patch would require even when you send a small patch in, expect that somebody might come up with a different way of doing things or point out something that you haven't thought about. So keep that in mind. For proposing a enhancement to an existing feature or a new feature. It is a good idea to discuss the, what you're planning to do at with upstream at conferences. That's probably one of the proposal conference talk and talk about it and see what how well upstream reacts to it. So that's that gives you a good indicator on what you need to change upfront or what you would be doing. So whether your solution will work as these are the refinements you have to make or changes you have to make. You might get a feel for that by discussing early on. Another thing to keep in mind is collaborating and sharing early sharing collaborating and sharing early at conferences and sharing ideas and discussing. It will is is going to make it easier for your feature to be the time for accepting time can be shoulder depending on depending on the refinements you need to make right. So it's always beneficial to collaborate start collaboration and sharing early. Another thing that would benefit the product is adapting it might or might not be possible. However, it would be beneficial to adapt upstream first rule, meaning upstreaming. Having the feature upstream first and then getting it into the product. That will that will be beneficial long run. So you don't have to to go out with a different solution than the one that upstream accepts and then you have, you might end up changing two different core basis or you might end up changing the product after its release to update it with the version that upstream has accepted. So it might be difficult to do later on. It is also important to keep upstream variations to a small percentage. You, this could happen in as you might even if you started upstream first, as you are fixing bugs, or as you are finding problems that needed to be fixed. If you don't keep up with the upstream, it might become difficult to keep them in sync. So you have to this, you have to first decide how often to merge upstream because merging upstream. You have to validate your product, of course, right, because upstream brings changes. It changes that could impact your product support. So you have to to figure out or decide what's the optimal merge points are for your upstream and frequent is better. So that's the best solution to do because, because upstream yours, you might see some regressions but you're also seeing improvements right continuous security updates continuous fixes. So upstream, keeping up with upstream and merging frequently is necessary to further health of the product. So you have to keep track of with that in mind you have to keep track of stable and long term releases to to to two different things. So many products release many products choose a long term release as their base for release, and then they keep taking on security updates as new stables come in fixes. They keep moving to the recent table as latest table release for that long term release. So you, this is all again, you have to kind of decide what happens if you don't hit that target, what happens contingency plan, if it doesn't, and then if the contingency plan is release and then you're out of upstream then you're balancing the act of what if you're the your feature doesn't make it as you expected in the same shape or form into upstream so you might have to make changes to product, as well as upstream. Let's talk a little bit about bridging between the two words, explaining. So you as a upstream developer that's also focused on product development. If you are straddling those two words. Then you are doing two things. One is you are explaining upstream to product teams, and you are telling them the benefits and importance of offering first and paying it forward for benefits and of collaboration with upstream and keeping the delta, the importance of keeping the delta from upstream small. You're also asking them to plan. You are talking to them about planning ahead, and then also bringing that into your product teams that knowledge into your product teams, and then explaining your feature or change to the upstream community. You are talking to them upstream community working with them to to meet your product needs. You are making you are working to get you get the features that support your product by the upstream community you want that content in the upstream So you are continuously talking communicating what's in it for you message to both sides. So let's let's talk about we have talked a lot about balancing both worlds and then challenges. Now let's talk about a few things to keep in mind. These would be helpful. Not just for people that are balancing product and then needs and upstream community needs are straddling that fence of upstream and product. This is beneficial to everybody that is contributing to to upstream open source projects. So be open to changing your solution or your idea. When you propose a your idea or a solution assume that will change and evolve. Don't be locked into your I first idea, you know, it will change. Be mindful of adding value to both worlds. You know, you want to get your product out you want to get, or you want to see a feature that you are using maybe perhaps for your in your, even if it's not a product you're just using that feature. You want to contribute think about adding, helping yourself and helping others right so. So that will go along with before getting your work accepted into the by the community. Be open to be a student and teacher at the same time, because there is a lot to learn from the community that and then you that this is a this is a collaborative community with the trust trust and respect. So leveraging this collective expertise of the community is going to be beneficial for the project Linux kernel as well as the rest of the community as well. So also adopting the attitude that I don't know everything is going to help in the long run to to be able to contribute to the computer and work with the community. So finally, I'm going to leave you with a few thoughts. Learn as you go. I find myself learning every day and keep learning. And there is really no textbook that teaches these skills, just have to, to make mistakes, learn from your mistakes, and then learn the balancing act. It's a perfect solution for every product and every scenario. Keeping the some of these upstream mindset in mind will help make the balancing act easier. Do you not but just again as a reminder to anyone who's joined us recently you are able to either add a question into the Q&A box at the bottom of your screen, or you can raise your hand and I can alert Shua that you have a question or you can also just unmute yourself at any time that feel free to ask a question or provide any comments, welcome your participation. So you are welcome to share your thoughts as well. If you don't have any questions. If not, we can wrap this up. We did get a comment Daniel said very thought provoking thank you. Just want to give it a few more moments. Alright, looks like we have a question from the comment. If a Linux kernel maintainer is employed by a company now how does the community keep him in check that he isn't doing anything in a hurry because of his company's pressure. That is a very good question. So, community doesn't have to necessarily keep a maintainer in check. It is all trust and respect right maintainer as a maintainer, even if you are not a maintainer. If you're an active contributor. If you're constantly balancing is it is very important for maintainers to not lose their credibility. If a maintainer doesn't do that balancing act of keeping it to both worlds, thinking about both upstream value to upstream and value to product. And if they are only thinking about the product. They end up losing credibility in the community. So they become not very effective. So that in itself is acts as a check. And it will help, but I mean, obviously there are ways you can if they if there is a you see a pattern if you see a pattern that a maintainer might be abusing trust the community and other developers are placing, then there are, there will be consequences of course right not not consequence in terms of in terms of consequences will be in terms of losing credibility so none of us really want to do that we want to be part of the community. So does that answer your question. We'll give him a few moments to respond if you'd like but we did get two more questions in. Okay. Oh Mohammed said yes it does thank you. And so another question we got from Armstrong said thanks for the talk. What happens to changes like bug fixes new features etc, that are not integrated into future releases. Haven't been accepted into the by the maintainers. Is that the question, because most fixes. Well, I mean, fixes do get accepted but are you talking about the ones that don't get a few moments to respond. Yes, those that are accepted, he said. So, so I will answer the question in two different ways. So, say, if you were a fixed came in and that isn't accepted. What about the problem being fixed in a different way that's one answer to that. If it's still an outstanding problem, then the submit submit or pass submit or can recent the patch and say hey this is still a problem. And then, if it is a stable release, and there is a fix upstream that hasn't been selected by auto fixes auto tools that select fixes, or any manual method methods, then that fix can be sent to upstream, sent to stable. So, I think a mainline commit ID in the stable, the patch commit log saying, Hey, this fixes already in upstream. It's not in this stable release, you can do that. The third possibility, possibly is that your product has, you have been fixing bugs in the product. That's where the keeping the delta small comes in. So if you do not upstream the bugs that you're finding in your product, or as your feature that you already upstreamed, then you are accumulating a lot of debt. So you want to keep that delta small. So that is three different aspects of the question you asked. If that's not what you have in mind, please ask another question in the follow up question. Great, we have one more question from Tito. They asked, what advice would you have for students interested in becoming a Colonel developer maintainer. They, we have, we have a lot has mentorship programs will, this is probably a good time to talk about that. So we have mentorship program that's designed to help new developers for this with necessary skills and resources to experiment learn and contribute effectively to open source communities. So there are other resources available, like the webinars we are doing, of course, and then you can join the Cardinal newbies and check Cardinal newbies website as well as IRC. So what I would say is that the best way to get started is you have multiple ways to get started. You can at the beginning, if you if you have never used Linux Cardinal, and if you haven't really played with it on your hardware. The first thing to do is go and start taking beginner classes available. Elef training has a class. I did a class for a couple of years ago, primarily to help mentees from in the mentorship program to get started as as a screening and also helping developers that want to be in a unstructured learning self study. It's Linux beginners guide for Linux Cardinal development, you can do that. You can start that offers a lot of tips, and then test stable releases, and then other Cardinal releases as they're coming in. That is one of the best ways to to learn and then also find problems so you can either report the problems you can take the time to fix the bugs. And another option is run tools, automated tools, come, um, compiling Cardinals with warnings turned out turned up and then you might find problems that way. So testing gives you avenues for are playing with Cardinals like I call it call playing with Cardinals gives you opportunities to find and fix problems as well. We don't have any other further questions. We want to just give it a few more moments in case anyone has anything else. Would you like to go ahead and show or we would you like to say on for Well, we can wrap up. Yeah. Okay, well, thank you for your time today and thank you to all of the participants who joined us. As a reminder this recording will be on the Linux Foundation YouTube page later today and we will also be sharing the slides on our website. We hope you're able to join us for future mentorship sessions. Thank you and have a wonderful day. Thank you.