 Hello everyone and welcome to navigating open source project hurdles to achieve community empowerment or how the heck do you get through graduation. Just as a quick poll of the audience, how many are potentially considering submitting a project to the CNCF? Okay. How many are associated with a project in Sandbox? Okay. How about incubating? Okay. No one at that tier? Graduated? I know you are. Okay. So with that, my name is Bob Killen. I am known as Mr. Bobby Tables across all the things. I am a program manager for Google's open source programs office. I am also a member of the Kubernetes Steering Committee and chair of CIG Contributor Experience. And I'm Ijomal Nurmamat Kuzi. I am a director of DevRel and company called Cyzdik. We also open source donated a project to CNCF called Falco that provides runtime security for Kubernetes containers cloud. And previously I was with Bob Killen in Google's OSPO where we work with many different projects both CNCF and not CNCF, including projects like Knative and Istio that we help to open source and donate to CNCF. But what people don't know about us is that Bob and I, we started a project called Alpaca and it's totally a real project. It's a software packaging with middle-out compression. So very real. And we think it adds a lot of value to the cloud native ecosystem. And we're wondering, oh, why don't we donate it to CNCF? So it's quite safe to say that CNCF is one of the most well-known foundations and it's got a lot of support. It's like it has really wide ecosystem with many vendors, many players, many developers and projects within it. And there's a lot of interest and momentum for cloud native software. But CNCF is also very, very big. It can feel like it's very overwhelming landscape and it often feels so. So how do we know that CNCF is the right place for our little Alpaca? So we ask ourselves a question. Is my project right for CNCF? CNCF says that the project is a good fit if it first is the mission of CNCF, which is to make the cloud native computing ubiquitous. Some examples they give is software that extends or integrates with an existing CNCF project like Kubernetes or software that fulfills the current need like wasn't. Alpaca lets us package and compress software to ship across the cloud. So we think it's a good fit. By the way, it's totally real. However, that decision might be different depending on your project. There are other foundations out there with different cultures, with different focus. And sometimes when you apply to CNCF sunbooks, the technical oversight committee, they may recommend you to join other foundation that might be a better fit for your project. And examples of those foundations is like other foundations under a left umbrella like AI and data. There's also Apache Software Foundation. There's Open Infra Foundation and they have all different goals, purposes and different culture and focus. So when you plan to open source your project, it's just choose whatever is close for your project and close to your heart. And if you decide to join CNCF, on a very high level, you get a few really important things that help you grow your project. First thing is visibility. CNCF is a large organization and projects get credibility by being part of it. They also receive marketing benefits from the foundation that help you achieve the visibility and we'll dive into those a bit more later. Being part of CNCF also helps you to be aligned with the cloud native ecosystem. When you join CNCF, you join a large community of cloud native projects and their maintainers. By being within it, you can work across CNCF and meet stakeholders from different projects and that help you align your efforts with efforts of the rest of the players in the ecosystem. And CNCF also has a proven legal framework and it's one of the bigger benefits of joining any foundation. They have lawyers that have built the right licensing and ownership frameworks for the project so you don't really have to worry about that. As an example, you isolate it from any legal considerations, from accepting contributions from different individuals or organizations and usually that's a headache like to set it up properly. But perhaps the most important benefit that you get is a vendor neutral home for your project and this is very important like for multiple reasons. This gives trust and peace of mind for users that choose to adopt your project and when your users are people and companies and by seeing that your project is part of the CNCF, they will be sure that it will always stay open. The licenses won't change in the future and their governance framework also ensures that there's not a single vendor that drives all the decisions for the project. Okay, so I love the idea of being a CNCF project and your project may too and within the CNCF there are certain project maturity levels that roughly equate with the whole innovators, early adopters and early majority tiers of the crossing the chasm diagram. We're all here at the CNCF, I'm sure everyone knows a good amount of this and don't have to dive into a lot of detail, but the tiers for us are sandbox incubating and graduated. So our wee little alpaca has decided yes, it wants to join the CNCF and it's still a very young project so it's going to aim for sandbox. So with that, we let the journey begin and start to look at what it actually takes to become a sandbox project. The requirements for sandbox are not very strict and that's by design. The lack of strictness is to ensure the projects have the best chance for success. But before even joining the sandbox, there are few important considerations that anyone should understand. And specifically, you need to understand what happens to your intellectual property and what you will need to end up like transferring to CNCF. So you'll transfer a trademark, your logo, your domain name if you have and it is a legal process. So if you're a company, please check with your legal department before you go ahead with anything. And all of your code needs to be licensed with Apache 2 license and there might be other licenses that they approved by CNCF but Apache 2 is what is recommended. And anyway, you should adopt something very permissive. The transfer of the trademark to CNCF also doesn't mean that you can't use it anymore. There are clear policies around how to use the trademark in a commercial product and from our time, when we're donating K-native to CNCF, the big question for the company was, oh, can we use K-native in the name of our Google branded service? And the answer is yes, there are clear policies that you can follow and so can do other vendors if they launch similar products. So knowing this, if you still want to move forward, there are some requirements to join the sandbox and we'll categorize them into streaming buckets. First is governance. Like we said, there are not a lot of requirements. At this stage, you're just asked to adopt the CNCF code of conduct and have simple discoverable governance. It's worth mentioning that as you grow, these requirements will change and CNCF mapped out the whole journey and things will change over time. There are a few things that are not hard requirements at this stage but they're really nice and useful to have even at this stage. And that is documentation and contributions and how to build your project, how to test and what you expect from your contributors and also ensure that your project metadata and resources all vendor neutral. So when you usually start your project on your laptop or your company, it's all testing, infra is managed by you and a good goal is to have that infrasherable so others can join you in building your project once you are in the sandbox. And we've seen over and over many projects struggle with just vendor attach. It can take a long time to move stuff out of vendor controlled CI into community owned CI. It's also good to have a discoverable communication channel. A lot of CNCF projects use Slack and you can use CNCF or Kubernetes Slack for your project. You just start a channel there and you invite people to join there. And probably at this stage for your project, more than governance, it's important to have a clear proposition for your project. What it is, what is it exactly? What does it do? Is it real like our Alpaca project? How does it fulfill the need in the cloud native ecosystem? And usually it will be a read me file in your repo or on your website like landing page if you have a website. And it's not the requirement but it's always a great idea to have cool demos for your project. Can you show in a concise manner why your project is awesome? And that's usually what gets people convinced. And good guides for new users to pick up and use your project are also useful. Okay. So now we get to dive a little bit into the security requirements. Again, for joining the CNCF, it's pretty minimal. The biggest thing is that you have some access controls. So things like 2FA, if you have org members, they don't all have admin access. And while this is the only requirement for Sandbox, there are a few things that I highly recommend implementing early and they'll help you down the line. Namely, having a process for handling vulnerability reports or other security issues. This can be as simple as creating a security mailing list that a few of your maintainers are on. And having a process for this is required for incubating. So getting it going early just sort of takes care of it ahead of time later. Another useful thing that is not strictly required at any level would be to actually set up some sort of group secret manager, like one password or a key base. You seriously would not believe the amount of secrets you can accumulate quickly. Zoom, CI, Google workspace, it's a lot easier to have those in a secret manager from the get-go instead of trying to deal with it later. Like, trust me, it sucks trying to get creds out of a vendor account than the person left the vendor. So, Alpaca has done their homework. They've created some basic governance. They've checked their licenses, created a lightweight vulnerability triage process, and they think they're ready to apply. So, yes, it applies for Sandbox. It's pretty straightforward. There's an issue template in the C&C of Sandbox repo with all the questions that you need. The big thing I want to call out is that there are a lot of people entering Sandbox, and they essentially go into a queue to be reviewed by the TOC, and they batch review them roughly every two months. They will keep your issue updated with various labels that will indicate its state in the entire process. I recommend checking out the README in the C&C of Sandbox repo, and that will cover it all. So, Alpaca got pretty lucky. There wasn't a lot of projects in the queue. About two months later, congrats. It's now a Sandbox project. It's in the C&CF. It's excited. They aren't even going to bother looking at the incubating stuff. They're going to look at graduated projects for inspiration for their governance, their policies, and they're going to talk to some of the maintainers there to get some good advice. So, don't just copy Kubernetes. Now, a lot of new projects look to it as the golden standard, and that's all well and good, but they think they need to implement its structure, governance, and policies word for word. And Kubernetes has 10 years of history to get where it is, and it's a very big and large project. Now, that's not to say there aren't pieces that are great for smaller projects, and I'm sorry, I'm getting a phone call apparently. So, it's great to be inspired by Kubernetes governance and Kubernetes policies, but I really don't recommend just copying them at this time because you're going to have a bad time. Alpaca has heated the advice of other maintainers and scoped things down to just focus on the right priorities for this stage. And at this stage, we're still trying to find the market fit for Alpaca and build our identity, and we focused on developing solid use cases. What are big problems or problem categories that we're trying to address, and we document them and explain them very well, and we advocate for a project. Like, we join all the meetup groups, we write blogs, we present our project to different, like, audiences, and, like, we talk to different groups in CNCF and get feedback continuously and try to improve. We also need to love and nurture our first users, and luckily, we have those because we got the really cute mascot, and it really helps with acquiring new users. And users will always be our priority, right? But in early stages, you will know them well, and it's a really great opportunity for you to work closely with them, understand their pain points, and, yeah, like, get really high-quality feedback. And we use that feedback to build what users need, hopefully. And also, in early stages, we build features and try to build them fast. So it's good to stay lean and keep simple development processes. But also, it's very important to have a good, reliable testing. And this will help you to build new features fast, but also make sure that the core of the project will always work as intended. And Sunbox is a safe environment to experiment, and you really need to experiment a lot. And not everything will be a hit. You might even end up changing your vision for your project, and that's totally fine. Or you might find new interesting use cases that can further solidify your belief in your project. And to help you grow, CNCF offers a few services. By the way, we know that Costco is not Alpaca. It's Lama, but it's pretty close, so we used him here. But here's the list of the services that CNCF provides for Sunbox projects, starting with CICD, with some limitations, of course, ending with supporting, like, you in events, supporting you with documentation and design, like website development, so on and so forth. So you can check that link out for details. So with your great vision and your friends and CNCF support, things are going really great. And maybe a few hundred PRs later, you look around and think, what is next, right? And the next step for a project with CNCF is to join the incubator. And you're still in the Sunbox, and you have been working hard on your project priorities, which we mentioned earlier. But now we, to get to the next level, we need to do a bit more. Specifically, you, like, you have built identity for your project, and now you want to solidify it by documenting real-life production case studies. In fact, it's a requirement from the CNCF to go to the next level, and they want to see evidence of production usage of your project. It's always a good idea to add a file where adapters of your project can add themselves easily. This is usually done with an adapter's MD file in your repo. And I think it's better to go beyond company names and logos and actually interview them and document them into detailed case studies. And your readers will appreciate technical details, and so go deep. Don't be afraid to end up with five pages of case study, right? And these case studies will be your amazing marketing material, and they will help you to get more users who relate to your use case. And your users will always be your best advocate, so try to get as many of case studies as possible. We talked about users earlier. Well, as your project grows, some of your users will become your contributors. Sometimes it's hard to keep up with everything, so issues, pull requests. So you need to have clear processes for validating and accepting contributions. A good tip is to have a good issue template, and so your users submit issues properly, and you can triage them easily and faster. Finally, while you used to focus on building and shipping lots of features really fast, at this stage you probably need to start prioritizing stability more highly. A roadmap can be a useful tool, and it's not a hard requirement from the CNCF just yet, but it can help you to communicate and set some expectations with the users on what is coming next. Okay, so we have worked to mature our project, and we think we're ready to graduate from the CNCF Sunbox, and how do we actually do it? Well, to move to the next level, you will be asked to fill out DD, which stands for due diligence document, and the purpose of this document is to provide the TUC the information that they need on your project organized really nicely and made readily available, so they can form their opinion whether you're ready to move to the next level or not, and this document also helps them to drive consensus when they're taking the decision. And there will be many questions asked, and what is the purpose of the project? What are the primary use cases? What are the performance or governance or security considerations? So we will dive into those requirements a bit more here. Okay, so starting with the governance requirements. The requirements for incubation aren't that bad. It's mostly just documenting a few things and demonstrating that you've had growth. So maybe more members from various different organizations, you'll be able to get more information from DevStats. These are all just things that just kind of evolve with the project as it matures. But somewhere before, there's a few things that you can implement here that will really help you in the long run. First thing is like documenting your contributor lifecycle. This is technically a graduation requirement, but it's very useful to implement earlier. A contributor lifecycle or sort of creating the policies around what defines a contributor, how they can join the org, become a member or maintainer, and when they should be potentially moved to emeritus status and removed. It's useful to implement early in the project's life sort of like once they've started to firm up their identity and are in the stage where they're going to be potentially adding a lot more people to the project. It removes a lot of the ambiguity around becoming an org member and defines like various sort of activity requirements to continue to be an org member. On the removal side, it's not fun to remove people from a project, but it's important and necessary. Having people in the org potentially with elevated permissions that aren't doing anything there is a legitimate security risk. The other thing is like people are allowed to grow or have other life situations. Things might change and take them away from the project. And this just helps sort of defines it and can help like remove like stem off any sort of like conflict that might arise if someone is potentially has their rights pulled from being an admin or removed from the org. For alpaca, they've decided to like to be an org member. You must sustain 50 dev stats contributions a year or actively serve in a noncode role to stay on as an org member. People might nitpick what meaningful contribution is, but there's a big difference from a single type of fix to opening a PR with a design proposal. Now these requirements may change in the future, but it gives alpaca a good base to work with. Another thing that isn't a requirement at any level but is very useful is a contributor ladder. So similar to documenting the life cycle as sort of like entry exit criteria, a contributor ladder compliments it by creating roles and paths for maintainers into leadership. So many projects start out giving all org members right or admin access or like with merge rights, but that really doesn't scale well as you grow. So having a ladder with defined conditions like on what sort of knowledge does it take to have a new contributor move to a org member or maintainer or to give them merge rights can just like really, really help there. It gives contributors a growth path and can reduce instances of like why was that person granted admin access or why was that person promoted to a leader instead of me? So quickly going back to our fluffy little packaging project, they've decided to delegate to giving people merge rights to the code owners for an area or repo and with it requiring two leads to sign off or a simple majority of say the root owners in the event that a lead for that area isn't available or is no longer active. Next on the like technical docs front, technical docs and processes front, most of the requirements here, again sort of like just naturally evolve as the project matures. The big thing is just remembering to document them. You wouldn't believe how many projects like these naturally things happen, but they don't actually document them and just doing that can like save you a lot of stress later, especially when going for bumping up to the next level. So one of the things that's like not mentioned here is licenses. This has really bitten multiple projects in the rear and has caused problems when they go to try and move to the next level. They'll accidentally import a dependency with an incompatible license. And it's, you know, it is possible to get exceptions, but that can take a long time and requires both the governing board like lawyers and the governing board to sign off and you're talking, you know, months to get that taken care of. It's easier honestly just to like implement a process or CI check to prevent that sort of thing in the first place. Okay. With that out of the way, let's cover the last thing, security requirements. Yay. So we covered the security vulnerability port and sort of like triage process earlier. The remaining things for incubation are like the open SSF best practices badge, which isn't really security related, but just has a bunch of things you should be doing already. Like, are you really seeing with a version scheme? Are your docs up to date? Do you have an SSL certificate? You know, fairly, fairly basic things. And then the last thing is a self-assessment, a security self-assessment using the sort of process created by tag security. And it's really there just to make sure you are following security-minded best, like development best practices. You can see, like, check it out in the CNCF tag security repo for a bit of info. Okay. Deep breath. That's a lot. But let's come back to, you know, Alpaca. At this point, the DD doc is done. They've gone back and like documented the things that were missing. A lot of the things, you know, they were already doing, they just, you know, weren't written down. So they apply for it. And after, you know, some more time in the queue with a whole slew of other projects. The TUC gets to vote and has approved Alpaca for incubating. Lots of people are happy with that. You know, turns out the whole, you know, middle out compression thing works out pretty well. There's even this Pied Piper company that's interested in it. So it's a huge milestone, right? So we take our time to celebrate it with our friends. We organize a meetup, go to local events. We share our journey and plan for all the exciting future. So when you hit such a milestone, please be proud of yourself. Take your time to celebrate and recognize and reward the contributors who help you get there. And once you officially join the incubation, the priorities of your project will be well supported by the processes that you set up, everything that Bob mentioned. And at this point, you'll be experimenting a lot less, right? Now it's time to introduce more controls and processes around releases. And gross is still important, but you want to make sure that you have a path for people to grow. And that's where the ladder that Bob mentioned is really useful. Also, this might like be a great checkpoint for you to revisit your governance, right? You have a lot of people involved in the project working from different organizations, geographies, time zones. So it's very important to have a formalized decision making for each part of your project, like how you're, how you vote on new features, how you make changes in the community like management, or how do you veto certain decisions. And all of that needs to have a formalized approach. On top of those processes, now you get to access to more support from CNCF. This support can help you achieve the user and contributor growth and bring more awareness to your project. So you get to participate in CNCF co-located events like Kubecon. You can have your own small conference like Istio Day or Argo Con. And you also get help with like running community surveys or more help with documentation, etc. But incubation is a process and your project will be part of it for a while, at least a year. It usually takes more like three, four years. You'll continue to build your project, grow your users, your contributors, release quality software, deal with a lot of bugs and open issues and always wish that you were documenting more things. You will continue this journey and after some time you will want to go for graduation. Everyone does. Graduation is the last official milestone within CNCF and the graduation indicates the maturity and long-term health of your project. So again, we find ourselves in a similar situation where we look into requirements. At this stage, you hopefully got all the basics when it comes to software management like releases, issue triage, security, and you do those things really, really well. To go for graduation, you also need to demonstrate that the governance of your project ensures vendor neutrality and long-term survivability. So CNCF looks into the diversity of vendors who participate in your project and how the decisions are made. And to make sure that the project meets these criteria, we go back to our favorite process. And actually, we talked a lot about what requirements we need but we haven't spent a lot of time sharing why there are such requirements. Well, CNCF is 501C6 and ultimately it's responsible to its members. The governing board and TUC are extensions of that. So members join CNCF to ensure they have a voice and get reliable, scalable, predictable software that is vendor neutral with transparent processes, ensuring decisions are made in the open. And this is more important for graduation than any other level because it signals that the project is ready for widespread adoption. Okay. So a package in the home stretch here. You'll actually find there isn't a lot more things for it to actually do. The main thing is now, again, you just must document these things like any sort of like leadership path and how your people are promoted. Similar thing here for technical processes. If you have a roadmap or some other way of tracking your enhancements, something like, you want to have something like Python peps or Kubernetes caps, just some way to show that, you know, docs are getting reviewed or designs are getting reviewed and, you know, give people time to make suggestions and changes. One other thing I will say, it's easy to fall in the trap of making things too complicated or convoluted and you don't want to make it impossible for people to propose, you know, work and to do on like new features. So you need to balance usability and gathering the information needed to review and track your enhancements. Last thing, the security audit. The CNCF has partnered with like multiple vendors that can help you perform it. The biggest thing is like just be available for them and it will help speed things along. So our package has been diligent. It's going well. It means it's time for graduation. We're running a little slow on time, but I promise we're close to being done. Yeah, so we officially graduated project, right? And with this status, we get a bit like more support from CNCF but it's not a lot more. I will try to be faster. So the graduation status is basically a reaffirmation from CNCF of the maturity of the project and its community. So you're stable, you're used in production all over the place and you must be winning, right? Now what? What do we do after that? And well, often by now your project is a few years old and there's broadly a shift in priorities. Okay. So this is where things really start to change. You're being used in a lot of production environments. People want to know that they can upgrade and roll back. That if there's an issue, there is logging and troubleshooting. You are naturally trending towards more stability. The other thing is like your code base might be large and more complex and things can slow down and overall your project kind of becomes boring. But that's kind of expected. You're stable and hopefully you have like the various technical controls in there to help you at that point. And honestly, it's just a big part of being a more mature project. The other thing at this point is you don't have as many good first issues and getting up to your speed in your code base takes longer. By now for one reason or another, people might be leaving the project. You're out of the phase of just trying to get more contributors and now you need to find the balance between your current contributors, your maintainers, and recruiting new people. No, I'm not saying you shouldn't just stop trying to get new people coming in, but you should be more intentional about it and where to direct them. And you need to know where in the project to actually do that. And part of the problem is we all love our projects. We love the people in them. And like I've said it before and I'll say it again, but like open source is how you make friends in your 30s. But because we love our projects, we don't want to let it or our friends down. So it's a lot easy to fall into like bending over backwards for it instead of finding a productive way to say we're down to one person here. Hopefully you have a good ladder and you have some community members in the wings that can backfill some of those people. But it's a good time to think about how you can streamline more things for your maintainers. They're going to be lower on time. The other thing when it comes to streamlining is it includes your governance. Just because you graduated doesn't mean you shouldn't reassess your policies and make sure that they work for you at that point in time. If it doesn't, marry Conduit and if it doesn't spark joy, time to revisit. Okay. So by graduation you should have pretty comprehensive documentation. You'll have your own website. But you need to think more about how you communicate the state of your project. Publishing information on the wins and risks may seem kind of odd for a long-term sustainability, but trust me, its importance can't be overstated. Many maintainers participating as part of their job duties and quite often they literally have to justify why they should be allowed to continue to contribute. So like with a list of achievements a maintainer could say, hey, this is what I helped do and why it's worth it for me to continue to contribute. And if they aren't doing for the job, it's a good thing to include on their resume. The other part of it is looking at the risks. People will just assume you're well-staffed and everything is going smoothly unless you surface it in some way. If I'm an end-user and like Alpaca or another project is critical to my environment, we're not going to commit resources unless there is some way that signals that it is. So the other aspect of communicating all this, well first the risks and achievements complement each other, but when it comes to communicating these things, we're all very, very technical and we tend to communicate in very, very technical ways, but a lot of the people that are making the decisions on allocating headcount or doing this stuff aren't going to understand that stuff. So you need to think about how you can rephrase what you are doing to really reach them. The big point in all this is it actually all boils down to being transparent. Everyone wants growth and to look like they're great in a healthy project, but it doesn't help to hide when there are issues. Being transparent can buy your maintainer some grace because there's certainly a lot of assholes out there, but I swear most people have a bit more empathy and are more forgiving to you like slow repies or slow reviews when they know you're down to like one person. Okay. So you might think you've graduated, you documented everything, everything that Bob covered and you set up all the necessary processes, the project is healthy, you have maintainers, everything is great, so you might think, oh I'm done, right? But as we've seen, there's not the real magical end state and it's ever-changing thing that needs to be managed and the journey will continue and you will think it might look like this straight line from now on, but actually like everything else it will be bumpy and there will be lots of ups and downs and there will be big wins and issues and you're not going to win them all. But building a solid foundation with a mind for sustainability and being able to adapt as things come along can turn your hurdles from something like this to this. And with that, we're done. Thanks for putting up with us going over for a little bit. Yeah, it was too much.