 Welcome to my talk today. I'm going to talk about evaluating risk for open source cloud native projects. I'll start with a brief overview of why it's important to think about risk. I'll talk about the differences between company owned open source projects and projects under foundations, along with some thoughts on neutral governance and how all of this impacts the risk. I'll also share some thoughts about what policies and other documents to look for, along with how to evaluate the community risks. And then I'll wrap it all up with a few final thoughts and links to some useful resources. But first, I'll tell you just a bit about me. I've been in the technology industry for over 20 years, working mostly on open source projects from within companies like puppet and Intel at Pivotal. I was responsible for driving Pivotal's Kubernetes contribution strategy, and I was actively contributing to the Kubernetes contributor experience special interest group. Now I'm working in VMware's open source program office, and I'm responsible for our open source community strategy. So lately I've been active in the CNCF contributor strategy CIG, in particular the governance working group. I'm also on the steering committee for the Linux foundations to do group where companies collaborate on ways to run successful open source program offices. I'm also a board member of Open UK, which is focused on developing and sustaining leadership and open technology here within the United Kingdom. I'm a governing board member and maintainer for the Linux foundations chaos project, which is focused on using metrics to evaluate the health of open source projects. I also have a PhD from the University of Greenwich here in London, where I was researching how people working at many different companies collaborate within the Linux kernel. Unfortunately, the scenario on the slide is probably more common than many people realize. One of the biggest examples of this was a couple of years ago when it was discovered that open SSL had a security bug. In their cryptography library referred to as Heartbleed. Open SSL is a technology that almost every company in the world uses in one way or another. And it was during this crisis that most of us learned that open SSL received about $2,000 in donations per year, and was maintained mostly by a single person. So yes, that's right. Software that is widely used by almost everyone had almost no resources to maintain the software. And there are probably hundreds or thousands of other open source projects in the same or similar situations. In this case, the Linux Foundation got some companies together to provide funding for people to work on open SSL and other similar projects on a full-time basis. And I know VMware and other companies have also been contributing. While this situation was resolved relatively quickly, we might not be so lucky in other cases. So evaluating risk is important because it can disrupt your business. It's also important to think about risk as being on a continuum, like with most things in life, there are no sherbets and no project or technology is completely without risk. There are only projects with more or less risk across a variety of categories. And whether you should accept that risk is a business decision that should also take into account how you're using the project. And in this session, I'll talk about how to evaluate the risks for open source projects, but all projects and all software have risks. So evaluating risk is something we should do for all projects, not just the open source ones, but I'll talk about how you do it for the open source projects in this talk. I wanted to start by talking about why it is so important to think about who owns or controls an open source project. Because it's something that many of us don't spend as much time thinking about as we should. The projects governance documents can help you learn more about the ownership control of a project. And from a risk perspective, you should be looking for projects with neutral governance, where decisions are made by people from a variety of different companies. Projects that are controlled by foundations and ones that are owned by an individual company can have very different dynamics and different risks associated with participating in those projects. I'll start with a couple of examples of what can happen with projects that don't have neutral governance. Licensing changes are one thing that can increase our business risk, especially when an open source project is controlled by a single company. In 2018, there were a couple of high profile examples of companies like Redis Labs and MongoDB, who became fed up with the cloud providers who sold software as a service offering based on these open source projects, without giving as much back to the companies who were doing the development as the companies would have liked. And these companies, their solution was to put more restrictive licenses on their software. In the case of MongoDB, they created a new license called the server side public license. They would be used for MongoDB, but it also be used for security patches for previous open source versions of the software to prevent people from just continuing to use an older open source version and applying the security patches to it. The problematic bit of the license is in section 13, which says that if you use the project in a software as a service offering, you have to not only make your source code and modifications available, but also the source code of the applications used to run the service. Recently, Elastic followed suit by moving Elastic Search and Kibana under the SSPL, and they made this license change as part of a minor point release rather than a major release, which is kind of interesting. It's important to note that while they say that the SSPL is based on an OSI approved license, they were not able to actually get the SSPL approved by the OSI primarily because of some concerns about the restrictions put on how people use the software. So the SSPL is not an OSI approved open source license. This means that by most definitions, MongoDB, Elastic Search and Kibana are no longer open source projects. Situations like this one increase your business risk and can impact your ability to deliver software. License changes require time from lawyers to help us interpret these new licenses. And new restrictions on how we use an open source project could delay our product delivery and decrease our productivity if we need to spend extra time finding and implementing a suitable replacement. Governance issues caused by corporate ownership of open source projects can undermine the project's success and lead to disruptions within the project. In some cases, these disruptions make it difficult to make progress on a project, but they can also result in the project working and becoming two completely separate projects. Forking a project in particular can be extremely time consuming for the project. This takes precious time away from the development of the project and it slows down progress until things settle back down after the fork. It increases the business risk for companies who are using or contributing to the project since it can make it difficult to get contributions back into the upstream project during periods of disruption. When a project works as a company, we need to decide whether to continue with the original project or embrace the fork, which is a gamble, especially at the beginning because it can take months or years to see which project will eventually prevail and go in a direction that works well for how we use the project. So here are a couple more tangible examples. MySQL was forked into what became MariaDB when Oracle acquired Sun. Many of the original MySQL core developers, including the creator Monty, didn't think that Oracle would be a good owner of MySQL and they wanted to ensure that the code base would remain open. In a similar fashion shortly after the same acquisition, LibreOffice was forked from OpenOffice. And less than a year later, the developers who were being paid to work on OpenOffice were laid off and OpenOffice was donated to the Apache Software Foundation. Both of these forks caused a significant brain drain away from the original projects, since many of the original developers began working on the new forks. And in the case of LibreOffice, the fork became more popular than the original project. When the Istion K-Native project started, companies like Pivotal, VMware, Red Hat, and IBM contributed a bunch of code to these projects with the understanding that the projects would be put under the CNCF. Later, Google decided that they were not going to transfer them to the CNCF and because they owned the trademarks, the rest of us had no real choice in the manner. In both of these cases, the communities have been able to negotiate for better governance, which has helped stabilize the projects. But I still think these projects are at risk of eventually being replaced by other technologies or eventually forked if they're not put under a more neutral foundation or neutral governance. One way we can avoid situations like the ones I just talked about on the last two slides is by supporting projects that are under neutral foundations, like the CNCF, whenever possible. However, it isn't always clear whether a foundation is neutral or not. If the Board of Directors or other leadership bodies are made up of people who work at a wide range of companies, especially competing companies, that's a good indication that it's a neutral foundation. If the Board members are primarily made up of people from a single company or a couple of companies with close ties, it's not likely to be a neutral foundation. Trademarks are another thing to look at. In a neutral foundation, the foundation should own the trademarks associated with especially the name of the foundation, but probably also most of the projects. If the trademarks are all owned by a company, especially a single company across most of the projects, that's a red flag that the foundation is probably not actually neutral. It can also help to look at the projects under the foundation, were they originated as a single company, or did they come from a variety of companies? Do the projects have leaders from a variety of companies? Do they have CLA's from one company or is the CLA from the foundation? The more you see references to a single company across the projects, the less likely the foundation is to be neutral. The CNCF, the Linux Foundation, the Apache Software Foundation are all solid neutral foundations. On the flip side, the open usage commons is not one that I would classify as a neutral foundation. One of the reasons that Kubernetes has been so successful is because it was contributed to the Cloud Native Computing Foundation. Putting Kubernetes into a neutral foundation provided a level playing field where people from a bunch of different companies could work together as equals to create something that benefits the entire ecosystem. The CNCF has a big focus on helping Cloud Native projects set themselves up to be successful with resource documents, maintainer sessions, and help with various administrative tasks. Open source projects that are controlled by a single company are higher risk because they operate at the wins of that company. And there's little recourse for outside contributors when that company decides to go in a direction that doesn't align with the expectations of the other participants within the community. Think back to the examples from earlier where projects moved to non open source licenses or were forked. For company originated projects consider the reputation of that company as a steward of open source projects. But always keep in mind that they can change their strategy at some later date. I'm not saying that you shouldn't participate in open source projects owned by companies, but you should think about the risks that you're taking. In some cases, it makes a lot of sense to accept this higher risk, and we do it all the time. But you should try to make sure that you understand the risks that you're taking and how those risks could impact your company. Governance can be found in all open source projects in one form or another to specify the processes for how people work together within the project. Ideally, it will be it will be clearly documented, but for some projects, especially smaller ones, it might be a bit more ad hoc and informal. Regardless of what form it takes evaluating projects governance is an important element of assessing the risk of using or contributing to that project. Governance helps outline the expectations around roles and responsibilities, along with the decision making processes, but it will vary quite a bit by the size of the project. A project with only a few people won't need elaborate elections to select leaders. But the project should have at least a few basic roles to find like maintainer, for example, for CNCF projects governance documentation is encouraged for all projects. But by the time a project hits the incubating stage, they need to have a governance MD file showing who is currently leading the project and how future leaders are going to be selected with a requirement that the process be employer neutral. Knowing how collaboration occurs and how decisions are made is vital to being able to make contributions that are more likely to be accepted. If the processes for collaboration, leadership and decision making are not clearly documented as a part of the project governance. This can increase the risk of participating in or using the project because it introduces a lot of uncertainty into the mix. There should also be some kind of project charter, which can take many forms and frankly, in most cases, it's often not actually called a charter at all. But instead it takes the form of some sort of mission statement, a statement about scope, values, principles, and similar concepts that are often found either within the governance documents or in the media files for the project. The overall cloud native ecosystem is complex with many different projects containing overlapping functionality. A charter, especially a scope can help end users understand how your project fits into the overall ecosystem and what functionality it does and does not offer as compared to the many alternatives. Having this documented early can help avoid issues and misunderstandings later. One way to help mitigate this risk is to have some discussions about governance within the community to gauge interest in getting some help with governance and a lot of communities. There can be various types of informal governance that just isn't documented. Quite a few people tend to avoid working on governance. So if it's missing, you might be able to volunteer to help and it's a great way to learn more about how the community operates because leadership is such a critical piece of governance. I wanted to spend a little more time talking specifically about it. At a minimum, there should be some kind of documentation about leadership within the project. For small projects, maybe it's just a list of maintainers in a README file. Or maybe owners or maintainers files that indicate which people are responsible for the various aspects or areas within the project. For bigger projects, you'll want to see more details about the specific roles and responsibilities for different leadership positions along with how others can move into leadership roles. There might be committees, working groups, special interest groups or other groups with opportunities for leadership. If you become an active participant in a project, you'll want to understand how your contributors can move into leadership roles. Having employees and leadership roles helps reduce your risks as a company since these employees will be in a position to understand more about how the project actually works and be able to help mentor other employees and bring them into the project. There are quite a few different options for selecting leaders as part of defining governance. And the ideal is to have a process that provides a fair and level playing field that defines how contributors become leaders. This should be documented so that all participants can clearly understand the criteria and process for moving both into and then out of leadership positions. Most of the bigger projects like Kubernetes have an election process, at least for the top levels of leadership like a steering committee. You should also look for programs that give people opportunities for shadowing, mentoring and sponsoring new potential leaders. This shows that the project is serious about growing a diverse set of people into new leadership roles within the project. As I mentioned earlier, CNCF incubating projects need to have a governance.md file showing the leaders and how they're selected with the requirement that the process is employer neutral. The key is that having a fair and transparent process for selecting leaders like elections, for example, reduces your risk of participating since your contributors can participate in the elections and have a say in the governance of the project. Higher risk options would include organizational leadership seats where a certain number of positions are allocated to specific companies where those companies get to decide who gets their leadership positions. And there's no easy way for new leaders to work outside of those companies to move into those leadership positions. Regrets of how the leaders are selected, you'll want to make sure that there are opportunities for your contributors to move into leadership positions to reduce the risk for the projects that you contribute to. In addition to the governance documentation, there are also quite a few other policies and documents that you should look at as part of your risk assessment for a project. There are so many abandoned open source projects just sprinkled across the internet. And there is no single reason that projects are abandoned, technologies evolve, people change priorities, maybe the community loses interest. One thing that can reduce the risk of embracing an open source project is if the project is adopted by a bunch of other people since active projects with plenty of users are less likely to be abandoned. It isn't always easy to know who uses a project, but an adopter's file is a good start. It's a project has one. An adopter's file is actually required for CNCF projects to reach the incubated stage, although it may not be public until the project is graduated. Other projects might choose to have this earlier. And for CNCF projects, it's usually in an adopters.md file, but having a strong user base reduces the risk that the project might be abandoned. People often worry about security for open source projects under the premise that if everyone can see the code, they might also be able to find and exploit security vulnerabilities. But the flip side to this is that there are also more people who can view the source code for the purpose of fixing those security issues. This is where innovation and diversity of ideas comes in. So within a product team working internally on a project at your company, you'll have a limited number of people assigned to the product. And an increased chance that your shared backgrounds lead you to think of things in similar ways. With people from many different companies and different backgrounds contributing to an open source project, these innovative ideas will also apply to proactively making the project more secure, along with then of course, reactively fixing any vulnerabilities that minorize later. However, this certainly does not mean that open source projects are automatically secure. One thing I look for when assessing the risk of open source projects is whether they make regular releases and quickly patch security vulnerabilities. You should also look at the project's security policy documents to make sure that they have a solid process for allowing anyone to privately report vulnerabilities and for dealing with those reports quickly and efficiently. Projects that take a proactive approach to addressing security issues and releasing fixes are going to be a lower risk. There are a few documents, even the small ones, or few documents that every project, even the small ones should have. It should go without saying that open source projects need appropriate licensing documentation. And some licenses require things like notices file or notices files or license headers within each source file in addition to just putting a license file in the repository. Projects should also have a code of conduct that people can easily find with details about how to report incidents, along with the consequences of violating that code of conduct. The contribution documents should provide enough details about the contribution process so that someone new to the project can make their first contribution. The communication process should also be clearly defined so that people know where and how to communicate within the project. If everyone understands how people within the project communicate with each other, it avoids frustrating experiences for both new and existing contributors. One way to mitigate these risks is to volunteer to help some of this documentation and establish projects long term contributors may not notice that their contribution documents are incomplete or even incorrect. And someone new to the project can often be a big help in finding the problems that are needed to improve that documentation. And the better all of this is documented the lower the risk since your contributors would know what to expect and how to navigate the project when they want to make contributions. The community can make or break an open source project and the overall success of the project often depends on the community. The community is usually the first thing I look at when deciding if a project is healthy. Is the community actively engaged in productive discussions around the project. Are they helpful when people have questions or problems. Are they welcoming for new contributors and new users. Do they treat each other with kindness and with respect. One of the reasons that Kubernetes systems are so successful is because people truly enjoy participating in the community. From the beginning I felt welcomed, the people were fun, and there were plenty of opportunities to step in and help out with things. Kubernetes also has loads of resources documentation and training for new contributors, along with mentoring programs, which helps people get started and want to continue participating. You'll want to look carefully at how people treat each other on the mailing list. And in GitHub reviews and it's like an IRC and all of the other communication channels projects with a culture of treating each other with respect and kindness are going to be a lower risk. Because your employees will want to continue to contribute. On the other hand, projects with toxic cultures put your employees and other contributors at risk. There could be real mental and physical health risks for your employees. And these toxic environments can impact retention since employees might be so eager to leave the project that they'll look for another employer where they won't have to work in this toxic community. And if the culture is too toxic, it can be very difficult or possibly even impossible to mitigate that risk. And if the community isn't amazing for a variety of reasons, this, this is a risk. It doesn't necessarily mean that you shouldn't get involved or use the software, but you should approach it with caution and understand that there is a risk that could go terribly wrong with the project as a whole. If things get even worse within the community and your key contributors just start disappearing out of the project. If you're actively using an open source project, it's likely that you'll find bugs or want to contribute fixes back upstream. For new projects you're considering, you'll want to make sure that maintainers and other contributors typically respond in a timely fashion, seeing large numbers of issues and pull requests on a project is a red flag for me, because it can indicate that either they don't have enough contributors to handle the incoming contributions, or worse that they don't actually care about or want contributions from others, which is something you see in some company owned open source projects. But these two scenarios not having enough contributors to respond to pull requests and issues is probably more common. And most projects I've worked on have struggled at one point or another with keeping up with full requests. In some cases you can help solve this problem by making contributions to open source projects part of the job for some of your employees and helping work through this backlog. As I mentioned, there are legitimately some projects that don't actually want contributions from people outside of the company or outside of their existing circle of maintainers. And this can happen when a project was open source for the wrong reasons like politics or marketing without buy-in from the people who are responsible for the project. And these projects are a much higher risk since you are way less likely to be able to get any changes incorporated into that upstream project. We don't always spend enough time thinking about the contributor risk associated with many open source projects. For any open source projects we rely on, are there enough contributors that if one of them won the lottery and retired on a beach, could the project continue with minimal disruptions? This is a bigger risk for projects that are highly complex or contain technologies that require specialized expertise, like we saw a few years ago with the open SSL, Heartblade example. You should also look at organizational diversity as a part of the health and risk for open source projects. If all or most of the contributors are from a single company, what happens when that company has a shift in strategy or gets acquired or runs out of money and just goes out of business? Would the project be able to continue if that company pulled all of its employees out of the project? And again it comes down to risk and whether we're willing to take that risk. Now before I wrap up the talk, let me leave you with a few resources that you might find useful. The CNCF contributor strategy CIG is a governance working group and a contributor growth working group, which provide guidance about contributor experience, sustainability, governance and openness to help CNCF projects develop strategies for maintaining project health. The to-do group has a bunch of guides that have great info on all aspects of creating and managing open source projects. The open source way guidebook has an entire chapter on governance along with loads of other details about all kinds of open source projects. And these are all great starting places for understanding best practices for open source projects to help you evaluate your risks. Most business decisions boil down to an assessment of risk and making trade-offs. We should all be thinking about project risks in light of how we're using these projects within our business. If we build products on top of an open source technology, we want it to be as low of a risk as possible. On the other hand, if we're using an open source project as some small piece of some non-critical infrastructure, then we can accept more risk. Now no project is all good or all bad, all risks are no risk, and no one solution is going to be right for every single situation. Project risk, like most things, exists on a continuum. Deciding which risks we're going to accept is the beginning, not the end of the process, since we should also be thinking about how we can monitor and mitigate those risks over time. We need to think about the risks we might be taking and make informed business decisions about which risks to accept. With that, I will say thank you for coming to my talk and we'll open it up for questions.