 So today I'm going to talk about evaluating risk for open source projects. I'll start with a brief overview of why it's important to think about risk from a strategic perspective. I'll talk about the difference between company-owned open source projects and foundation-owned ones, along with some thoughts on neutral governance and how that impacts risk. I'll also share some thoughts about what policies and other documents to look for, along with how to evaluate community risks. Then I'll wrap it all up with a few final thoughts and links to some resources. But first I'll tell you just a little bit about me. I've been in the technology industry for more than 20 years, working mostly on open source projects from within companies like Puppet, Intel, and now at VMware, I'm responsible for our open source community strategy within the open source program office. I'm also on the steering committee for the Linux Foundations to Do Group. I'm a board member of OpenUK, and I'm a governing board member and maintainer for the Linux Foundations Chaos Project. I also have a PhD from the University of Greenwich here in London where I researched how people collaborate on the Linux kernel. Now, unfortunately, the scenario on the slide is probably way more common than a lot of people realize. One of the biggest examples of this was a couple of years ago when it was discovered that OpenSSL had a bug in their cryptography library, Heartbleed. Now, OpenSSL is a technology that almost every company in the world uses in one way or another, and it was during this crisis that most of us realized that the OpenSSL project received about $2,000 in donations per year and was maintained mostly by a single person. So yes, that's right. Software that was used by almost everyone had almost no resources to maintain it. And there are probably hundreds or thousands of other open-source projects that are in similar situations. And while this particular situation was resolved relatively quickly, we might not be so lucky in other cases. So evaluating risk is important because it can disrupt your business. And it's important to think about risk as being on a continuum, right? So like most things in life, there are no sure bets. There are no projects or technologies that are completely without risk. There are only projects with more or less risk across a variety of categories, and whether you should accept that risk is a business decision and a strategic decision that should also take how you're using the project into account. It's important to think about risk as part of your company and product strategies. I'll talk in more detail about the various elements of risk throughout this presentation, but I wanted to start by talking about why our strategies, it's important to take those into account. So I'll start with an example of why this is important from a VMware perspective. We're shifting more and more of our business to software as a service or a SaaS model, which means that we're embracing quite a few cloud-native technologies. So as you can see from this iChart, or that's not an iChart, that's just even a subset of the CNCF cloud-native landscape. But you can see that there are a lot of open-source projects in this cloud-native space. Some of them are more risky than others. And one of the things that I've been evangelizing within VMware is that we need to think carefully and strategically about which of these projects we embrace as part of our products. We don't want to build a key product on top of a really risky open-source technology. So it's important to think about what elements of an open-source project might actually introduce risk for us and how we can watch out for warning signs and mitigate these potential risks that I'll talk about throughout the presentation. I wanted to start by talking about why it's important to think about who owns or controls an open-source project. Because it's something that many of us probably don't spend as much time thinking about as we should. The project's governance documents can usually help you learn more about the ownership and control of a project. And from a risk perspective and a strategic perspective, you should be looking for projects with neutral governance where decisions are made by people from a wide variety of companies. Projects that are controlled by foundations and ones that are owned by an individual company can have very different dynamics and different risks associated with participating in those projects. I'll start with a couple of examples of what can happen with projects that don't have neutral governance. Licensing changes are one thing that can increase our business risk, especially when an open-source project is controlled by a single company. There have been several companies recently who are a bit fed up with cloud providers selling SaaS offerings based on open-source projects without actually giving back to the companies or people doing most of the development. Now, their solution was to use more restrictive licenses, which is not great. In the case of MongoDB, they created a new license called the server-side public license that would be used for MongoDB but also be used for security patches to previous open-source versions of the software to prevent people from continuing to use the older open-source version. Recently, Elastic followed suit by moving Elastic Search and Kibana under the SSPL. Now, it's important to note that the SSPL is not an OSI-approved open-source license. So what this means is that effectively MongoDB, Elastic Search and Kibana are no longer open-source projects by the OSI definition. So situations like this one increase your business risk. And when you're building your product strategies, you should be thinking about whether this is a risk for the projects you're using or incorporating into your products. License changes require time from lawyers to interpret those licenses, and new restrictions on how we use an open-source project can delay our product delivery and decrease our productivity if we need to spend extra time finding and implementing a suitable replacement. Governance issues caused by corporate ownership of open-source projects can undermine the project's success and lead to disruptions within the project. In some cases, these disruptions can make it difficult to progress on a project, but they can also result in the project forking and becoming two separate projects. So forking a project in particular can be extremely time-consuming for the project. And all of this takes precious time away from the development of the project, and it slows down progress until things settle back down after the fork. And it increases the business risk for companies who are using or contributing to the project. So for a couple of examples, MySQL was forked into what became MariaDB, when Oracle acquired Sun, and many of the original MySQL core developers including the creator didn't think that Oracle would be a good owner of MySQL. And they wanted to ensure that the code base would remain open. In a similar fashion, shortly after the same acquisition, LibreOffice was forked from OpenOffice. And both of these forks created a significant brain drain away from the original projects, since many of the original developers began working on the new forks. And in the case of LibreOffice, the fork became way more popular than the original project. One way we can avoid situations like this, the ones I talked about on the last two slides, is by supporting projects that are under neutral foundations, like the Linux Foundation, whenever possible. However, it isn't always clear whether a foundation is neutral or not. If the board of directors or other leadership bodies are made up of people who work at a wide range of companies, especially competing companies, that's a good indication that it's a neutral foundation. If the board members are made up primarily of people from a single company or a couple of companies with very close ties, it's not likely to be a neutral foundation. You can also look at who owns the trademarks for the foundation and for the projects within it. And you can look at the projects themselves. Did they come from a single company or a variety of companies? And the more you see references to a single company across the projects, across the board, the less likely the project is to be neutral. CNCF, the Linux Foundation, Eclipse, Apache Software Foundation, those are all neutral foundations. One of the reasons that Kubernetes has been so incredibly successful is because it was contributed to the Cloud Native Computing Foundation. Putting Kubernetes into a neutral foundation provided a level playing field where people from a bunch of different companies could work together as equals to create something that benefits the entire ecosystem. Open source projects that are controlled by a single company are higher risk because they operate at the whims of that company. And there is little recourse for those of us outside contributors when that company decides to go in a direction that doesn't align with the expectations of the other participants within that community. Think back to the examples from earlier where projects moved from non-open source licenses or were forked. For company-originated projects, also consider the reputation of that company as a steward for open source projects. But always keep in mind that they can actually change their strategy at a later date or at any point. And I'm not saying that you shouldn't participate in open source projects owned by companies. But you should think about the risks that you're taking. In some cases, it makes a lot of sense to accept that higher risk. But you should try to at least make sure that you understand the risks you're taking and how those risks could impact your company and your strategies. Governance can be found in all open source projects in one form or another. And it helps outline the expectations around roles and responsibilities along with the decision-making process. But it will vary quite a bit by the size of the project. A project with only a few people will not need elaborate elections to select leaders. But they should probably have a few basic roles defined like Maintainer, for example. Knowing how collaboration occurs and how decisions are made to being able to make contributions that are more likely to be accepted. If the processes for contribution, collaboration, leadership, and decision-making are not clearly documented as part of the project governance, this can increase the risk of participating in or using the project, because it just introduces a lot of uncertainty. One way to help mitigate this risk is to have some discussions about governance within the community to gauge interest and getting some help with governance. In a lot of communities, there can be various types of informal governance that just isn't documented. Quite a few people tend to avoid working on governance. It seems a lot like paperwork. So if it's missing, you might be able to volunteer to help. And it's a great way to learn about how the community operates. On the other hand, sometimes governance is missing because someone doesn't want the governance to be documented. For example, the project might be tightly controlled by a company or a group of people in ways that they don't want to make explicit. Or maybe someone doesn't want a neutral governance process that might require them to give up a little bit of control. And in some cases, the project leaders, frankly, are just naive enough to think it isn't important. In these cases, your attempts to help with governance will likely be met with stonewalling, avoidance, and at the worst, even hostility. So this makes the project a much higher risk since your governance concerns aren't likely to be addressed or mitigated. Because leadership is such a critical piece of governance, I wanted to spend just a little more time talking specifically about it. At a minimum, there should be some kind of documentation about leadership for the project. There might be committees, working groups, special interest groups, other opportunities for leadership. And if you become an active participant in the project, you'll want to understand how your contributors at your company can move into leadership roles. Having employees in leadership roles helps reduce your risk as a company because these employees will be in a position to understand more about how the project works and be able to help mentor other employees into the project. There are lots of options for selecting leaders as part of defining governance, but the ideal is to have a process that provides a fair and level playing field that defines how contributors can eventually become leaders. And the key is that this fair and transparent process, something like elections, reduces your risk of participating since your contributors can participate in the elections and have a say in the governance of the project. Higher risk options would include organizational leadership seats where a certain number of positions are allocated to specific companies, where those companies get to decide who gets their leadership positions. And there's no easy way for new leaders from outside of those companies to move into leadership positions. Regardless of how the leaders are selected, you want to make sure that there are opportunities for your contributors to move into leadership positions to reduce the risk for the projects that you use and contribute to. In addition to the governance documentation, there are quite a few other policies and documents that you can look at as part of your risk assessment for a project. People often worry about security for open source projects under the premise that if everyone can see the code, they might also be able to find and exploit security vulnerabilities. But the flip side of this is that there are also more people who can view the source code for the purpose of fixing those security issues. And if you work within a product team internally on a project, you have a limited number of people and an increased chance that your shared backgrounds lead you to think of things in similar ways. And with people from many different companies and backgrounds contributing to an open source project, those innovative ideas will also apply to proactively making the project more secure, along with reactively fixing any vulnerabilities that might arise at any point. Now, of course, it certainly doesn't mean that open source projects are automatically secure. One thing I look for when assessing the risk of open source projects is that they make regular releases and that they quickly patch security vulnerabilities. You should also look at the project's security policy documents and make sure that they have a solid process for allowing anyone to privately report vulnerabilities and dealing with those reports quickly and efficiently. They should also have an embargo process for privately notifying the companies who use their projects in their products to give these companies time to fix security vulnerabilities before that vulnerability is made public. From a strategic standpoint, having time to update your products in advance of the public knowing about a vulnerability is critical. And in short, projects that take a proactive approach to addressing security issues and releasing fixes are going to be a lower risk. There are a few documents that every project, even the small ones, should have. Open source projects need appropriate licensing documentation. They should also have a code of conduct. The contribution documents should provide enough detail about the process that someone brand new to the project can make their first contribution. The communication processes should be clearly defined so that people know where and how to communicate within the project. To understand how people within the project communicate with each other, it avoids frustrating experiences, not just for new contributors, but the existing ones too. And one way to mitigate these risks is to help volunteer to help with the documentation. Long-term contributors may not notice that their documents are incomplete or incorrect, especially the new contributor documents. And someone new to the project can often be a big help in finding those problems and making improvements. And so the better all of this is documented, the lower the risk, since your contributors will know what to expect and then how to navigate the project when making contributions. The community can make or break an open source project, and the overall success of a project often depends on the community. The community is usually the first thing I look at when I'm trying to decide if an open source project is healthy. Is the community active and engaged in productive discussions around the project? Are they helpful when people have questions or problems? Are they welcoming for new contributors and new users? Do they treat each other with kindness and respect? You want to look carefully at how people treat each other on mailing lists and GitHub reviews, Slack, other communication channels. Projects with a culture of treating each other with respect and kindness are going to be a lower risk because your employees will want to continue to contribute. On the other hand, projects with toxic cultures put your employees and other contributors at risk. There could be real health risks both mental and physical for your employees. And these toxic environments can impact retention since employees might be so eager to leave this project that they'll also leave your company because they don't want to work in that toxic community. And if a culture is too toxic, it can be very difficult or even impossible to mitigate that risk. If you're designed to keep diversity, equity, and inclusion top of mind, it will also be lower risk. Building a diverse community where all people feel welcome and included doesn't just happen. It requires putting work and thought into it. I mentioned Code of Conduct earlier and having a Code of Conduct that's actively enforced helps make sure that everyone feels safe and welcome in a community. Providing an environment where everyone, including people from marginalized populations, feel safe is an important way to lower the risk of having your employees participate in a project. You should also look at whether the project has a diverse set of people at every level within the project, but especially at those top leadership positions. If not, do they have active programs that help underrepresented folks get involved and move up within the project? And ideally, having programs that give people opportunities for shadowing, mentoring, sponsoring new potential leaders can help the project grow a diverse set of people into new leaders within the project. The Kubernetes contributor experience SAIG does a great job of this. And projects that make a concerted effort to bring these people in from a variety of backgrounds and help them grow into leadership positions are more likely to be lower risk, benefit from increased innovation and just have a healthier community overall. And this increased innovation from a diverse community is particularly important from a strategic perspective for companies who use these projects. If you're actively using an open source project, it's likely that you'll find bugs or want to contribute fixes back upstream. You'll want to make sure that maintainers and other contributors typically respond in a timely fashion. Seeing large numbers of issues and pull requests on projects is a red flag for me because it can indicate that either they don't have enough contributors to handle the incoming contributions or even worse that they don't actually care about or want contributions from others which is something you see in some company owned open source projects. In some cases, you can help solve this problem by making contributions to open source project part of the job for some of your employees. However, there are legitimately some projects that don't actually want contributions from people outside of their company or outside of their existing circle of maintainers. And this can happen when a project was open source for the wrong reason like politics or marketing without buy-in from the people actually responsible for that project. And these projects are a much higher risk since you're way less likely to get any changes incorporated in that upstream project. We don't always spend enough time thinking about the contributor risk associated with many open source projects. For any open source projects we rely on, are there enough contributors that if one of them won the lottery and retired on a beach or on a road, could the project continue with minimal disruption? This is a bigger risk for projects like OpenSSL which contain highly complex technologies. You should also look at organizational diversity as a part of the health and risk for open source projects. If all or most of the contributions are from the employees in a single company, what happens when that company has a shift in strategy or gets acquired or just runs out of money and goes out of business? Would the project be able to continue if the company pulled all of its employees out of the project? And again, it comes down to risk and whether we are willing to take that risk. There are so many abandoned open source projects sprinkled across the internet and there's no single reason why projects are abandoned, technologies evolve, people change priorities, maybe the community loses interest. One thing that can reduce the risk of embracing an open source project is if the project is adopted by a bunch of other companies, since active projects with lots of users are less likely to be abandoned. It isn't always easy to know who uses a project but for CNCF projects they have an adopters.md file which can help you figure that out. Now before I wrap up the talk I'll just leave you with a few resources and these slides are uploaded to the schedule. The to-do group has lots of guides with great information. I'm part of the CNCF contributor strategy technical advisory group and we have a governance working group and a contributor working group which give lots of guidance about governance, contributor experience, sustainability and openness. The open source way guidebook is also a really great resource. They have a whole section on governance but loads of other really great information. Most business decisions boil down to an assessment of risk and making trade-offs. There's a lot of research strategically about project risks in light of how we're using these projects within our business. If we build products on top of an open source technology we want it to be as low of a risk as possible. On the other hand if we're using an open source project as some small part of non-critical infrastructure that doesn't matter that much then we can accept a lot more risk. Now no project is all good, all bad, all risk is right for every situation. Project risk, like most things, exists on a continuum. Deciding which risks we're going to accept is the beginning, not the end of the process since we should also be thinking about how we can monitor and mitigate those risks over time. We need to think about the risks that we might be taking and make informed and strategic business decisions about which risks to accept. Thank you for coming to my talk. I'm going to start with the late start and I'll bring it up for questions. Yes? From the risk perspective I think all the points are good. I was just wondering would testability of the code be a factor that could be considered for risk assessment by the clients? And if so, what would be the type of testing that would be minimal to consider at low risk as a point? Yeah, that's a good question. I do certainly cover as much as I could within the time allotted. Testing infrastructure is also a good thing to look at when it comes to risk because if a project is not well tested you can end up with all sorts of issues that you don't expect. I'm also not an expert in testing so I don't know where that threshold would be. What I would probably look for is do they have tests? Do they run automatically? Seriously, when something doesn't pass a test? And that would be kind of a starting point. And then I think probably, depending on the individual technology, looking more deeper into what types of tests they run would be a good idea. Is this something that is being kind of like discussed or encouraged in the CNCF projects for graduation of the project? I don't know the answer to that. Yeah, I don't remember. Alright, thank you. Yeah. Is there a policy in all these places that says these are the automated checks that you go through to evaluate some of the policy guidance and stuff like that? Yeah, we do. We have an internal process that every product has to go through and it does look at every single open-source project that's within it. And we do have certain things. We have licenses, for example, so if something is under the SSPL that product does not go out because that's one of our prohibited licenses. So we have some checks that can be more automated in place. And so those are really good, but unless it's a strategic project or something that is important enough for us to notice it's really hard to tell because we consume so many open-source projects. And like you said we kind of have free reign. So what I've been evangelizing internally within VMware is really trying to get some of those developers to think about the risks they might be taking when they incorporate this random project from the internet. Yep. It's really a tough problem to solve. It's one of those things that you can't entirely believe it developers will just sort of include what they include and you may or may not notice it when you go through these automated processes. So all I can do is educate people and try to get them to think about it and we do sometimes dig back through the tooling to see what people are doing. Like how old is that version of Elasticsearch that they're trying to use in their product? The old one that was under the open-source license. So we do look at some of that, but it's a hard to solve. I mean all I can do is talk about it and try to get people thinking about it. Any other questions? I think we're probably almost running into lunch. Yeah, go ahead. That's a good question. We tend to do it. It's a bit more ad hoc. So we do assessments on some things, not like a full risk assessment like what I've got here. I don't think we've quite gotten to the point where we have people doing that. I would like that. We do some risk assessment for what we consider our strategic projects, which is about 30 projects and it's some combination of third-party projects we contribute to heavily, so things like Kubernetes and individual open-source projects that we run within VMware. And so we do more health assessments on those and part of what we look at is the contributors who's it owned by some of the things that we're talking about here. But I would say, oh, sorry, I came back to your how often you should do that. I think we do that about quarterly ish for the strategic projects and I know Red Hat has a process for doing this as well. They do pretty regular health assessments out of their open-source programs office as well. And I'm trying to remember how often they do it. I think it was like once a year, twice a year or something like that. Any last questions? Okay, I think I'll let people go to lunch. Thanks for coming to my talk.