 Hello! We are Ivana Tanasova and Stepka Dimitrova from the VMware open source program office. It's nice to meet you here in this virtual talk. And as you probably already got familiar, we are going to talk about open source projects held. Do you know how often open source projects are being abandoned? Based on a survey held last year, 72% of the people included have left or neglected a project recently. This means that three-fourths of the people here in this talk may have a really great idea that they would abandon. So why do you think open source projects crash? Feel free to take a second and think about it and us to share your thoughts with us in the chat. We have asked several open source practitioners what are the first reasons that come to mind when they hear about an unsuccessful open source project. And this is what we got from them. Yeah, you probably see some of your guesses and proposals here. And there are probably many others which are missing because the open source projects are so diverse. But we've seen some repeating patterns and we wanted to summarize them. So here are some of the key points that prove to successfully crash any open source project. And we try to think them as the red flags that we want to look after and pay special attention to. Starting with the cases where key contributors have lost interest or they just have no time until our lack of resources to work on a project and then it's low maintained. It's followed by legal issues, also strong competition and any security concerns that might lead to putting on hold a project and eventually abandoning it. Very critical can be when there are interpersonal conflicts within the team and especially when they evolve. But luckily in open source we think that the failures contribute to the next great thing that comes up. So this is a quote from a person who implemented a very successful open source idea. We have a lot of presses used in more than 30% of the web. And we also believe in that and we try to think of instead of problems just talk about improvement opportunity solutions and learnings. So that's why we transform that common problems that we have just observed into key health indicators. And these are the areas that need special attention. Governance, contributor risk, diversity and inclusion, activity and security. There are also others but we want to look into more detail in these five. So talking about governance, it has several aspects. One of them is related to how the project is maintained. This means are there potential leaders that in addition to the maintainers can grow and take such a responsibility. This ensures less chance of maintainer burnout and that there will be a continued support should the maintainer exit the project. The other aspect is whether the project is owned by a company or given to a foundation. There are pros and cons for both cases but there are certain risks if it's led by an individual contributor or company owned. Regarding projects independence and future especially because in that case there is no guarantee whether governance is going to turn into radical direction that do not comply with adopters requirements. This can include license changes and functionality and compatibility. And the third is related to diversity. Our project contributor is coming from a diverse set of companies or organizations. The case here is pretty much similar. For example, a company may force an open source project to depend on their enterprise product or use an inappropriate license. So having contributors and maintainers from different companies guarantees that there will be a balance in the direction of the project development. And talking about diversity, it's even more important because it has other aspects as well. One of them is the welcoming side. An environment from a single type of people may result in not welcoming contributors different than the rest of the community and this increases the risk of governance issues. Regional diversity is also very important because it increases the chance to include people from different time zones. It also allows the so-called fall the sun type of development and it's very efficient because of its 24-7 availability. We also saw from our survey that toxic, unfriendly and even aggressive environment may force people to step back and can result in a project to become dormant over time. We also saw in the survey that it's coming for people to either use interest or lack time to work on a project. The so-called contributor risk is also known as the bus or truck factor indicates the probability of a project to become fully unsupported. For example, if it has only one maintainer and this maintainer steps back, it's possible for any changes to be merged anymore and for any issues to be addressed, to be fixed and so on. This health indicator includes not only the number of project owners and maintainers, but also the tendency to give such responsibilities to new people and also whether the maintainers try to share knowledge and educate the rest of the community so that they can take that role in the future. Another very important aspect is how active the project is. How often changes are submitted, how often poor requests are being revealed, how long it takes for them to be merged. The lack of reviews from the community or the extremely short merged period indicates risks for bugs, regressions and poor quality. But the opposite is also not the best because a reasonably long time for a change to be merged indicates that there is not enough maintainers and the project does not grow with the desired rate or that the maintainers don't have time for it which indicates risk that it can become dormant over time and can be abandoned. So the truth is in the middle, in the balance between both, the other thing is responsiveness. So the easiest way to see how a responsive approach is is to open an issue and see how long it will take for you to get a reply in it. If it takes too long, this means that the community is either not active enough or very small and doesn't have time to reply. This hides concerns that bugs or security issues are not going to be addressed in a timely manner. A few months ago I was working on a feature on a networking-related project that required external library for network visualization. And I found out one really cool project on GitHub. It looked like in a sci-fi movie. I said, oh, that's great. I want to use that. And in my excitement, I didn't notice that it was not updated since two years. And yeah, of course it didn't run when I tried, but I updated all the versions because it was incompatible with the latest versions of Kubernetes Docker and so on. But yeah, this was not enough. I was trying to fix some ancient issues. And I was so enthusiastic that I was two weeks trying to fix that project and make it working. And after two weeks, I realized that I cannot even estimate the time that will be required to resolve all the issues. And that we will need to support it in future if we continue using it. So it's not the best approach. And the most obvious indicator of how active a project is is when is the last commit made. And the other aspect, the last but not least, is how often the project ships new releases. It's an indicator of how fast it's growing, how active it is in developing new features and fixing bugs. But even more important is how often security concerns are being addressed. A well-maintained project requires a well-established security protocol. This includes documentation and advertising how to report security issues. It's crucially important that those are addressed and released on time so that for anyone using the open source project, they will be safe in not exposing any security vulnerabilities. Yes, we've tried to summarize all these aspects to give you some details, some of our own experience and the mistakes that we also did because they're all important to be considered, especially when you're starting a new project. But everyone who has started something from zero knows how many tasks that involves and they keep popping up and observing some project health indicators might seem like the last thing to do. Even if you're able to take off without having that in mind, how do you know what direction are you heading toward and are you still on the right course? And keeping that right course, that's important. That's why we want to not only just explain the overall areas to be looking into but also help you define concrete parameters to be observed. You'll be all compass or monitoring tools that you can refer regularly to. So it's really about making them specifically measurable that there's something attainable and relevant for your project and it's best if they could be defined over a certain period of time and you probably recognize that on through the smart methodology of objective setting which is just one of the possibilities but it's really useful when you're finding your health metrics. Just to give you an example, diversity and inclusion is important but really broad topics. So if you want to really look into that, you can define demographic or gender diversity and look into that metric or add documentation accessibility or issue label inclusivity to help you understand whether your project is welcoming to your contributors. You can apply all that metrics and many others. And we would advise you to check out health project with open source project and many metrics are defined there and you can also contribute to that project. With open source, you're not only the creator but the one that also brings the project to life and is responsible for all the different aspects. This is the head indicator as we talked about can help you balance all the parallel tasks. And especially when you don't think of just one single project but hundreds and thousands of projects and that's what the open source program offices needs to do and especially if it's in my case as I'm working with focus on project health assessment I have that great fun in balancing this aspect for a large number of projects. So just to name you some of the use cases that make it even more challenging and they challenge me to be more creative or new acquisitions, reorganizations, some business or customer demands that will require us to adapt. So we need to have a magic to make it all work and every open source program office does it differently. We had the assumption that everyone considers project health is important for managing their projects and wanted to test that assumption but by asking and around. So we asked them to do group and all the participants in their survey have confirmed that. So they also were rating pretty high all the indicators the health indicators that we have mentioned so far and you can see on the next chart that most stress is put on activity and responsiveness as well as security but all of the aspects are rated as important and the truth is that no one can provide any five-step solution we just can't share our best practices with them and this is based on our experience so what we have what we encourage everyone to do when starting to evaluate project health it's just first to shorten the list make sure to archive or sort out in any way or inactive project then you could also group them by some common criteria such as community relevance or maturity of the project so that you can choose which metrics to apply in each of the groups project then keep close contact with the maintainer so that you are aware of what's going on and when there are any changes and also be able to communicate early enough if there anything on a more strategic aspect that needs to be adapted in the project and do it in a way that it's really easy to be in constant search and receive regular feedback and provide as well feedback so use a survey or form or any other way that it would be best to communicate with your group of projects and last but not least is have an overview of what's going on and use a tooling to do that especially when it's up to hundreds of projects and since Ivana is contributing to open source 2 that we are also using it's the ogre project she can share some more details I'm going to share some insights on how ogre approaches those metrics what the project does is collecting data from the github and github API regarding before request issues and all the different aspects that we talk about anything that can be provided by the API is collected stored in databases and then metrics are calculated and visualized and that can be run against an organization or set of projects and you can check it you can run and collect that data and see for your specific needs and your specific metric requirements it's not the only available open source project that does that there is Grimorewap, Vitergia DevStats which is a CNCF project there are some open source offices prefer to use internal tools or maybe they don't hear about existing open source solutions it's up to everyone's needs and requirements my personal opinion is that it's more effective to use an open source solution because this saves time of developing your own project and in such solution various teams can put efforts so that you can grow faster but in no cases it's up to personal preference and we are going to share some more details on how we approach metrics and examples from projects we are observing yes we've chosen just some of the metrics that we are monitoring just to show you how we define them as already said we are using and here are the examples of our projects which are foundation on but we applied those for our internal project so first is to start with responsiveness we've seen that it's rated really high and it's really important so we have defined criteria which will rate the project as healthy or at risk and then we could evaluate what at risk means so in this case it's percentage of the pull request which are responded within the next two business days next one is to contribute the risk this graph it's really nice visual representation which shows what percentage of the commits are done by the key contributor so if there is any risk that would be easy to see and the third example was about the severity of the releases as we heard that it's really important also in terms of security issues so we just monitor the last six months and see how many releases are and what's their cycle of releases so to conclude it we've talked about failure first and then learning from the mistakes and what to observe and how to be able to monitor all the time it's really important to learn from that mistakes and from everyone else's mistakes because that's the opportunity that we have and how to prevent moving into the same traps first what we see is by defining clear processes for else this opens our program office for the maintainers and the project so that it's known what's expected and to be able also to adapt that process quickly. Next is to be in close contact with the maintainers and can spot ahead of time if there is any risk of abandoning a project or lacking any resources and third is to have the guidelines to communicate and to be able to prevent the lack of any really important documentation like licensing or any kind of security issues that might come up to just be proactive and to conclude it all it's not just about matrix and although we talk a lot about matrix we want to stress that you need to be aware that any matrix can lead to a certain and will lead to a certain behavior so make sure you know what you are aiming with a specific metric and don't over quantify it it's all about people community and you need to get in touch and understand deeply what what the indicators are showing and that's also what's in life it's not just about matrix it's about people and community thank you all for joining our talk and feel free to reach out to us for any questions or feedback thank you