 One my name is will go until gay and today we're going to talk about a package quality assurance. I have actually a lot to say so It is high chance that we will not be able to ask any questions and so fasten your seatbelts We are any so I'm from Russia from city called you could remember which is placed near Ural mountains and also we have Monument about border between Europe and Asia and I live with my wife and three dogs that we are adopted and one of them really miss me home and I'm working in a company called evil Martians. We are mostly focused on driving your startup at warp speed and the only way we could achieve such results is open source contributions and Open way of work, which we call called the Martians. So it's a problem definition As I said today We built business Software using a lot of open source dependencies and let me show you some statistics about Ruby open source. So here is the data about sound year and It's the usage of Ruby gems. So the first line here is the total usage of Open source Ruby packages. We have here a community of 90,000 contributors and around 100,000 projects and It's pretty okay, but if we all remove Only one person tile here Which I mean the one percent of the most highly adapted project We'll see that actually a number of downloads drops and While the number of project changed only by 1,000 and number of contributors also drop drastically and Actually the problem reveals when they see the last line where we have still hundreds millions of downloads of packages while we have more than 100,000 of project and Number of contributors is much much less. So we have less than half of a developer per project and So what's the problem? Of course we have high demand a lack of participants and Especially in the Field of not highly adapted projects. So high risk of enterprise failures and of course high pressure over maintainers Also, I want you to compare that we have only less than half of maintainer per Not highly adopted project versus 20 contributors for highly adapted projects Yeah, a couple words why I'm doing such research a while ago I created a project called assert its open source and It helps you whether you decide to add or remove some open source dependency. So let me show you how it works so this is the main page and We see here a score about maintenance about popularity of the project and maturity is just a combination of those groups We have a matrix for each group and each metric has the score How the scoring system works pretty simple. So I've just gathered information about every open source project developed in ruby measure the Metrics and divided values in five groups a to f and then I'll be able to find the distance between current project values and Some of the group the name of the group actually the score. So it works for the metric and for a group of metrics and problem actually with the author not the ranking system, but the names of the grades and marks and the colors so actually When I created it the idea was to show amount of risk So when you use highly adapted software and the score is high There is a less chance of failure in production and also the higher chance that fix will be introduced soon On the other hand for low score. We have the opposite But the problem here is that Such system encourage people to more focus on an already adopted software which Which was proved and which also has high quality at the same time the other project and There is a huge amount of them needs more support needs Contributions, but people will not use them because their score is low And it's unclear why some developer could choose a project with low score so that's why today will focus on the What I call mid-sized projects or the project that were not highly adopted because for them Quality is the main issue and also the lack of resources So we need a way to optimize their process to achieve higher quality so What what is the way to solve the problem at first we need to understand what quality is and how we could measure it Then we need some kind of risk risks model Which will show us how we could Actually action in this field so when we know risks and we know some measurements we could make decision about How to behave in the process so we could improve and user experience so when I started I found existing research of course and Hear my complaints about it most of the papers were outdated another another were too specific and The that doesn't mean that they were not good, but they were not not applicable But one thing worried me the most is the focus on similarities with Operator software so those papers tried to apply techniques from appropriate software to the open source And let me show you why it will not work especially to a mid-sized projects So let's say we have Some a business application. We have a customer which pays us money which create obligations requirements But at the same time gives us resources so our customer is the source of requirements We could gather gather them and then try to spend some of the resources to To just check that our software meets the requirements, so it's pretty simple, but if we try to Do the same stuff the the same way in the open source we'll see that no money no obligations and no obvious requirements because Who are the customers if you don't know who's in who's in stack so The main problem also here lack of resources how we could add here some QE resources if we don't have resources for development Remember half of developer per project so Many it's another possible solution and I decided to go to the Region of the quality assurance to the manufacturing process. There are several techniques that could help This could help us and they have manufacturers to Build better product. So the six sigma techniques tell tells us that improvement of product quality is a function of improvement production process and also Six sigma tells us which step we should take looks like a framework. We have First step to define measurable quality Then or measurable goal, but then we should actually measure the metrics analyze them and finally when we have data we could make some improvements and Of course after any each improvement we should control that action were successful so another thing is the lean process as the evolution of the six sigma and It's also added some important fact that you should not only focus on the process abroad, but you should focus on reducing resource wastage in Manufacturing process it was about time. So they wanted to spend less time on product Production so they succeeded in that. So let's try to apply it But before oh, yeah, this Moment a lesson about risk once I've turned home and found that my dog is near ruin sofa and I was surprised because she never behaved that way and I Made the research. I found that under the sofa. There was all toy and also remembered that so many times I saw my dog sniffing around and That was a lesson about RISC when you know your risks you could control them and also You could be in the same on the safe side so That is why we should do risks analysis and stay on the safe side and achieve better quality. So What do we need we need to build efficient process measurable quality definition? and Analyze risks and of course waste optimization. So let's take a look under the hood of the mid-size project Here is the statistics about Again top Ruby open source I gathered it using the other and on the right side you could see downloads or install average installs per project and we see that growth is exponential between the level of adoption. So here is the Person buckets based on the person tile. So here we have 1000 project here is five thousand project and so on as we have More project and less adoption in them so on the left side for the same buckets we have Critical metrics about community behavior and we see that behavior is not the same as the growth of the Adoption so there is another law about those key metrics and It's almost called hype cycle and The hype cycle describes us how any new technology adopt Becomes adapted and the highly adopted open source of clear Feeds right there on the plot of productivity But the mid-size projects are spread around other part where we see different Crisis is in in the flow so this chart helps us to understand that To find the optimization or to find the way to improve the quality we need to split the solution between several parts we need to Choose one of them and then apply the framework from manufacturing and That will give us a whole solution if we'll sum it up. So let's start the trigger stage is the Last part of the chart one you just invented something and those projects is pretty hard to discover so quality for trigger stage is Pretty high in terms of manufacturing process because you just create a library for yourself. And of course you have Customer that is actually developers. So Also, we need me need no optimization here but When we when the technology and became more popular we have here some risks and we need to Get some popularization here because actually if you want to verify that our project was Make any sense we need to have enough number of users who will say that yeah, I will use it. So Let's say we have here for the project 100 customers for users and still one developer. So the quality could be calculated here as the function of customers growth then the Again number of defects or bug reports and in overall we'll see the Kind of quality of these of the project on that stage So let's try to understand the risks here. So key risks here is lack of interest and Ability to improve the work So of course on that stage we need to focus on cultivating interest So we need to write some articles or a post or at least sharing it with friends and with others and the other thing is The feedback especially a negative feedback we should Work on but also we could here optimize our resources We could define initial requirements and just close for example a half of the bug reports because they Do not meet the initial requirements. So we've got some optimization and if we fix the bugs Fortunately we have Some growth and we have a stable number of users. So we have here some modest adoption We are ready for optimization because we close many bags till now and we somehow prove the solution for a particular group of users So let's try to understand the quality on that stage here we have more number of customers we still have around one developer and For now we are ready to focus on feature requests as I suggest previously we were mostly focused on bug fixes that means that we have a lot of open feature requests and That is why we should focus on source quality and sometimes when you know the amount of the adoption growth and you also need to focus on solution optimization. So Let's try to understand the risks the risk here here is Sustainability because when you have 1,000 users you actually could not just You couldn't just leave the project but You could help other maintainers and contributors If you will decide actually to leave so I suggest to write sort of documentation to cover this as a risk and the second thing make it right or introduce here more features and also as I said I prove the source quality by known techniques and optimization here is also available. It's the It's available in case of overgrown scope when you don't know which of the features you could choose You could define strict feature requirements. So finally if you you have closed enough feature requests and made all these Actions. So hopefully you'll be on productivity stage and you're very close to the highly adapted open source software. So somehow we've got to the Point where we could understand how to improve the quality of the mid-size project on each stage Also the idea that I expressed right now pretty briefly I expressed in the document called maturity model and I'll show and I'll share them in the end and You could try to discover maybe something interesting for you. So Let's try to define the quality overall In measurable way. So measurable way is to say that we could measure the community or adoption. We could measure Docs Implementation and support. So finally we could somehow create a tool that will show us Some clue about What quality of the project is especially as we know that amount of research around the documentation checks implementation Quality checks is pretty huge But I want you to focus on another moment. It is the idea that we could simplify our Function if you'll say that actually implementation documentation support. This is all the community but not the growth of community It's the community behavior That leads us to a hypothesis that quality is just a function over the community behavior But let's try to understand What's our community consist of? Community is actually the spectators who just discuss the technology but never by they never Use it and they maybe just share some Posts in Twitter or stuff like that and users are actual people who at least try to install your software bug reporters and Contributors and maintainers is obvious and I mark them greatly spectators and end users because actually we could not calculate exact number of those people and why it's Important because overall is the whole size of community and if you don't know the size of community Community we could not have any clue about the conversion from the user to the Participate to the participant in the open source community But actually we have a clue and here is the table about Boundler conversion and boundary just one of the packages in ruby And here is the data about 16th and 17th year So we see here Some revenue in the number of downloads per year and also we see revenue about participants So we see that the changes are different. So Participants were growth were was less three times But actually when you have 1,000 participants in your project is It's not a problem, but let's take a look on another project. It also has high adoption So we have during the year almost one million downloads only for that one project the revenue well in downloads per year is Not not big, but the number of participants in the project decreased and Actually, if we have here a trend, maybe it's time to some warning or at least some mention for the maintainers to take some actions But it's not the case. The idea here is that actually we could measure our quality and we could And if we could measure it, maybe we could even improve So here I want to introduce you the multi-agent systems. The multi-agent systems are semi-autonomous decision-makers Which cooperate to optimize the manufacturing process? Why I mentioned them here because at first they're again about manufacturing process and the second thing is that because the behavior in open source is is Very similar to a behavior in multi-agent system because again where everyone makes decision and Sometimes we try to cooperate to optimize our process but the difference here is that Agent in the system has some prescription How it should behave a while in the open source. We don't have any restriction or rules instead of one that in open source we could leave the project whenever I like and That means that Maybe it's time to introduce some general prescription for the open source at all to make it more obvious for any participant even for a spectator or any user how it could affect the overall community behavior So I introduce you all the open source and to this playbook or My interpretation of smart open source agent behavior, so the idea here is that You could kind of play in the open source. You You have a framework how you should behave In the open source field, so you should start from the problem. Let's say I want to know something about blockchain Because it's popular, but I don't have intention to create here something or an event I just want to discover what blockchain is so I go to the empathy axis and I'm here around the curiosity Part and that means that The role that I will choose preferably is the spectator or a user Which gives me which could give me the better experience? So the second axis helped me to understand the focus group of the project, so you see here the names of the maturity stages that we passed and I just want to need to answer the question what I'm interested in in the innovation in some field or on Efficiency innovation some scaling or stuff like that. So when I made a decision on the crossing I'll get some for me. It is those blocks It's the most interesting for me types of projects and Then I could make a research find the existing projects and choose the action So for each role I have set of actions that They'll help you or guide me through the open source interaction and for example for spectator it's very important for sharing your Impressions over the for example reading the documentation or Reading of some article that was mentioned in your read me and stuff like that For example for the high stage another interesting topic is the problem demand and the and the overall interest in the predictions of Future in this field and so on for any other Role, so it will guide you whether Whether you decide to contribute or just to know something how to have fun how to Learn something and at the same time how to help open source community just as a side product another interesting thing here Is the participants conversion so using the same? Chart we could find the way to optimize resources of maintainers as well so we know that Which risks we have on each stage of the maturity? Due to hype cycle so we understand which users are most wanted group of participants and We need to focus our efforts to attract them to your project. So for example on trigger stage for some blockchain project I will prefer to Write some fancy read me add here some maybe movies or images and it will Help me to go to to get to the hype stage on the hype stage I should also introduce maybe some how-to's or different articles that shows how simple it's to install and to try the particular solution and and so on and while we focus on the particular group and we grow the more harmonious solution more harmonious community which also will help us to in more efficient way to go to the Group of highly adopted open source so let's summarize quality is the function over the community and presentation dogs and support and Likely we'll soon see a tool that will help us to measure the quality the second is that we could try to measure the community using some Techniques maybe one that I've just introduced and also Introduce solutions for the problems that we have in the beginning the risk analysis for risk analysis we have maturity model that is that I'll share in the end and as actionable model assistance to make decision which for example project to choose and why for example I could choose Project with a low score on other is the open source enthusiast playbook and final thing is I want to give a new life to the Author to if you even if you don't work with Ruby I'll ask you to go to the page take a look and you find if you'll find interesting the calculations or representations just share me with me your impressions share me with me your in issue an issue and I'll try to implement a sort in other on other platforms or for other languages Thanks. This is the open source quality GitHub. I owe and treat me So thank you very much That's why I've created the maturity model and I have documentation that I need to even describe more because it shows the pitfalls like that that you describe and And it's actually solved during the inside the maturity model You could just read through it and and that's it. I'm sorry my time is up But if you have questions just with me and I answer you