 Good evening. Good evening. Thanks for joining. I'm just updating the attendees list on the document. Hi. Hi, Karthik. Hi. Yeah. Yeah, let's actually put in brackets here. I'm going to just say it must space my data. I can rename. I'm just renaming. Okay. I can rename. I'm just renaming the zoom so that they. Thanks for taking time. It's an important call. It's okay. Hello everyone. Hi, guys. How are you? Hired today, honestly. So I'm afraid that not necessarily a lot of people are going to show up today. Yeah. Yes. But let's still give people a chance to, not if you wanted to present something. So you might not have the biggest of audiences, but anyways, we have to recording and we can share it later on. That's still something we can do. All right. I'm posting as always the link to the document here waiting a couple more minutes then for people to join. Taylor, I really like your profile picture. Hey, thanks. I feel watched a bit when I look at it, but okay, I just posted the note stock in here. I'm adding one more item in here. So obviously we have to operate a working group update today, which is greatly making progress here. Then litmus integration services. Captain. Again, it's not too many people are going to join today. We'll just record it and then post the link to the on the mailing list later on. The recording should be available to tomorrow. And then I briefly also wanted to give you, maybe we do this bit. Then also as one of the first items, a brief update on the potato hat project. Okay. So usually we give people five minutes to join in here, joining today. So I'll be back in two minutes. Okay. So I'll be back to give the update on the operator working group today. Yeah, I will do it. Thanks. So a short update from the operator working group. And so we're meeting. Once a week at a small, very small group. Three more people and myself. And on that meeting, we are talking about the operator trying to get it. Into something we can share. We are sharing the results week by week. So in the definition of an operator, you can see. Sorry. In the operator definition working document. I will edit in the links. You can see. Progress is being made. We are open to comments there. And if you want to join the small group. And I will also share how, or you can just ping me. The last time we talk about capabilities of an operator. And we are trying to define. What are the best phases in the life cycle of an operator and how an operator can say, I'm doing that or I'm not doing that. And so we finished defining that and we will add it into the document. And so keep on watching. And that's it till next time. Sorry, it's been a long day. Thanks. I just wanted to say a broader audience that just started. There is already like, just quickly sharing it here. This is like all the new operator definition. What could you have done already? You wanted to record this meeting. It's getting automatically recorded. All right. Okay. Thanks. I think is something we already discussed. The yellow is something we discussed, but it's not in final words. So we're having a week. And we will refine it. So we talked about it. This is the output of what we talked in the meeting. And now everyone of us took a small chunk and we'll write it in the better. And explain words. Yeah, I think that's great. So the next meeting will be canceled anyway, due to QCon, QCon, obviously. So if, if you would have maybe a bit of a longer update, not the next meeting, but like the actual next one, like in a month from now, that, that, that would be great. Yeah, I believe that also in a month, we will have something more concrete to show. So maybe we'll schedule a longer. Read of the. Yeah, I think that might be that that's going to be great. Yeah, I think that's great. And notice has been in there working for, for quite some while and then we kind of lost track of it. And now we are back on it. So I think that's great. And ideally we can then also like share it with a bigger audience later this year, but that looks very, very promising already. And yeah, thanks for taking this, taking on this work. And if you want to discuss specific topics, also feel free to use the mailing list. I found that there's way more people reading the mailing list than I did you to time zone issues and other reasons. Okay, looking at the agenda. Some people were just adding something in here. Since you have conformance and new CNCF working group who added this one. That was me Taylor. Okay. So I'm just looking at also having the demo and I assume you're here to really give the team here a small update. So I was wondering whether we should just do this or the demo, how much time would you need for, for that topic? Roughly. And it's really just more of a notice and request for people to be aware and join if possible. So just a couple of minutes. Yeah, then I would propose you go first and then we dive into the demo if the demo team is fine, because that gives you more or less time for the demo till the end of the meeting and being sure that Tyler gets some time to present on this one. Okay. So I would then pass over regarding CCF conformance and. All right. I can. I guess share my screen. If that's something that. Y'all like to be. Yeah. All right. So, and they, I don't, I don't know who's aware of. But it's been, it's been going for about a year as far as. The program start. But it's been focused on. A. Test suite project up until recently. So it's. Just. Recently started was a new working group. That's the new thing here. But to give you. A little bit of background. And by the way, this may be a repeat for some people if you're on the TOC meeting yesterday, because this was shown there. So I'm not going to go through all of it. You can go review that. These slides are from the TOC meetings. But the main thing here is. The CNF conformance is, is looking at. How to help telcos and being more cloud native with their applications. The telco industry. And we're looking at the service providers themselves. So AT&T, but if an orange whoever. And the creators of the telco applications. So vendors and everyone else. I'm just going to kind of go through. Very quickly. So. This is just an overview of saying what's been happening. And. And the telco is more and more interested. The, you're starting to see more adoption of Kubernetes. And on a platform side, it seems like everyone wants to do certified Kubernetes. And so. So I'm going to go through some of the recent surveys as. Talking about. Moving the actual. Networking and telco type workloads. Which they're usually referring to like network functions. Moving those applications on the Kubernetes. And so there's. There's issues with the, the telco industry. And I think it's a little bit different from the normal applications. So that the conformance goals as a program is similar to like Kubernetes conformance for interoperability at a. Platform level. Of course, that's really core Kubernetes. Then you look at additional specs. So what it's trying to do beyond that is. What are the things. That can help the telco industry to understand best practices. So there's two parts that the existing part is this. Project for this actual test suite. And that's, you can come check it out. There's existing code. And this is being developed for. A year now. Trying to build. Test and stuff working with. Different telco community. Communities and. Getting feedback. There's a lot of, a lot of different things. Going into this. But the, the new thing here is. This working group. Going around defining like a certification. And definitions. And. This is, you can see an open pull request. If you go to the CNF conformance where it's the. The charter and. All the other things are being added here for. This new working group and what's in scope. So I'm, I'm. I don't want to take up a lot of time in this call to go over it, but the, the main. Point here is there is going to be overlap and many. Communities. CNCF and Kubernetes. And one of the main ones that we've talked about is. And one of the things that we've talked about is SIG app delivery. And. Another one would, I'm just pointing out would be SIG network. Cause there's, there's an overlap on both of those. When you look at telco and the type of applications that are run. As you have a lot that deal with like user data planes. So not using the standard networking. So what do you do for those type of applications? What are the other applications that may go beyond what. The standard. Common applications are. So that's what the focus of this group right now is. To, to work with the telco and cloud native community to, to try to look at what is the scope. What are the things that we need to do to extend and, and look at best practices on how they can be utilized in the. In the telco domain. And I should really just say that the networking. Domain. This telco would be under that. Platform is likely to be part of it in some ways. When you talk about stuff like a user data plan. And you look at other mechanisms within the. The platform or you could say layers and add-ons of Kubernetes. And that's where it ties in with. The network plumbing group and. SIG networking from. Kubernetes and as well as the CNCF SIG networking, which looks at service meshes and stuff. So this is more of. This working group is forming right now. We're going to be having community meetings to talk more at. I will start having regular. Working group meetings. And we would, we expect to. Collaborate more directly with SIG app delivery than most. Most of them and probably SIG networking as well. I'm sure there's going to be a lot of overlap in different ones. But if, if. We'd love to have your participation in that group to help telcos. I think that's probably it. Yeah. The, the, the test suite. Itself maybe just to throw out is this, this is going to be put forward as a. A sandbox project in the next three to six months. The test suite. And maybe a quick overview of the test suite is. You could think of it as. A combination of what. You could think of it as a, you could think of it as a combination of what you are and, and Kubernetes when you think Kubernetes conformance, you have the. E to E tests, which are managed by SIG testing. And then you have sauna boy, which is what people usually use to make it easier. You don't have to use some boy, but many people do. So the. What's currently called the CNF conformance test suite may be renamed. It takes a lot of ideas and concepts of sauna boy. And, and then supports doing many different types of. Test that you can do. We support. Pretty much any layers. And I think that covers it, but happy to answer any questions. Yeah. Thanks for sharing. So I definitely think that there are reasons. I think that there are points of overlap here going forward. So I think having everything makes sense. Interestingly, the first time we had a touch point with. Regarding telcos was really on the air gap work that we kind of. Well, we didn't really stop it, but it didn't get as much momentum as we thought it would. I think it doesn't really fall into CNF category. So I think that there's something always started to look in as well, especially when it comes to. Telco provider appliances and how to actually deploy applications and as well, Kubernetes there. I got something I reached out to the telecom user group, but also look keen on how this plays on the, on the networking side. I don't see the immediate overlap, but I think it might be good to at least have like a quarterly analysis and see where, where things overlap if that's, if that's fine for you. Absolutely. And I'd say the, it's more of a compliment than anything else. And potentially it's going to be a specific on the application side when versus when we get down lower, that may be like a platform or the signal working. I would say it's a specific case. And if it wasn't for the, the platform items where it starts getting a little bit gray, this probably would be a working group under SIG app delivery. There's a little bit of discussion on, does it fit more on SIG network or SIG app delivery? And I see Matt Farina, I think you said in the TOC yesterday that it kind of bridges both or agree, agree on that. But he gets for a quarterly update and we'll continue to collaborate and think of SIG app delivery kind of as a base for a lot of the cloud native application pieces. I think obviously we can think more frequently, but I would at least put like a quarter of the update on the agenda to ensure that people are aware, but obviously if you have something that comes up earlier, you can obviously talk. Yeah, I imagine it'll be less common stuff. I think we'll start saying more of that. Okay, then next item on the agenda. I see you wanted 35 minutes, we won't make it like with a full 35 minutes, you might have to speed up your demo, but it's great to see updates from two projects and especially to CSF projects working together on the chaos engineering and delivery integration between captain and litmus. So I would ask you to maybe sort of we stay in time. Can you do it in 24 minutes because that would keep us in time? Yeah, I think that should be possible for us. I'm going ahead and just share my screen. Yeah, thanks Alice. And as you already mentioned, that was really a joint effort between the litmus and the captain team. And together with Kartik from the litmus team, we are going to present our joint effort here how to integrate these two projects and actually why we're doing this and which problems we want to solve. We have a little bit of a presentation prepared and then the demo Kartik. Maybe you just let me know when I should move on with the slides and I will do, I will let you start and then I will handle. Thanks. Hi everyone. Great to see Taylor Watson and Lucina here. We've met recently and should get on soon to contributing to the CNF conference this week. Okay, so we have some items on the agenda. There's a couple of slides we have explained what we're trying to do and then go to the actual demo and try to keep it short on the slides and move to the demo quicker. We could probably go to the next slide here again. Okay, so we are all aware that resiliency on microservices is hard and whenever we are applying applications on Kubernetes, we know that there's a lot of services running there which we have not built ourselves and 90% of our resilience depends upon all the infrastructure components that we are running on top of. So it's really difficult to predict what kind of outage can happen. So it's very important to have a system where we are going ahead and injecting these failures ourselves. So Cures Engineering has been compared to vaccinations. So that's something that we are injecting failures effectively and trying to find where the system is and that information can actually help us for application, business logic or our department practices. So I think that's really important. I think we can go to the next slide. So one of the things that has a project we looked at is Kubernetes with the onset of Kubernetes I think and a little before that the way people spoke to infrastructure change, infrastructure as code came in and then Kubernetes came in and Yamens are beginning the standard. So with Kubernetes all aspects of managing applications and managing resources and policies is all done in a declarative way via Yamens and we wanted to adopt the same UX when people are attempting residency tests or chaos on their clusters. So that's where the custom resources came in. So we were able to provide the same user experience describe the chaos intent in custom resources and have an operator understand that and implement the right functions. So if we go to the next slide and litmus we have each of these activities that you see there in the block diagram. So you can see a traditional flow of the chaos experiment. We have a hypothesis that is describing some steady state for the application of infrastructure which is validated first followed by a fault injection and there's going to be a steady state verification that's done at the end of it or even during the fault as the chaos occurs in parallel and if our hypothesis is right and if the steady state conditions are met or they are regained within the toleration seconds that were specified we're going to qualify that infrastructure or application as being resilient otherwise we see the waveform to be less and the way to describe this in litmus is via custom resources so there are some off the shelf custom resources static ones called chaos experiment CRs which is what is listed on the chaos hub and this will be pulled into a cluster and the chaos engine is user facing CR that maps a custom chaos experiment resource with an application instance that's described by the namespace and labels and other attributes. So when the chaos engine is created the chaos operator launches a set of chaos parts which actually implement the experiment the chaos injection logic is implemented by them and the results are captured in another CR so they are present CRs as each of these has a scope for a lot of information to be placed inside it and acted upon so this is a iterative way of doing chaos that was the seed for litmus and that's something that's been adopted and taken quite well and it lends itself to a lot of paradigms and cloud native space like GitOps which we will talk about in a short while when kept in is being discussed so this is about why we needed it and how we are doing it the next slide is talking about what are the ways in which projects like litmus are being used today initially chaos engineering was viewed very much as an interesting very much in the domain of the SRE but there is been left shift and people are using it as part of the release that is they are using it as part of the delivery pipelines so chaos experiments with the frameworks like litmus are allowing themselves to be used in this model so you could actually do a chaos experiment as part of a CI pipeline with different popular CI frameworks with LAN etc where you can actually store the experiment before hand and run them as part of the pipeline and use the research to determine the CI stage success or the CI job success and application artifacts the two parts of the stage can be placed into other clusters which are doing some kind of scheduled chaos which are typically for longer durations and then we can interface with CD mechanisms with CD solutions which can actually take this and put it into a staging namespace run its own set of validations and promote it to other stages depending upon the success so this is something that we are seeing in fact this is one of the dominant use cases of litmus in the recent times and we are seeing the community and has this part in the interesting integrations which we will talk about that's where I would like to hand it off to you again to talk about what these integrations are about Thank you oh it was a little animation but here we go, yeah thanks so before we will jump into the demo and show you how captain and litmus chaos work together I just want to give you a brief heads up on what captain is maybe you have already heard about it it's also a CNCF Sandbox project and it's really a control plane for cloud native delivery and operations and in this sense it can orchestrate the whole application life cycle for today we will really focus on as an old driven delivery that means that captain has as a central piece coordinates based on SRE principles like SLIs and SLOs it has this as a central piece already baked into the captain platform so we have this part and we can trigger different tools different integrations like litmus and then after they do their job captain can then take its next actions for example evaluating how actually these tools have been executed and how the quality of a microservice that should be delivered to production is actually affected same as in litmus also everything in captain is declarative and captain itself comes with its own github approach so everything that's stored inside captain managed by captain is also versioned and stored in the github approach and that is true also for everything that we are doing now with the litmus experiments so everything really works or it's moved into the github or git repository and can be linked to github git lab or whatever so how captain works in the sense how can you really connect other tools the two main definition files for captain are a shipyard file that describes your environment and a uniform file that the concept of a uniform that describes which tools you want to connect to captain and captain is this control plane then connects all the different tools together so for example captain starts by deploying a new version of an artifact your container image deploying this after the deployment captain will automatically trigger the testing tool you will define which testing tool you want to use we will be using in the demo two testing tools at the same time that are both connected to captain after the tests are finished captain will execute the evaluation of the tests but captain can also be linked or other tools can be linked and connected to captain like the chat box tool you can control captain by a chat bot or you can have everything that's going on inside captain sent as a notification to slag and of course you can use these tools for observability like for me for a dashboarding with profana you can link this to captain and when you're using captain captain can distribute the events from the control plane to these tools and then they can take action so with this a lot of automation is possible and a lot of automation is baked into captain for this use case today we will only focus on the deployment testing and evaluating we will take a look at the automated configuration of dashboards or promotion to production we will only take a look at one piece of captain let's say so what we have prepared for today is really how can we include chaos engineering into a pipeline and we start with deploying the potato hat application I just saw it's also an item for today's agenda of this SIG meeting so we will hear maybe more about this it's a small demo application that we are going to use for this demo we are going to deploy this into a chaos stage so it's a single stage environment in this case because we're just interested in evaluating the resiliency of our microservice we're using help for deployment and captain is executing the deployment with help for us after the deployment is finished captain will trigger chemeter tests but not only chemeter tests but actually at the same time also trigger litmus chaos so with this we can make sure that we have actual traffic on the service under test our potato hat and we are also triggering chaos on the service so we can then later on evaluate the resiliency once the tests are finished the captain quality gate is triggered and this will reach out automatically to Prometheus and gather some data that is defined for this quality gate and will gather this data for the exact testing timeframe and for the service under test in this case our potato hat application and then captain will go ahead and evaluate based on the quality criteria of the quality gate what we have defined in the SLO .yml file we will see this also in the demo captain will go ahead and evaluate this quality gate and will come up with a total score and this score can be either let's say a thumbs up that means that the resiliency is satisfying based on our service level objectives or if we cannot reach the score our resiliency is not satisfying and we can just re-run the whole workflow again for example usually what captain can also do is we can just promote it to the next stage from pre-production to production if everything is fine I will hand over again to Karthik to explain our demo environment and then we will see the demo the demo environment is very simple we have a GKE cluster on which we have the potato hat app with hello service that is running we have a hello page that is a decent point and it is configured with a readiness pro this demo we will basically look at the difference between single replica app and multiple replica one and see how the quality gate actually fails in the first case and goes through the second case so it is highlighting the deployment issue the flow is something like this so there is going to be a deployment that is going to be managed by captain and once deployment of the hello service is complete the readiness experiments are going to be triggered on that and in that process we also have a black box exporter and Prometheus instance running from it managed by captain and we have also a black box exporter that is hitting the probe success and duration metrics which we are specifically querying for and the Prometheus SLI provider of captain is going to that is received and it is going to provide that to captain which is going to carry out an SLO evaluation against this metrics we can see that when we do the demo the rules that we have set as part of this evaluation and if like you can say we are successful in meeting the criteria that is set against the metrics that we go ahead and pass the stage and then we are going to set up what it is so this is what we have in terms of the demo I think we can go ahead with that thank you sure okay so what I have here is the user interface of captain what we call the captain's bridge I will just open this it is actually from earlier today when we did our experiments but what we are going to do is send a new deployment event to captain and I will do this with the captain's CLI but with the option that I just send the cloud event to captain the reason I do this via the cloud event and not via the built-in capabilities of the captain's CLI is that we can take a look a little bit on the details of the cloud event so I will go ahead and deploy my potato identification micro service and I will just do this with one replica and I will instruct here captain to start the deployment and what I have already told you on the slides will happen so captain will start to actually deploy the service and we will see this in the captain's bridge the captain will deploy the service and what we can already see here is that the chaos runner and the pod delete will start it so after deployment it was really fast because actually it was already running in this exact version on my cluster so the deployment was finished and captain is triggering two different kind of tests the first one is the chemeter tests they are running in a different namespace so we are not seeing the pod here we are only seeing the pod in the litmus chaos namespace but what we can see is that captain also triggered the chaos tests we can already see it's killing the pod here so our chaos tests are the pod delete chaos experiment that means a pod that from our hello service replica set the random pod will be killed by the litmus chaos experiment and it takes a couple of seconds for the next pod to come up we have added the readiness probe with a little bit of delay so we can make sure that once the pod will receive some traffic it's already up and running and now after a couple of seconds it's up and running so let us now take a look on the captain user interface what we can see here so we just triggered the chaos and we can see the configuration change was received and the deployment is finished after the deployment is finished captain will start the tests and we have already seen the test execution of the the litmus chaos experiment but actually we will wait here for the full execution of the chemeter tests usually they take about two minutes for them to be finished so we should receive the event just in a second so we can already see the tests are finished the chemeter tests with a couple of thousand requests has been executed against the service triggered by captain captain is now doing the SLI retrieval that's some captain internals let's say to reach out to Prometheus and query all the data and captain is doing an evaluation and we can see the evaluation is done we can see already here the total score so it's red, red of course means not good, we got a result of zero we would need at least 75% to be able to proceed to the next stage or for a green result and we can also see all the SLIs and SLOs that have been evaluated so we can see actually our probe success percentage and the probe duration was not satisfied the reason for this is that the pod was actually not available for a couple of seconds since the chaos experiment killed the pod there was no other pod that was able to receive the traffic and it took a couple of seconds for the pod to come up so we with this experiment we could already evaluate the resiliency of our application and what we can now do is to play a little bit with our replica set or with other settings in our experiment we will just increase the replica account that saved to three I just saved this file and I just send it again so in this case I'm sending the same instructions I want to deploy the same version but I want to have a replica count of three this time so taking a look we can already see Captain was triggering Helm to deploy two other instances of this pod once they are finished then Captain will go ahead and trigger the tests again it's always the same workflow or pipeline whatever you want to call it it's always the same first the deployment test the evaluation and then the result of the evaluation with the promotion or rollback for example of the artifact so we can see the deployment is finished and we can already see the chaos runner is started at the same time also the Chemida tests will be started the chaos runner now starts our pod division experiment and this is also triggered and orchestrated by Captain so actually Captain is starting the chaos runner and the chaos runner is then starting the pod delete pod in this one will kill a random instance of those three instances of our hello service I'm not sure which one will be killed but one of those will be removed and another one of course will come up just in a second a new one is created because this one was killed by the experiment everything is good it will take about 30 seconds for it to come up and to be ready but in this case we would assume that the two other instances they are ready and they are here to receive all the traffic that should go also to the killed instance so we would assume that our quality evaluation regarding our resiliency would be quite good this time because we have a higher replica set than we have some backup pods that will receive the traffic of one of those goes down so let us take a look in the Captain's bridge it's again it's a new instance of our configuration changed it's the same version but this time it's a higher replica set deployment is already finished so right now the tests are being executed in the background and we are not only waiting for the for the litmus experiment to finish but also for the jmeta test to finish we can see jmeta service has finished and we can see the evaluation and this time we got a score of 100 since both of our SLOs are evaluated correctly or satisfying so our success percentage is 100% there was no downtime this time although one of the pods got killed and also the probe duration was quite fast faster than this 100 milliseconds that we had set as a limit so what we can see here is basically we included chaos testing into our pipeline into our CD next two performance tests and we called it the chaos stage that's the reason why you see a chaos here it's indicated right here because it was failing but with this you can easily integrate chaos testing to your CD and make sure to evaluate also the resiliency of your microservices that's the main idea it was a short and easy demo but if there are any questions we are happy to answer the questions otherwise we just have two more slides as an outlook what will come next with this integration but we are happy to answer questions here as we have just settled the demo in one sorry we are almost on top of the timing here so if you have anything else did you want to share with do it and maybe also reach out to people on the mailing list sorry yeah we just want to share this but Uma please go ahead no I'll put the slide I wanted to ask Alois if I can take two to three more minutes to give quick updates and to go through three to four slides so what will be the next for litmus and the captain integration we will also take a look at the litmus chaos results right now we are using Prometheus as the data provider for this evaluation but also litmus is exposing a lot of data for the experiments and we are also taking a look at the litmus data and we also want to improve this work by not only including it into the CD part but actually testing self healing methods and testing auto remediation by having litmus experiments as the tests and then auto remediation scripts and auto remediation instructions orchestrated by captain as the solution for problems or chaos introduced by litmus and if you want to learn more about this in a little bit more detail we have a joint webinar next week on November 11th we will share the slides and if you want to give it a try yourself you will find all the resources here thanks so much and Umar please share your thoughts sure that was great integration and I wanted to thank with the projects they were working on working sessions for three to four weeks so that was great can I share my screen as well again yeah okay so we were this is just an extension of you know the presentation so we were chatting with Harry about you know what are the next steps for the project and he suggested that before going to QoC it would be good if you could present it into the six chairs and that will be safe so primarily we have been seeing a great momentum in the last four to five months just before and right through after the project has become a sandbox so there were some people in the community to maintain us feel that you know we are ready for integration so to state that intent and provide a quick snapshot of why we think so so the real reason where why we feel that we are good for applying for or at least showing the intent to move to incubation is the adoption and which indicates the project maturity we are 1.10 release and also there were some huge deployments in production for some time now and they have become the contributors as well so we feel that we will definitely pass the due diligence very easily and we also are happy to state that this project now has been you know managed not just by my data but great contributions coming from a lot of others but we have 4-1 maintenance also in terms of incubate and amazone and and autopilot continuous solutions have been contributing recently to the project and we have established the open governance model where we have a lot of six inside platemas and there are about 16 meetings that have happened that have still come or some of some of the contributors or chairs co-chairs inside itmas and we actually integrated with the captain of Tito Spinnaker that's one of the big ones right so for all these reasons I know the project team maintenance team thinks that we will move to the next step as a quick snapshot these are our users some of them are vendors vendor users and some of them are they qualify the end user category and definitely Intuit is one of litmus biggest users and our CNC end user and then IHE, autopilot fn networks that have been using litmus in production so in the last two months since we have presented the project here at the Lurie chairs we have added six experiments and we have done about eight community meetups and I'm very happy to say that there are 39 contributors and 16 members and our Slack has been busy as you can see that in the last two months we almost grew again about 40% in terms of the experiment terms so we do a release of every month so the community can expect that definitely there is a release coming out on the 15th and last six months a lot of development happened and we released the litmus portal which is a platform for running chaos engineering for Kubernetes so as part of that effort we integrated with our Go workflow now you can run super complex chaos workflows that are closer to your real-life scenarios in parallel, in sequence you know so it is possible to take chaos engineering to a production level and as part of that we also introduced probes where we don't tell what should be the end result or a verdict you can define what you think should be a possible failure for a given test in the hypothesis and as part of the requirement for incubation we have updated all over the CACD inside the litmus chaos development they are all open so we have a full-fledged project and also monitoring is a very important aspect so we started the architecture completed it in showing chaos interleaved dashboards for example you might have a soft shop application on any other dashboard for doper now we have chaos interleaving as well so when you run some chaos test there is a red mark that comes right on top of it and also chaos is now possible to be limited to a namescoped and you can have multiple chaos operators in the cluster so developers can run as a small application so that's a very quick update and as you can see this is from DevStats it shows about from my data IntuVate Redat, HSBC, Microsoft DINs, OptiTo, Autopilot these are the top 10 contributors since sandbox so very very happy and thrilled to say that this project has received a great adoption and they are actually contributing back and it is working in the way it is supposed to be in this new project so this is another stat that says how the folks are increasing which shows that people are taking it changing and upstreaming it back so this is a quick update on who is actually contributing these are some of the primary contributors Redat from Israel is adding infrastructure changes to Likmus so that you can run chaos from Kubernetes on a resource that's outside Kubernetes so for example a VM that is orchestrated by Overt and the container solutions which runs Likmus on some of the huge production departments they have been contributing back with the network chaos, chaos stress and open ship support and there are other Microsoft team from Japan has complicated state the AKS certification Likmus in all 33 experiments at that time so Autopilot team has been using Likmus but they are also contributing back in short so these are some of the in the interest of time I just wanted to skip this share the slides and thank you for giving me few more minutes into the agenda but this is really about how Likmus project is running things open governance model meeting 6 and integrating with the ecosystem projects and actually gaining adoption and production and that stuff so with that I really want to stop this presentation and I wanted to thank for being this opportunity Thank you for the update I think it's great to see that project moving forward and obviously that's what we want to see is moving projects from sandbox to incubation I assume you officially created a PR for a TUC because that's the official process how these get routed thanks for the heads up and let's work on this and it's great to see the project working forward and it's great to see you collaborating closely with other CNCF projects as well and I think with this we conclude for today I wish everybody a nice rest of the day, nice evening depending on your time zone and talk to you again in 4 weeks not in 2 weeks so next 2 weeks we have CubeCon and we decided not to have a meeting during CubeCon, thanks everyone bye good job again bye thanks thanks everyone