 Hello, I think this is probably going to be one of the most heavily attended to see meetings, hopefully. All right. Welcome to the March 21st meeting we're going to pause for a bit and let everybody come on in. Morning books. Morning. Greetings, we're given a few more minutes to be able to have everybody come on in. Hello, we are holding to be able to go everybody dropping in. So, one more minute or so. Nice to see so many faces. All right, and at three minutes past the hour and with roughly 30 people on board, we can go ahead and get started. So Emily passing to you. All right. Thanks, everyone. Quick reminder about the Linux foundations anti trust policy notice you're here. We're happy to have you meeting logistics are available in the to see repo. We have a fair amount of to see members today here, as well as some tag chairs and tag members as well as a lot of community members that are interested in some proposed changes and items for discussion around milestones as well as changes to the graduation criteria. So, I'm going to try to structure this into two sections to make sure that we have ample time to get through everything if I'm going to ask everyone to use the raise hand feature and to try to help me out and email and others and making sure that everyone has an equal opportunity to be heard. I'm respectful of everyone's time and questions. If we don't get through everything here. There's always issues that you can comment on and provide feedback on we're trying to reach a collaborative consensus and a lot of the discussion. So first up is a discussion around the cloud need of milestones. Issue number 961 on the TOC repo concerning around the sustainability of sandbox projects and kind of driving a little bit more maturity with the existing projects within the foundation that are seeking to move levels. The concept here being not to introduce an exorbitant amount of new targets for projects delete reach rather providing other successful measures that other projects have adopted or hit or showcased as guiding points or potential opportunities that other projects could look to and leverage as they grow in maturity things that we've seen other successful projects that have graduated or move between levels such as whether or not they have a healthy amount of early adopters for the project. Whether or not they engaged early with a particular tag to receive good feedback build better governance. More clarity on how to contribute to a project things of that nature so things that have worked well for projects moving levels that have already graduated that are seeing a higher level of maturity. There is a corresponding PR. It is number 997, which has an initial starting point we had a lot of good active discussion on both the PR and as well as some of the comments that we received. There was good constructive feedback on where we can make changes there's a few observations from community members about whether or not we should provide specific numbers as ideas for projects to target, or should we make it a little bit more generic because different projects have different sizes different maintainers different scopes. So setting something like having five contributors or five maintainers on it may not translate well to everybody so how do you provide that level of guidance if projects are interested in pursuing that. Liz had some excellent feedback on how we can make this more informative and less checklisty, because there seems to be still some confusion or lack of clarity that these are not criteria. That's a separate conversation will be having later today. So I wanted to open the floor to feedback suggestions. For those that have been following along on the issue if you have more comments that you want to provide publicly or open to discussion and solicit feedback. Now it's the time. So I'm afraid I'm only going to be available on this call for the next 23 minutes sadly but I wanted to just join and make a few points. I think that you really need to consider a few things number one is should all projects in the CNCF be subject to the same set of criteria. When is that useful. My view is on the whole that it's not useful at all, except in a small number of cases because projects are so different and very so widely. I do appreciate that there is a strong desire to have standards, and you know, standards especially around sort of security sustainability and trust really important topics. But I believe there are a number of assumptions people are making about these things which I'm not sure entirely warranted and it's the really important thing with a long lasting foundation is to have a wide range of different kinds of projects that can flourish under a common umbrella and not assume the certain pattern for one type of project is the right one to copy for all types of projects. I believe that, you know, we have many extreme cases Kubernetes is a huge project. I don't know these days whether whether it's bigger than it's probably not but it's, it's probably on a set similar order of you know cyclotomic complexity of nothing else. And there are other projects that are really flourishing with a very small number of maintenance and contributors. They're one or two person projects and they do very well. And in the past, I am was privileged to run a team that included the Tomcat team as I discovered when we was only one person maintaining Tomcat for years and years and years and years and years. And it was a great surprise everybody when we realized this is the case that the entire industry at the time instead of 2014 or so was using Java and serve it serve let's J2EE. And Tomcat maintainer and he worked at VMware in the Pivotal region spring source. And that was okay although it does, you know, lead to those XKCD cartoons where you've got the entire industry on a huge stack and then there's one tiny little domino holding it up as a sort of pivot underneath but that I appreciate people have concerns there. I know there's also a discussion around sustainability for those kinds of low level infrastructure where there is a common funding model across foundations. I believe this has been discussed with open SSH for example, sorry open SSL that you have maintainers from different sponsors put money into a pool, and then they found the salary of somebody who works for the foundation I think that model can work okay too. But it's quite difficult to do it across the whole industry. And it's very hard to do it with innovation. And what I particularly worry about with our innovative side is I think we've lost our way in terms of the original vision of encouraging innovation from small companies. And I think that CNCF partly because it's trying to cater to a lot of different audiences, the wider community, big end users, big vendors responses as well. Sometimes these are sites of the difficulty of managing a very small company and running in a running with the big people. And what I see is a lot of people either crumbling under the inability to maintain the demands and CNCF imposes on it, which are often feel much higher to the maintainers and lead to serious burnout problems, then they might seem to people sitting behind a desk. And I'm not sure if people are fully understanding how sustainability can take many many different forms in an open source project. There is an assumption that for example, you know, every project wants to have multiple maintainers from multiple companies. But I've seen many cases where there are large companies that employ somebody who works on a project who then moves on. It's just not particularly reliable. There's also cases where large companies don't want to hire their own maintainers that want to have a single or a couple of vendors. They can work with and sponsor maintainers through that company, which is the case with NAT, the NATs project, for instance. And I just feel that sometimes I see discussions where all of those concepts are simply ignored. And then I look at the demands placed on the four people who are open source maintainers working on this stuff. And I just see them crumbling under the weight of all the things they have to do. And, you know, being stretched beyond comprehension by different committees, expecting them to do stuff or add stuff to the roadmap or pass the security test or go through this administrative process. It really kills people and it kills the morale and it kills their productivity. And I'm really worried about that. Okay. I appreciate that there's a lot of good will behind this stuff. But please, please, please consider this as you go through this process is my main request. And don't assume that just because you've say got, you know, five maintainers from different companies, you've automatically got something that was sustainable because you don't, because if you're then burdening them to work for free. And it's very hard to, you know, to make money. Those people will not be able to sustain that sustain that commitment to a project for a long time. Okay. So I'll stop talking there. And, and, and shut up because I've said my piece. Thank you, Alexis. So I want to just verify and like re summarize what I believe I heard you say we should be can make, we should be very clear on which things are criteria for projects. And I'm hopeful well that it's not going to be the same for everyone and I think Liz did an excellent job previously identifying like this is one of the reasons why the TOC had is initially set that criteria because there's so much variance between projects and evaluating their maturity and that's pretty reasonably documented. So that should be a strong consideration moving forward. Also, making sure that there is space given the different constructs for engaging and supporting maintainers of different quantities for a given project. There's a lot of different models out there with large scale organizations and small scale organizations for providing appropriate support that also the administration burden on a lot of some of the open source projects can be leading to a significant amount amongst maintainers so we need to be considered of what that actually looks like and how much additional work we may be imposing on those maintainers that are working to innovate. You also said something that was very intriguing around the discussion for sustainability of low level infrastructure for projects and that some of those considerations will look probably very different than something that's more user interactive. I miss any part of the concerns that you raised before I get handed over to Josh and then Liz. I think so I mean I just think that. Let me put this another way. When I was the TOC chair I tried to convince people it was a good idea to do the projects and the CNCF and we had some early successes. But now I just see if I was starting a company and I wanted to be an open source company, I wouldn't put my project in the foundation. It's just too much work and everybody tries really really hard to stop you making money out of it. But it's much better to do a curated open source product outside like Grafana or GitLab and you know just ignore the foundation and that means the foundation is on track to be a home for second class projects. That's really worrying to me. Thank you Alexis. Alright, Josh. Hey there. I'm coming speaking here from the tag contributor strategy perspective. We work with a lot of projects that are effectively for some of the people involved their first public open source project. And so I'm really actually interested in developing this milestones proposal as sort of a track of, you know, what should you be okay you've been accepted a sandbox what should you be doing to get to incubate it. And okay you made it to incubating what should you be doing to get to graduate it. Because currently the way things tend to work is projects put off a lot of things around project organization until they're applying for the next level in CNCF. And then they try to put things together in time for their DD. Which, you know, if you're redoing the project governance doesn't end up working very well it ends up being a huge delay for them. So, the idea that we'd have, you know, sort of this path like we have for users right with the cloud native maturity path that we have for users to have the sort of path of project maturity with the understanding that there's a lot of different sort of parallel paths that you could take. I think would actually be very helpful to a lot of projects. And, and so, you know, we do a lot of fine tuning of the individual sort of guideposts etc. I actually kind of wonder if guidepost might be a better name than milestones to make people understand what the intent of this is. I think it would be really helpful for a lot of a lot of our folks and a lot of the maintainers and a lot of the sponsors that we've worked with through Type CS. Thanks, Josh. Anything else for you. No, that's it. Okay. Awesome. So I've got that Liz and then Aaron. I just wanted to reiterate what I put into the into that cloud native milestones issue and I think what Josh just said about maybe calling it something like guideposts. The terminology that we use for this is really important and I really, I think there's some amazing ideas in here about how we can advise projects, you know, what kind of activities might work what kind of things could help them attract more volunteers and I think if there is a number one concern that projects probably want to address it's how do I get more active contributions into my project or active contributors. So there's some great guidance there some great help that the CNCF can offer to projects, but I really think we need to decouple that entirely from any mention of the maturity levels, even as somebody says this kind this is a sort of thing you should be doing as an incubating project or this is a sort of thing you should be doing as a graduated project. People will start to interpret that, even if it's explicitly stated that this isn't a criteria. They will interpret it as well it's not really a criteria but it's going to count against me if I don't do it right because that's the expectation. But even, however, I think we need to be incredibly clear that they are not this this advice is not tied to those maturity levels so that people don't misinterpret it and they see it what it should be which is supportive advice to help projects grow and and get through whatever problems they might have. That was it. Thank you so much Liz Aaron. So, and I think the counterpoint to that was which we've talked about in the past is though it can, it cannot be a checklist either that assumes. Once I have done all these things my project will be accepted I do think that as we grow in scale. There is power in not accepting everything and in while I welcome, you know, projects to grow and give visibility and, you know, Alexis to your point around, you know, giving a stage for those small projects to have an opportunity to do that it has to be. It has to be some sort of guideposts or something that we're able to attune to because we're, we're a few people with now hundreds, if not even more projects on the landscape and if we want the foundation to continue to be useful and viable which I was a little worried. Alexis when you said if you if you were new project you wouldn't go this way that we need to stop and make sure that what we feel like we're offering is the right path and I do feel like this I'm 100% behind that statement, Aaron, I would somebody came to me today and said I want to be, you know, the next confluent Kafka whatever to use the CNCF I would say, do not do that. Absolutely no way. So, saying that now, then we need to stop and refocus and say what is our value to this community, how are we best serving all the people and all the projects within it in a way that fosters this innovation that bring small projects that allows them to grow because I think if we've lost that we've kind of lost our north star. I mean, when I remember when I sat down with Craig McClucky and tried to sort of sketch out what CNCF could look like before it was so launched off the back of the lessons of cloud foundation we talked a lot about this. And the sort of the stakeholders that we envisaged were, we looked at other foundations and what they've done right and wrong. And the stakeholders we envisaged were absolutely. You know, all the classic big vendors super important for tons of various reasons. And the best thing we could do is think of ways of getting all supported at the same time, which I think will achieve. We also thought cloud family had shown us that end users were a fantastic source of innovation and support. And I know that there's been a big effort, especially by Priyanka and Chris in the last few years to encourage end users. But some folks have taken on, you know, VP of end user roles, which has been really important. What I would say about that is thumbs up but be aware that after a while an end user company cannot be the sole sponsor of a project it needs to spin out and we've seen this countless times. The for me the vanilla example is Kafka coming out of LinkedIn. It took four years to benefit from being an anchor project inside LinkedIn and then it became conflict the company and there are other other folks involved in the Kafka ecosystem as well but I think that's a great pathway. And then the other two classes of stakeholder were individual contributors in the community. Again, I think CNCF has done a pretty good job here with spin some ups and downs over the years with events and marketing versus community of course, I think it's a it's a triumph actually overall. And finally, small innovative companies and they've got to have an incentive to be involved. And if they if they can't succeed, they will go and do something else. And I think that if you take away that group of folks whether they are starting from scratch or spun out of LinkedIn, you won't have a successful foundation anymore. That's my word. Do you have anything else. No, I prefer to yield the time to everyone else to have their thoughts presented. Thank you. Doug, Josh, and then Matt. Alexis, I was wondering if you can elaborate a little on why you would not recommend coming to the CNCF. I have my own ideas of why that may be about like to actually hear it firsthand from you, because in my experience for the project I work on. If I was to remove the bureaucracy, shall we say of the whole graduation criteria process and all that stuff actually didn't care about that and didn't think about that part of the CNCF. For the most part in my project CNCF has no impact on us whatsoever. We do whatever we think is best for the project. And so I'm curious what you're seeing. You work for a large company, Doug. Well, yeah, but I don't think that influences the projects I'm working on. That is exactly my point is the CNCF serves the interests of companies like IBM quite well actually and I think those interests are totally legitimate. I want to see IBM, Red Hat, Microsoft be seriously excited about the CNCF and I can, you know, regain you with stories about the different things we learned as we grew up from a few companies to getting folks like Microsoft and Backcube and Eddie's enjoy. That was a very interesting times. Great. Can you elaborate a little because I'm trying to wrap my head around. Yeah, I mean, let me give you Doug an artificial example because I don't want to get too into the details. One of the things that Apache Software Foundation does, and was one of the reasons that we didn't choose the Apache Software Foundation to put all of this into a Linux instead, is if you have a commercial offering, whether it's professional services or an enterprise product, and you use the word Tom Pat in it, you immediately will receive cease and desist let's assume the Apache legal team. And that makes it almost impossible to make any money in the in the market around Tom Kat. So there isn't a market around Tom Kat. Instead, it becomes a tool that you can only a component that may or may not be part of other offerings. And that means that the investment you have to make in order to get off the ground commercially includes have a successful upstream project in the foundation and have something that is outside of that is totally separate and builds on it. And probably that has to be open source, which means you also need to have an enterprise version as well. So you need to go from zero to Prometheus Grafana and Grafana Enterprise on your own which is basically impossible. It's much easier to start outside and have a curated offering. And then if you become successful, maybe tear off a piece and have a shared version that's upstream and other people can build around. But that is, you know, if you become successful why do you bother doing that. Got it. Okay, that helps me so much. Thank you very much. Josh and then Matt. Okay, I want to reality check here because there's been a lot of discussion about why you wouldn't join the CNCF. But I can't recall the last time we emptied the sandbox application queue. But the fact is, we do have projects applying to join the CNCF more than we can logistically handle. That's actually one of our major problems is that more projects want to join, then we can in good conscious, except because we can only support new projects so fast. And yes that includes a lot of projects that belong to big companies but it also includes a lot of projects that belong to small companies. So clearly there's a lot of people who think that they are getting something for themselves and their company and their project to join CNCF. And for that matter I would say, we don't actually want every project for every company to join CNCF. There are a variety of different ways to build up your project and your business model. One of those that has been popular, honestly, since the early odds is what we call open core, for example. And yeah, there's, you know, that's going to be one of the strategies pursued particularly by startups with open source products. But we wouldn't want an open core project to be part of the CNCF. You know, it's a legit strategy but doesn't work together for being part of a foundation. But Istio is part of the CNCF and tetrate enterprises is effectively open core Istio and sorry, Leeds, but I think it's fair to say that by surveillance enterprises open core Cillian by any reasonable definition but I think where you were going Matt was projects that are deliberately held back by a single vendor in order to force people to buy the commercial edition. And I believe we looked at this at a great length and we talked about steering committees and came up with a number of pretty good strategies for mitigating that issue. But anyway it's a good point I mean people nobody wants to have feature hold back by a single vendor and being held hostage in some way that's really important. Yeah, so I'm just saying, not everybody's going to join and not everybody should join, but clearly the CNCF is providing value for a lot of organizations and companies that are interested in joining the CNCF the. So, you know, if, if we have major challenges it's actually more in how do we scale what we're already providing those projects. Thank you, Josh. Thanks, Alexis, Matt. Thanks. So, I have kind of multiple hats here, right. So the first hat I want to put on is as a maintainer of a project, right somebody who's done day in and day out work on an open source project, you know, inside of different foundations over the years, outside of foundations. And when I first got into doing things with the CNCF and it started. It started providing services, I'll be open and I'll say, I could see a lot of benefit coming along with it right. You got things like third party security reviews. And so it wasn't just a home to collaboratively work on the source whether the company is in a vendor neutral place, there were benefits to it and I think a lot of those still exist. But I also think that now we're seeing more things that make it more difficult for those developers expectations to show up for committees and to be involved in things that don't help them drive their own project. And so some of those benefits in the past now have other things that are stacking up against it. And I have heard frustration from developers who talk about the expectations and where they're being pulled in what they want to work on with the companies want them to work on. And so there's something there as a developer where things are not as great as they once were, for various reasons and I think that's worth talking about. And then, you know, Alexis has talked a lot about the company side of it and in some ways he's right. Things have definitely changed there's projects we probably need to say no to and we need to communicate more no to. And there are things we can definitely improve because he's not the only one who I've heard say, we wouldn't put it in the CNCF. And I've heard various reasons from people, and I haven't tried to collect them, but I've heard this. I don't know whether it's always been there and I hear it more now. Josh said there's lots of projects clamoring to get in. Is this something we should be concerned at. But it is one of those things that I think it's worth spending time looking into where are the different roles involved right because then you've got end users right end users say oh this thing, it's single vendor. Well what happens that vendor what happens it's a venture capital funded company. To be honest, what what happens this company do I my big enterprise want to rely on one single VC driving something, you know VC funded company, or do I want to have multi vendor so I feel that that project I'm going to pick up has more longevity to it, because you have very more points of failure with respect I have to drop up but I just want to comment on that last thing. If you have five people from five big companies backing a project it does provides absolutely no guarantee of sustainability. If that project is incapable of generating value for those companies that are backing it. You're right. If you have a single vendor company, which is doing well and can get a return from the work that people are putting in, whether or not VCs are involved, then it is providing value to its end users and to people building it and that is much more sustainable and can be much more sustainable than something which seems to have lots of maintenance but actually something goes wrong and those people will fall. And with all the due respect to the folks in big companies, we've just seen serious rounds of big tech playoffs. Big tech companies could get bored of this stuff and stop backing it quite so well and then suddenly where are we you know it's it's it's not obvious to me that having this for more than one company is a guarantee of sustainability. Anyway, with that I must drop off and I'm really sorry to interrupt. Alexis, I agree with you on that. So I agree with the point. Yes, have a good day. Thank you. Bye bye. Sorry. And I have to agree with Alexis on it. It doesn't mean it's value. I'm just saying there's lots of users who make that point of that's what they desire. And so there's a lot of complexity here. Now, we do have projects coming in and where's that value statement compared to where it used to be to where it was at. I think that's worth digging into, right. If I put in my different hats to see maintainer, somebody who works at a company and of course we need to derive value from it in some way. How does all of this work. It might be worth digging into again. So I'm going to pause as part of the discussion because we've shifted topics a little bit, but I do think that it was worthwhile. So I have a list of action suggestions and changes associated with formerly called milestone sounds like guiding post is a better suggested change. So I'm going to take this I'm going to consolidate some of that feedback I'm going to comment on the PR and execute some changes. So for those that are interested, please check out the PR for updates within the next two weeks because I am busy. I will try to get that done. I do also feel like there is a good and worthwhile conversation about the value or perceived value in projects within CNCF and how we're engaging with them how we're setting up maintainers and contributors for success while still providing value alignment to those organizations that are interested in both donating and adopting those projects. So I think that's going to be a good separate conversation for a later date. Just don't have the structure for how we're going to pursue that at this point in time, but I will take a note for the TOC to pull that back and figure out how we can move forward with that discussion. The next step that we have is around criteria for graduation and potentially moving levels just generally moving from sandbox incubation incubation to graduation. We've heard a lot of feedback initially around some of the confusing criteria that we've had or some of the differences in requests for projects moving levels whether or not it's redoing an entire due diligence or if it's just a refresh. So I want to kind of initially set a little bit of expectations around some things that we are already aware of the TOC repo does have a process for proposals. It was recently updated slightly last year with some changes to the FAQ to include better clarity on what adopters are. And also to include the concept of a due diligence refresh at graduation because sometimes projects come in and it's been many, many, many years since their due diligence was done. So obviously doing a refresh there, not many folks may be aware of that because it's not something we've actively discussed and there's only one line in the repo associated with it. So that being said, there are about three open issues related to moving levels and criteria changes. There's been a fair amount of discussion on both of them for things like clarity of a security audit versus a review and assessment during incubation or graduation other expression of potentially security measures or assurances that projects are looking to provide their adopters, updating a standardized template, as well as potentially timeboxing some of those states or moving projects out of the leveling framework, but still allowing them to flourish. So now I'm going to yield the floor to Liz who has her hand up first. Thanks Emily. Yeah, I just wanted to make another point about the time frame. So I recall when we first talked about trying to stop projects from kind of rushing up three weeks before KubeCon and going, hey, we want to be incubating by KubeCon. Who can you do that? And that was definitely a problem. So there was this idea of instituting a period of like a sort of moratorium period. That has now become, as I understand it, a six week period during which no projects can progress at all. And that means that for three months of the year, we're basically holding projects back, you know twice a year, six weeks, saying projects can't make any progress. I just wanted to flag that as potentially I totally think there needs to be some kind of limit and stop people from going crazy and it shouldn't all be about getting things done by KubeCon. And so Cilium has fallen into this box. I'm much less bothered about whether or not we make it by KubeCon other than that, if we do we want to, you know, celebrate it in somewhere and have time to prepare. I'm much less bothered about that than I am about the fact that it's going to be, you know, six months between the initial application and getting to the point where people can even make public comment. I'm just putting it out there that maybe pausing those that whole process for 12 weeks total in a year, maybe overkill on that particular problem. Yep. Heard and I know that we've we've talked about that before and I think it's worth revisiting for sure, given now that we've tried it a few times we've gotten good feedback from projects such as Cilium and others around this as well as to see members that are in the position of communicating this moratorium with projects. So I think that's definitely something that we can work on adjusting to be more favorable and taking a good, making good use of that timeframe for public comment potentially. Tom. Good evening. Good morning. Can you hear me. Yep. Yeah, I'm in the car. So if I drop it's because I'm switching cell phones but I wanted to echo what Liz was saying, and I'm here as Mike as a kid I maintain another Microsoft employee but we opened the graduation proposal for data to go to graduated in September and in all honesty this is the second freeze we're now in and it is a bit demotivating as a maintainer because as Liz mentioned, we're now six months in, all of which we're almost going to have three months of delay because of the freeze. So, yeah, I, I get why it's there, but it's also a bit. Yeah, of a worry let's say that, yeah, six months, three months, sorry, is a lot of time. The second thing I wanted to mention is that when you open a proposal. Okay, it's still somewhat unclear on what is expected. And it's sometimes in the small things like, do I add all the content in the PR that I'm opening, or do I need to open a Google Doc. Obviously the answer is a Google Doc, for which we now have a Google workspace where it needs to end up eventually but all these small things if we could like. document them and maybe consider going with issues rather than PRs would already be helpful to iron out these small things which don't take a lot of time but in the end it's friction that we can get rid of. So again, simple example, I spend a lot of time getting all the content in the PR, and then only to hear hey instead of this you need to redo everything and start this do diligence talk and I'm still in the process of moving things over because now I don't have time anymore to do that. So, just a bit of feedback from the other side. So it sounds like definitely revisiting the freeze period and what it is for those of you that were asking questions about why the freeze where it's coming from as we get closer to cube con the availability of community members starts to diminish both for to see as well as containers contributors and adopters for moving levels. Either because they're getting ready or they're doing plans or it just happens to be the time of year when their work life actually catches up with them and they got a lot of things to do. So part of it is the availability and the time commitment associated with those things. Doug. And just to that, I, why obviously I understand all those constraints. I don't think that necessarily justifies putting a freeze on the process. Right. All it means is yeah things will slow down, but that doesn't mean it has to be an explicit slow down that says we shall slow down if everybody happens to be busy. That's fine. It just not so slows down, but if everybody isn't busy. Why should, why should we, why should the TOC basically stop doing one of the things that has to do right it just work things change in terms of balances and think those slow and faster different times. I just don't think you need an explicit. We shall slow down at this period of time for this particular part of our process or part of our job and just doesn't doesn't make any sense to me. That's actually not why I raised my hand. It seems to me that a lot of these issues and Tom's comments and then my PR at the bottom there in the list are all around a desire to speed up the process and to make it easier for the TOC to get that part of their job done. And I'd love to know whether this is something that the TOC is interested in actually addressing or whether a large scale thing or whether it's more. Yeah, we think it's a problem but minor tweaks are enough, because I tend to think that significant changes need to be made. And if so, I would love to be part of a little group that goes off and says here's our recommendation for how to fix these problems. But I'm not sure I'm not sure I'm seeing a whole lot of process being made to actually address them through the issues and the PR there because they just sort of linger. And so I'm trying to figure out whether that's lingering because the TOC doesn't think this is a big problem to tackle or because they only think minor tweaks need to be made to the process. I'm just trying to understand how to make some progress here or whether we should stop trying to make bigger problem bigger changes than what the issues and PRs are trying to do. Awesome. I appreciate the feedback Doug. There's a lot actually there. From my experience in mind I have not been on the TOC a significant amount of time. The TOC members do have a lot of demands as Liz just posted in the chat for our time. And we do have existing commitments such as performing the due diligence, working with projects to get them aligned with moving those levels being available and to support them, as well as coordination with the tags as well. For their engagement with projects. So there is a lot there. We have heard in the past about how the process is slow but there's also the need to perform the appropriate level of review with projects to find some of these potential gotchas earlier on and proactively be well before they become larger more systemic problems later on in a project's maturity. So I think there is a lot of efficiencies that we could look at with this. Some of these proposals are capturing initial concerns or recommendations from various community members that we've heard in a variety of forums and trying to pull them together and make them more public for discussion and for feedback and iteration but we're always looking for suggestions ways to improve some of this Tom. I agree and I think if we just, again, from from the kid aside we we felt this is that I know upfront, I knew upfront that we needed a security audit, which takes months so we did that before we opened the proposal, and then Tommy just muted. Go ahead. You're better now. I was saying all of a sudden when we opened the proposal we needed three additional reviews from various tax which were not documented. So if we could fix that then we could also reduce the time that a proposal is open so that people are better prepared. Another thing we can do is if we go with GitHub issues, we can also use the issue forms template sorry so that people exactly know what they need to do so if you want to open a graduation proposal. So here is the checklist of the reviews you have to do this is the due diligence template you need to use and you can only open it once you've finished all of these. So that by the time the proposal is open to see and theory just has to review the material, sign a sponsor. Maybe continue to do diligence talk a bit more but all the reviews are already there. Anyone else with other thoughts on this. So we've not, we've not necessarily talked about criteria changes, we've talked about process changes and trying to gain efficiencies and moving levels. I think it is also valuable for us to like have agreement that all of the criteria regardless of whether it is required or not like that that we are able to identify, like what criteria is required to move toward graduation and I've also heard the voice. I've also heard Aaron say like we can't just provide a checklist right there's not like 10 things you do and then you are graduated. The point is that it's also important that whatever that, whatever that balance point is like you, you've reached these, you reach the gold, the goldstones that should get you to graduation. What is that last criteria that will get you across the line. It can't be a mystery. It has to be something that we document and it has to be something that we measure. And it has to be something that somebody can attain it can't, you know, like, I don't want to make I want to make sure we don't leave that as a mystery. Then Richie and Kathy Richie you're up. So I don't know. Having seen this from both sides with my project head on it's immensely frustrating to be stuck in a process where you don't really know where you're at the end. It's not what the, what the, even prospect of progressing is, and if something else might be popping up out of the perceived. At the same time with my TUC head on it's incredibly frustrating to have a project which is really not ready. Perceived checklist in hand and basically asking us why it wasn't done yesterday. I think they're given the incentives involved and given the, given the different perspectives I don't think we can find anything where everyone is truly happy. It kind of has to be a compromise and accepting this is not great. I'm not even saying not easy but it's not it's just not great. But I don't think we can fully square this circle. Yeah, I think I want to echo, you know what Richard said. And I think they, you know, there's a trade off here right we, in order to for PLC to evaluate the projects and also for the project maintainers to know why their party is accepted or not accepted. We need to have some criteria, some, which some objective criteria, but of course, we also want some flex some because parties are all different. Maybe need some for some flexibility. So there's a trade off here that cannot meet or meet everyone's needs or make everyone happy right. But I think you know, if without some clearly spelled out criteria, then there will be more issues you know people is going to say why you know my party is rejected and the other party is accepted right, and also, on what base, you know the TOS actually evaluate those project and make the decision. Yep. Thanks Kathy. Jared. Hey there Jared from the Rook and Crossland projects. So yeah there's a lot of effort going on here and definitely appreciate that one thing I'm kind of curious about is for projects that are, you know, in the process of graduating or you know opening proposals soon etc. There's a lot of open issues a lot of discussion here. So do we anticipate that, you know, since there are likely changes coming that you know projects that are, you know, working towards graduation should. We can pause for that because the criteria is kind of up in the air a bit or is there any guidance that you have around that. I can actually answer that initially. We have not discussed it. Okay. To be completely upfront. We have, we are aware that it is not prudent to levy in-flight discussions as requirements for projects. I think everybody's in violent agreement there that that is not the way that we want to go about doing things. But we also want to consider projects that applied to move levels at the state of time by which they applied they felt that they were ready. And if we are moving those goalposts for them that might be creating additional work or they might feel that it was wasted effort on their part to apply especially if we relax some of those. I think it's worthwhile for us to figure out what an implementation or rollout strategy actually looks like if and when we make changes depending on what those changes actually are and which portions of the process that they fall under. But we have not had a concrete discussion about it because we haven't even developed what those criteria look like. We've had a little bit of indicators, some active discussion on the issues. So I appreciate you elevating that as a public concern and it's definitely something that we are going to heavily consider when we look at making these changes. Awesome. That's really helpful. Emily, thank you very much. So we've got about eight minutes left. I think that we've had an excellent discussion today and we've heard a lot from a great many people both folks that have been in various roles both within the maintainers as well as within the TOC and some of the considerations and constraints that they've been encountering thus far. I wanted to open it up to some last thoughts associated with this discussion on criteria and moving levels. Doug? So what's the next step here in the TOC process? Is it simply way for people in the TOC to have time to comment on the issues in the PR? Is there a subgroup you were supposed to go? We should go off and try to come back with a concrete proposal that may get a little more attention than the PR issues out there. I'm just trying to figure out how we move the ball forward and not to say, okay, we've vented for a while and now we're going to go back and nothing's changed. That's a really good question. Thank you, Doug. So TOC members, I'd like to ask you all what you feel about this. We've been doing a lot of discussion internally on some of this, receiving feedback and fielding it from community members. And I think generally everything that's been said here on the call is probably something that we've touched topics on one way or another. How do you all feel best to move forward? Do we want to go off and try to strategize around this? Do we like the community to come up with a proposal and then review it and provide suggestions and feedback? So I think this is my suggestion. I think we have all these issues open for the party area. I think we need to collect all the feedback constructed into a document. And then, you know, probably TOC can discuss all these different comments, suggestions. We will see, you know, what we can change to what makes sense. And then, you know, we discuss and say what which part we should need to change or, you know, need to clarify. We come up with that list. And then how to create a PR or maybe before that we post maybe a PR to highlight the changes and then post for comments. Of course, you know, with all these, there are so many different comments, right? Sometimes the comments are not, you know, maybe some comments they conflict with each other. So we cannot, the change cannot meet, you know, cannot address every comments. So I think TOC needs to reach consensus on, you know, on the change. Yeah. Howard. Hi, yeah, I just wanted to give some perspective from a CNCF project going through graduation, and I apologize, I'm not an expert here, but attending on behalf of someone in the other time zone. One thing that we've seen is kind of this frustration of not having clear guidelines on what we need to do and when and how and, you know, so we get told like, ah, you need the XYZ report. And then we're like, okay, here we go. We got this done. Like, oh, actually, you need this TPS report and then, you know, the next phase. And now like we're also talking, well, maybe we should change the whole process. A lot of the process changes sound good, but I do worry a bit about why I kind of this ever changing goalposts for projects that are already in the progress. So the frustration we've had is like, we're all sitting there with our hands under our legs, like waiting to do something. But we just don't know what we need to do. Right. Yeah, I just wanted to provide that feedback. I think that's a great one. Actually, I think a lot, quite some frustration comes from the document is not clear or, you know, people don't know where to find those information. And probably the first step is to clarify that rather than make a dramatic change on the process, you know, and that would disrupt a lot of, you know, existing project and those are not the parties in the pipeline. I think that may cause more, you know, frustration. Probably the first step is to clarify, you know, I have heard a lot of, you know, comments on, you know, not clear, you know, the criteria is not clear or the process is not clear. Okay. All right, so we're nearing time. And there's some questions around how target dates for to see to come up with recommendations or tentative target dates. I don't know that we have consensus amongst ourselves to be able to provide you all with a date. So here's what I'm going to propose. The TOC is going to work together over the next two to three weeks to figure out what we're going to do if it is pull together a working group which we've made some changes a few years ago about working groups at the TOC level that need to be revisited. So we're going to make a decision on whether or not to turn this over to a working group to consolidate all the feedback and the notes that we've already collected in these areas, or to reflect internally on this. So I'm going to look at my calendar and Amy, if you could help me out figuring out our dates. Looks like we've got a few upcoming meetings. I'm going to look at things from here. We've got the 28th. We've got the fourth fourth is going to be a meeting with the TOC, like, egg updates from all of you. The 18th is our week before cubecon. And I don't know if folks are interested in being able to say like let's regroup and come back by like the 18th with maybe some guidance or next steps. Emily, is that unreasonable? I don't want to lose the momentum that we have on this. So pushing it out significantly far is not going to be favorable. So let's target April 4th for the TOC to issue a decision on how we're going to proceed forward with clear expectations for community members to get involved, depending on what that decision is. And then a proposed timeline. Does that sound good for everyone, Tom? Just something to add in the decision is what do you do with projects that are already in the process. So having that explicit. Yep, I've got it already written down and called out. Thank you for the reminder. Okay. So expect to hear from us April 4th, close of business for whatever time zone you're in. We're going to provide a path forward with details on how to engage and provide your wonderful suggestions and feedback again on all of these issues. It'll be broken up into primarily focused on the criteria, but you'll also receive some notification associated with milestones now and guiding posts once those changes are made. I'm going to attempt today to provide a summary of the items that we discussed in next steps forward on the TOC mailing list. So if you all are interested, subscribe to that. I'll also see if I can post it in the TOC Slack channel as well for folks that are lurking in there. Any last minute thoughts in the next 45 seconds? Well, I would like to thank everyone for attending today. Your feedback has been phenomenal and getting a path forward on a lot of these suggestions and remediations for improving and making the process a lot easier for everyone, both TOC, maintainers and community members as well as adopters. So thank you all so much for your time and your passion. I will see you later. Thank you all. All right. Thank you.