 Hello. Is my audio working? Hello. Hi. I just got a message from Dan, but he's late. And I am, I got my time zones mixed up. So do folks have the meeting notes? Hi, Sarah. Hello. Do we have scribe volunteers? Take the template. Apologies for. Thank you. I think we need some more time to talk about coordinating today's meeting. So I signed up to scribe, but I need someone else to second it because I'm going to talk a bit about the. OPA security assessment thing. Thank you, Justin. Any. Do we have any other volunteers? Also, people could add themselves as participants. Now that the template's there. I think we have a few new people. If I'm recognizing, in case anybody doesn't have the calendar invite, I'll put the minutes and agenda in the chat. I don't exactly have everything put up in front of me because I'm driving. So I was hoping that somebody could send me an email or comment me with the information. I can give you my email. Who's me? Oh, okay. I'm not sure what's your email, Emily. X-I-E-F-O-X-W-O-R-K at gmail.com. Should read the Moxie box at work at gmail. Okay. Thank you so much. I just added you to the attendance. All right, and you should receive an email from me in a moment. Thanks, Justin. So I'll help you, Brad, or somebody else. I can describe, too. Is it just in the Google Doc, the person that you have? Yeah, just the idea is just having two people, so when one person is talking to the other person. So that's great. Then I can focus more on facilitating. Sounds good. And I'm Karthik. Nice to meet everyone. I was at Justin's session at Kupon, and he was like, don't work to show up. So here I am. Excellent. Thanks, Karthik. So, yeah, we'll start with, I think that we don't have any big agenda items, and we have a bunch of new people. So let's do our attendance stand up. And that is where we each say our name and a little bit of who we are and kind of what we've done on security stuff or what might be relevant to the group share something about the last week. So I'll start. My name is Sarah Allen. I'm one of the co-chairs of this group. In the last week, I have been traveling and was at Kupon and did a PR on that a bunch of people chimed up on, which is turning our past events into something in the repo so that we can make note of any security related things at Kupon where we thought there were interesting sessions, both ones that people in this group gave and ones we attended that were interesting. And then I thought we could, when the videos come out, then we can pull in all the videos and people who weren't at Kupon can see interesting security related things, or we can see ones that we missed. And then the other big news is I did a lot of PRing over the last couple of weeks on governance. And today I noticed that the CNCF vote on our becoming an official SIG is up for voting now. So I'll dig up a link and put it in the notes. But I'll just call on people because I think the order is sometimes a little non-deterministic. So and Justin, why don't I leave you for last so you can close with the OPA stuff and I'm going to pick Mark Underwood because you've been here before and you can kind of people can kind of hear how this attendance stand-up thing goes. Are you muted, Mark? We'll wait a second to see if he un-mutes or maybe we'll keep going. I'll pick, I'll just pick from the bottom. So new people to Zach Arnold. Hi. Hello. How are things? Hello. As you've said, my name is Zach. I work for a money company, Wide Green Energy Fund. And I've actually been sort of like, I've been mostly assisting out in SIGs in Kubernetes played around docs and we have kind of a special interest in security in our company and working with Justin Capos and he suggested that I join and here I am. So hello. Hi. Thanks, Zach. Michael Hausenbos. Hello and welcome. It's my first time here so I'm responsible for container security at AWS and yeah, I know some phases from the dinner last week and super excited to be here starting to contribute going forward. Great. Michael Ducey. Hi everyone. So I'm Michael Ducey. If you haven't met me before, I'm one of the leads on the FACO projects. So over the last week as Sarah said, we're at KubeCon, we're kind of taking away from that conference is there was a lot of good energy around FACO. We had a lot of people coming up to us talking to us about it, offering to contribute to the project and other things like that. So we really like to see that we're really, feels like we're building a project that people are interested in and can contribute really greatly to the security community. Also really excited that SIG Security is finally becoming a real thing within the CNCF. I feel like it's something that we've needed for a long time so that's also great to see and I'll add my comments to that COC thread that they're looking at. Awesome. Yes, everybody can vote plus one non-binding if you think we should be a SIG. Thanks Michael. You're also here in part, you're also interested in being the subject of a security audit soon, right? Having FACO be that? Yes. We've already kind of agreed to, with Cure 53 around kicking off the audit the last two weeks, the security audit that the CNCF is funding the last two weeks of June. So whatever we need to do with the SIG Security Group, I know Lorenzo and Leonardo are trying to put those things in place to do the assessment and other things like that as well so let us know how we can be a problem or what we need to do rather. So yeah, I think we actually, now we have a template and a process so Justin, can you take the action item of kicking off an issue and tagging the FACO folks in on that? I want to say Lorenzo is opening an issue around the assessment. Okay, well that works. All right, why don't we go to Lorenzo next? I think you're muted Lorenzo. It's my fault, we're trying to share a speaker that didn't have video on different laptops. Yeah, so Michael, I was saying that I'm glad to see a lot of no faces from dinner here and super cool that a lot of you started to read the call. This is just my second call, I'm super excited to see what you're doing. So for the assessment, me and Leonardo here have been working on identifying the personas that need to be in the assessment first before opening the issue, we didn't want to open it like empty. We have seen the one that Justin Cappos if I would rep open for OPA and we are trying to do the same thing then we understood that there were the template but what. So we are just like putting all of that together and do the predictions when we have like people that are really committing to be like the least security engineer over there. So we look at that. Great, thank you. Lorenzo. Do you need me to go? Lorenzo Fontana, do we have more than one? No, I don't think so. Are you just like Sara? Oh, no, I meant, yeah. I'm so confused. Leo, I meant. Yeah, it's my first time here and I'm glad to see a lot of faces we had together in Barcelona. And as Lorenzo was saying, we are working towards that security assessment for Falco. One of the first things I was trying to get in place is to obtain a channel, a channel with the CNCF's lack for the security assessment for Falco since it's one of the bullet points and then as Lorenzo already said, we are working together to identify the persons that we need to involve on our side, external side for the assessment. That's it. Great, thanks, Leo. Lance. Hi, so I'm relatively new to the CNCF space. DIMM's actually suggested that I stop by. I've worked in a couple of different communities, specifically around auth and authorization, authentication, stuff like that. So first time here, excited to get up to speed and participate where I can. Welcome. And Karthik, thank you so much. Hello. Oh yeah, you bet. My name is Karthik. I work at Oracle. I have done a bunch of random stuff around auth and different things in the DevOps space. I've been talking about Kubernetes security at different conferences and kind of wanted to join the thing to actually help with some of the core stuff in there. So yeah, just looking forward to interacting with everyone and taking this to cool new places. Great, thanks. Joshua Locke or Josh. Yeah, Joshua. I work at VMware in the open source analogy center. Was at KubeCon last week and met some of the folks on the call and just looking for ways that my team and I can contribute. So yeah, got a lot of experience in the open source distro space, but not a lot in the open source cloud space. So looking forward to learning more. Lots to do. Daniel. Hi, my name is Daniel Zirov. I'm a security engineer in Adavinta. I was in KubeCon last week and I saw a couple of good talks and I decided to join the CIC to contribute somehow. Super, welcome. Anthony. Gaming company, but prior to that, I have experience with penetration testing and their security review roles at a previous, I was for a third party reviewer assessor. So just interested to help however I can and provide input where I can. Thanks, Anthony. And then also for next time, people have requested that sometimes it's easier to understand people if they're willing to put the camera on when you're speaking. So if people have a camera and can share, that'd be great. And Mark, I don't know whether you have audio. There you go. We can't seem to hear you or I can't. The hotspot decided to drop me right when I started talking. Yeah, so I'm the co-chair of the NIS Big Data Working Group, Security and Privacy Subgroup. The interesting thing from last week that I want to share with the group is we're going to be collaborating with the HL7. A security and privacy group to produce a working document to crosswalk between these two standards bodies. So I don't have a suspect yet who's going to collaborate with me, but we're putting out a call to the big HL7 security subgroup. Those of you not familiar with HL7, it's a kind of the electronic health record standard in specifically this is the fire interface, which is their newest technology. They have a premature methodology for things ranging from provenance to authentication to governance around data sharing that's highly granular. So looking forward to that and I'll share back to this group as we move along with that. Great to see the new faces, by the way, everybody. Great, and will you drop a link to that if it hasn't already gotten into the notes? Is there a website for the HL7? Thanks, Mark. All right, so did I get everybody except Justin? Is anybody I missed? Oh, on the phone, Emily. Hi, I'm Emily Fox. I work for the National Security Agency. I head up Justin Cormack on Twitter after his presentation at CubeCon, because a lot of the stuff that you guys are currently working on or figuring out is stuff that I've already done. So I thought I would chime in and assist where I can along with your processes. So that's about it. Fabulous. And yeah, and so Justin Capos is actually facilitating, coordinating our many volunteers on the security assessments. And so Justin, if you would also, for the new folks, introduce the process a little bit, as well as giving your update on OPA. Sure, I can do that. Okay, so I'm Justin Capos. I'm a professor at NYU. I'm also like the creator and maintainer of Tuff and one of the co-creators of Vintoto and a bunch of other stuff like this. And I think the assessment process that we have largely came out of an assessment that I did for Spiffy Inspire. So I've been doing these kinds of assessments in part in my role at the university, working with a lot of startups and groups like this over the last, I don't know, like probably six, seven years and taking ideas from that and experience from other people in the group. We wanted to have a more standardized method for doing this as part of the CNCF process. So I will post a couple of documents here into the chat part of this. So Emily, apologies for this. I'll send them to you in a moment as well. But the documents I'm posting here are two. Example, one is a document that... Hold on, let me send this to Emily real quick. Now you see my inability to multitask effectively with this. Okay, so effectively the process itself that we go through is that the project comes to us and wants to have... Sorry, so the goal of what we're trying to do is we want there to be an assessment and the assessment is not meant to be the same as like a code audit. The assessment is meant to be an examination to understand things like are they solving a problem in a meaningful way? Are they likely to have major security holes in the way that they're solving things? Is there any risk that projects take on by using the software and so on? So it's meant to be not like a code audit, but it probably can find problems at sort of the high level design level and can be used later on so we can point out to auditors or others. These are areas we think are especially important to focus on and also gives a little bit of high level context to someone who wants to perhaps use the project so that they have some understanding of the potential risks or other security mechanisms that are needed or more than just you would get from a marketing blur from projects like, hey, we secure X. Well, what do you mean by secure? That's a very loaded word. So as part of this process, we've gone through and done a few different assessments. The first two assessments were done in parallel in TOTO and OPA. Since in TOTO is a project that I'm a co-creator and heavily involved in, I wasn't involved, of course, in that assessment because that would be something of a conflict. But the OPA assessment is one that I also had sort of planned to talk about in this meeting to give people a heads up about where we're at and it might also be a good one to look at as a general introduction to what we do in the security assessment process. So I promise I'll try to keep this pretty brief just so you get a high level. There's a ton more detail about all of this in the repo and the documentation. But there's a few, the SIG Security Assessment OPA document is the one I'm referring to here. This is a document that's written by the project themselves that is sort of their initial gathering of information that we need in order to do a meaningful assessment. And this gathering of information explains all the types of high level things I mentioned before and it also describes just sort of the project's view of itself, its practices, how people are supposed to operate the project and use it in a cloud native context and also things around, for instance, the development and security issue reporting mechanisms and stuff like that for the project. So as a result of doing this assessment, they go, they provide that, they provide it to whoever's leading the security assessment. In this case it was me and then I respond with what we call the dumb question phase. And the dumb question phase is really just trying to get clarifications in the document so that a reader that isn't as up to speed on the project or maybe as up to speed in every case with security has an actual understanding about what the project's trying to say and what the project is and what the document means. So in part it's some low level stuff like getting people to define terms but it's also getting them to be just a lot clearer about what they mean as they talk through different aspects of what their project does. Following this, the group of security assessors which we had I think four participate in this one, we went through and we did a pretty deep dive into the document. I think we say it takes about 10 hours to do for an assessor. In my experience it was very slightly more than that but not substantially more. I'd actually expected leading it would take me a lot more but it maybe was 15 hours or something like that. And based on that we try to get that document to be clarified in all the potential security issues and things like that for them to revise the document in a meaningful way that explains their view on what the project's security is. Following that process we also as the SIG security like sub team that's doing the assessment write up a document that's about a page. You see there's a document that's a little more than a page I also sent that goes through and basically describes what we believe the project's security posture is and what are the benefits, what are the things to look at and if someone like the TOC were going to go and do an assessment what would the TOC perhaps want to or sorry an audit what would the TOC want the auditors to explore and what priority would they have for them and so on. And as these this one and the in total assessment I think the in total assessments in a very similar state where we're just waiting right now for I don't know what but the OPA assessment has been passed back to the OPA team they're going to look over our assessment maybe have some comments or questions or things that they'd like us to to talk about here. There are a few I see there are a few comments and things in here as well for areas that we need to address but then the idea would be is that both of these documents would both their full long assessment in our kind of summary of this would go in a public place in our repo and probably also be linked to off of the the project site and they would also be provided to the TOC when the TOC is trying to decide what to do to vote on a project to have the projects and enter incubation or sandbox or graduation or whatever. So I've said a lot and I'd like to pause now and have anybody ask questions and so on. Are there any questions? Hi this is this is Paavan I have a question around the assessor actually I was in the talk that was given by Justin last week and there's QOL 53 as an assessor is there any plans to have a pool of assessors or is it just just one assessor? Yeah there's a group that's trail of bits and who's the other person at treat at treaties or something so there's a continuity has a report and the report says it was assessed by QOL 53 it's based in Germany so I was just wondering it'd be nice to have an assessment by them as well but there are a pool of assessors there is another team that's trail of bits and another security consultancy that I don't recall the name of but it starts with an A and so they're colloquially their assessments are colloquially being called trail of whatever the A name is so there are two assessor groups now QOL 53 as a single entity and the trail of bits plus the other group and using our nomenclature those are audits and the assessments are a different process yes doing as a SIG right so I think for the time being we're the only groups that are doing assessments but the assess like if you've had an audit then you've probably you know you've gotten a deeper level dive then what you'll usually get from the assessment although I think the assessment process is a good precursor because the documents and reports that come out of that will be very handy for anybody who wants a like a quicker look at what's happening I think the audits like if you look at our Q53 audit or you look at some of the other security audits we've had they tend to be very much written for the project's consumption and the point of the audit that we're doing is it's really written for or the assessment that we're doing those the document we produce is really meant more for the public and also for the TOC and so I think there's some you know there's some value in in having that context especially for projects that aren't or for people that are trying to decide what projects they might want to adopt and don't want to read through a bunch of like oh we found a buffer overflow here and we found this issue there many of which have already been fixed yeah that was my question so I have a different question and it might not have been clear to me on the site and going through some of the docs the security assessment is more of the review of the proposal template that they're filling out submitting an issue to get for and that's intended to be prior to incubation prior to entering into the incubation phase of CNCS is that correct so the idea is that the TOC will ask us to assess some projects that have that they are that have been proposed to the CNCS it's CNCS at whatever level and then once we've done an assessment we would we do it annually we would only use the like sort of sandbox to incubation or incubation to graduation if if there were some things flagged that we said well we'd expect this to be fixed in a certain amount of time or sort of thing then the TOC might ask us to take another look at it regardless of the annual review cycle but the the assessment we're doing is a bit decoupled from the graduation stages okay and the assessment is independent but can contribute to the audit and the audit occurs at later stages correct yeah so the audit occurs during incubation before graduation typically although I think you know like it could happen it could happen at other times at the discretion of the TOC right sorry I was just going to say Emily this is all a new process so you should treat everything we're telling you as what we think is happening at this moment but everything could change in the next you know it could change tomorrow as far as we know there's not a solid track record of this yet yeah and generally our process just for context is we're going through this and then where we've set a checkpoint after we do five of these so that we can reflect on it but your feedback and questions are totally welcome and thank you that's more what I was asking is like how young is this process where does it need to go do you have stuff already written down and documented a lot of what you guys were saying are things like a kind of guess that but I wasn't sure it's written down and being addressed and being adhered to in any of the docs so so yeah so just to see this in history we have we have formalized the process as a SIG that was gone through by individuals at request of the TOC last year so like so basically the TOC tapped you know like Justin Kappos and Justin Cormack that said hey can you take a look at these project the security profile of these projects for us right and then and then we thought well we wanted to do that we wanted to understand that as a group and we asked Justin Kappos to present what he'd done with Spiffy Inspire and then that evolved into this process and so we when we've taken in Toto through the process but it's kind of gone through it multiple times as we've shifted the process so in the assessments directory in the repo is a definition of the process that I believe is exactly what Justin just said but it's possible that it doesn't exactly articulate what we're saying and so we're in this phase of attempting to follow the process as written but also adjusting the documentation as needed and OPA is the first project to go through a written process and we have four of us who've been doing with Justin Kappos as an exception because he was on the project side of in Toto who've been through the in Toto assessment and the OPA assessment and then the idea is that we will add new people into the group and we will have a different group of three or four do the asset so we don't wear out our first four people do the next few so that we end up cycling in and expanding to a team of 10 so that we have bandwidth to do these different assessments without burning people out okay thank you for the explanation yeah and thanks for asking kind of forgot some of the new people might not realize yes we just started this and but it's a yeah and I think that the key thing is to kind of attempt to reiterate the point of this is that the it's really so that people consuming these projects might like even know do I want to read the audit of this what what things would I look at like first I can figure out whether this project is at a stage and this project does what I think it does or might hope it does or maybe it doesn't before I would dive deeper into it so the assumption isn't that somebody looks at our assessment and says oh now I know enough to use this project and integrate it and deploy to production but that is a step that allows them to weed out things that they wouldn't look at further and then know to which things they should look at further I don't know if you if it was mentioned but is there a plan to include like security best practices in the investment so for example how developers are developing a particular application under assessment meaning scanning for code vulnerabilities in a in CICD process or if it has like a user facing interface going through OS checklist for example like application security verification standard or something like that so we have we have sort of we we ask people right now to kind of report on there where they are on the CII best practices list and then we're really looking to the right now it's this self assessment because we are not we don't have believe that we have enough context to put forth all of the best practices we have a new group with a lot of different opinions and we don't want to sort of set the bar for documentation weirdly high so that it's too hard to get through but we do expect some things to emerge from this and maybe Justin Kappos if you could chime in on that yeah I I think a way to say it is is that like what Sarah said is basically right we want it to be something like a week or two of effort for the projects like all told like like one engine one to two engineer weeks of effort to get them through this through our assessment and so we do recommend exactly the types of things that you say here but there's not in this assessment process there's not like a real forcing function other than tell us where you're at with CII best practices we are also as a group there's substantial interest for creating a bunch of tooling and other things that makes it easier for us to have sort of best practices for projects overall and you know perhaps do scanning and other things of cloud native software but that's something that is not something that has any flesh to it right now it's just to think an idea that several in the group had that I imagined they'll will pursue and flesh out over the next few months also if people have ideas like that and you know particularly specific standards that you think that we might include in the future as recommendations please like if you're new to the group read through the issues the open issues we're trying to develop a tagging system so that they're more approachable but definitely I've been trying to add a couple of other people have volunteered to help triage these things so that but it really helps to have other people chime in in terms of you know kind of putting flesh on the bones of the future stuff and so we kind of queue up things that people are interested in working on in the future through GitHub issues so definitely welcome and if you if you've read through the issues and you don't and you have a question or concern or something you think we should explore that's not an issue just write one up also so other other questions so I'll take that as a no I wanted to answer where we were within Toto unless that was somebody chiming in with a question I've just put into the we've got sort of a draft summary which I just stuck into the docs I'll actually share my screen we are it's the sort of leading vetting of the format that we're still going through and so we've gotten to it so basically to answer Justin's question of where we are is the process was a little interrupted by travel but basically we want to the the summary to include links to the issues rather than actually having any issues in the summary because there's a bunch of things that we've gone back and forth and we're like they're not things that would they're just fyi these are things that need to be done nothing like critical but we want those captured so we're just sort of going back and so writing them up so that the summary can be more concise and just have links to things that are open in github rather than being a like in the narrative of the summary and then it's also easier because then the status of them is in github rather than in our document which will become stale so this is our quick question is this is this document supposed to be public or is it still like hidden it's it's not secret but not advertised so you can drop if you look at the security assessment issue on github so I'll give you a peek into our process so in our issues we have a label which is this is sorry this is the wrong repo in our sick security repo we have in our issues we have a yellow assessment tag and you'll see that we have two in process in todo and open policy agent and then this shows that in todo is you know like it's actually they're both nine out of 12 check boxes through the process so this provides visibility so if you're you know if you're the project or you're interested in where we are in the process you can be like oh look they're kind of here in the process and then the slack channel is listed in here so the slack channel is public but it's you know you're wearing the weeds of it so if you're interested if you want to like dive into this you can go and dive into the slack channel and chime in on the docs but they're really like not ready for consumption because they're in progress and there's still a bunch of notes and wrangling and so we're basically in the slack channel with the security reviewers who are volunteers in the SIG and the the project lead working through open questions and things and so that's where we're at where it's just not quite ready to for us to say hey you know this is our assessment of the project right now it's this is what we think it is and there are some open questions and we're editing it so you know there's already people who've you know jumped in in our various slack channels and and so it's the kind of thing where if you're very interested you are welcome but if you just want the results way to weak or so and it'll be easier to consume does that make sense yep gotcha thanks and so just as a sort of sneak peek is the this is sort of our what we've come up with as a way like this the TOC requested like a one slide summary and then we've come to the that this is kind of a short form of the one page one to two page document that Justin went through and so we checked in with everybody that like and I it's just on me I was going to run this by Liz and Joe beta who are TOC liaisons to ask them like is this the format that you want so that while we're going through and you know you notice there's like an X companies and and issues and there are links here that don't go anywhere so those are the last things that we're doing because we kind of made up part of this questions after the initial write up from in total so so that's why the the data is a little like backfilling based on what we questions we came up late in the game so um so yeah so I don't know if anybody has questions or comments on or a wrap with this you said that one slide is required for all the assessments right I'm sorry you said that this one slide is requested for all the assessments right yes and so the idea is that typically the assessment will happen before the project gives a presentation to the CNCF and then the ideally the project would be giving a presentation and include this slide that the project would communicate I think in a where instance we would well we hope I don't know we have different thoughts on whether it's rare but in some instances we may have a disagreement with the project in which case the project you know like then there would be a different kind of discussion and so but basically the idea is that this is something that ideally we would and the project would come to an agreement that like this is kind of what we all think that the project's security profile is and then then it's something that is at the TOC's option presented live or it could be certainly if it's part of an annual review or something that is kind of not queued up for a presentation or the the TOC is pretty busy then it might be an async review so that Joe and Liz will kind of coordinate the other question I have is like this part of the process is probably described somewhere right this last part about the TOC presentations is only described in this checklist because we don't we haven't been through that yet yeah that's that's all I need to know just fun it's kind of it's a different plate right yeah exactly so basically we're like oh there's this thing that we haven't done yet we're expecting to we're expecting that in total we'll do but we don't really know whether it's going to be make sense for projects that have already that are already like deep in the CNCF and it may be more like we are providing information that hasn't particularly been requested and then you know like it's sort of like basically the TOC would like us to kind of oversee and coordinate with the security related projects as a top priority but then some of them they're very familiar with and maybe they won't actually want to be presented at this particular time so we'll we're that'll be it's own adventure great thank you all right so I'm gonna stop sharing and if there are urgent questions shout out or time in in the chat but Justin I wanted to give you a chance to finish up if there's anything else you want to close with uh I don't have anything really pressing I guess there've been I guess I think Emily just posted a question if we want to discuss that so do we have a maturity measurement indicator for consistency or is this gut wide range of adopters to me seems it could be kubernetes size of adoptions or telepresence size what is a wide range I think so this is I think an area that we had discussed and debated a lot I was actually pretty opposed to having anything related to adoption in here because I think it is hard to quantify it and it's hard in some cases to get really meaningful accurate numbers about this so I don't really know what to do in this case I would be very happy for one of the other assessors to propose text for this part of the assessment document and we could discuss this but I wrote draft text as I was just trying to capture everyone's thoughts and that was the closest I could get to being specific which you'll notice is exceedingly vague so I'd like to see and maybe this is just me that the community maturity adoption not necessarily be included because I mean CNTF is probably going to end up looking at a a wide swath of projects that maybe brand new not a whole lot of people know them super early in the adoption cycle but can have a large impact on community and some other ones are just like Kubernetes it's taken so long for communities Kubernetes to get to the community and now like everybody's using it so perhaps going more down the maturity of from a development standards like and their development business practices and models how mature is the entire software development team if you look at the three ways of DevOps are they stuck in the first way trying to figure out what their product or services or are they all the way in the experimentation rapid deployment automation feel fast on all of that stuff and that may be a better thing if we're trying to provide maturity of the particular product because I as a customer trying to research and do all of these things may have higher confidence in an application that has a higher maturity in their development cycle and not necessarily care so much about user adoption I guess I'd be interested in you talking a little bit more about this development cycle maturity the three stages that I may not be familiar the specific three stages but when I'm evaluating adopting a project whether it's used indicates a certain kind of maturity you could have something that has very mature software development practices where it to presents itself as having good documents and you know like all sorts of things that make it seem really polished but it's never actually been used in production and that doesn't mean it's bad right like that could be like well it's way better than me building it on my own but it is not like I want to know that versus while companies that have millions of users are using it in production versus a couple of open source projects are using it on their production test sites yeah and and that's part of the struggle is when you start talking about maturity either development or end user adoption it's always like a snapshot in time from when the report is being written so I think if you were to include it no matter what definitely publish the scale at which you're providing that ranking value if you're going to do one the other or both of them it just depends because anybody can go to github and see how many stars and works and like active user community there is like that's not hard that's a Google search way if that so it just depends if the group is intending to provide additional value above and beyond what is natively accessible over the internet or whether or not they're just regurgitating content for that time slice of when the assessment or the audit occurred well the whole thing is a time slice except for like the goals I mean even that can like the feature set can shift over time so it's definitely a time a moment in time thing but I think that at least from my perspective like the number of stars is an indication but it doesn't really tell me anything about like just because developers like it doesn't mean they're actually using it for reals yeah and so the project actually saying like this is all project like we're not like going and interviewing all of their users and so what I had written for the intodo thing is like in production by x companies meaning there's a number there or you know where like which we have to I need to go back and forth with Santiago and Justin Kappos but like it could be to our you know x companies that have we talked like some concise way of saying x companies that have told us they're using it in production right it could be more than that right or at least x companies so that tells me something right or you know versus we may have another a sandbox project which is not yet used in production by anyone and those that is like for me a big differentiator whereas like opa is used by you know you know like x would be I'm guessing right dozens or maybe hundreds I don't know but it's at least dozens which is very different from three and but you might look at it and say well those three you know that are being you you know you have to dive in there right like dozens doesn't it doesn't it just means like do I look at this more it's not supposed to be a yes no but anyhow this is this is a long way of saying this has been under great discussion but maybe you could speak more about these maturity models that you were talking about which might be easier to articulate yeah I think it's always hard to I was just going to say like people always end up doing maturity matrix in like any company that I know of and it's always hard to like distill that into just a few words yeah it's typically easier to come up with a standard unit of measure post it publicly as a reference point and then refer to a particular value within a published document for whatever that state in time was and then as you go through you can update your independent matrix for like how you're deriving the value and make adjustments therein it just it's hard when there isn't another thing to go off of and the landscape is so large and it varies so much for what people look at without having something to point to and say this is our true north you're going to continue to have that problem as new people come into the project or come onto the effort and start asking questions that like well what does that actually mean how do I interpret that how does it apply to my company so in the what so just to be clear would you given your experience say until we have such a thing don't say anything no I would argue usually so this is the policy lawyer side of me unless other further guidance is provided or subject to change or some other caveat that you're good at a jail free card for that date and time or refer to it by a particular date value that the maturity of a particular project is considered good given this time slice or given this date given this set of criteria or using I don't know whatever standards body is providing metrics for us to make this decision off of but always have that kind of caveat that get out of jail associated with anything that you're posting in those documents because they can and will be subject to change or they may be overwritten by a new policy or new requirement whatever it is that you're doing so not saying get rid of it it does have value if I'm doing an assessment it's certainly easier for me to go to one place read all the information I need to about a thing and then go and talk to a like my technical lead or my lead architect about a particular project then to not have it and spend time doing research so it's a trade off yeah I think I actually the point about having a date on the summary is really important so thank you I don't think that's actually in my doc so I will fix that so Justin we only have a few more minutes and I wanted to check in with Dan to see if he has anything or JJ in terms of forward looking stuff before next meeting that's all done excellent Dan or JJ do you want to chime in a little anything on anything we need to cover in terms of plans for the future I'd love to see us get back on track with the white paper you know that's the biggest sort of tracking item you know now that we're you know landed we have the opportunity to request those resources and we were you know blocked by formal ratification to to line that up so yeah I'm gonna I'm gonna follow on that with Chris and actually maybe we could let's talk a little bit I think we need like we should queue up a meeting in the upcoming weeks to go over the roadmap so we've had we've had a bunch of sort of small group conversations about the roadmap and we've been corralling the github issues and so that might be a great thing to queue up for a future meeting JJ is there anything you want to touch on because I saw you that's what I was going to say the roadmap discussion that we had the other day I think it'll be good to and white paper if you want me to super hard to hear whoever that's JJ so let's let's not dive into the white paper let's queue up the roadmap discussion and as part of that figure out how we're going to corral the white paper project but like take it offline because we've just got a few minutes but thanks for mentioning those that stuff I should have I didn't think to talk about the future and is there any other last announcements I think there are some people came in late so I want to give a minute if anybody has anything urgent or interesting all right well thank you all for joining especially the new people if you're new feel free to PR yourself into the repo as a member and welcome everybody and hope to see y'all again Hi Hi Have a good one Thank you sir