 Good day! Good morning! Loud and clear. Apart on that, it's a Cherry Blue mechanical keyboard. I don't dare use it at work, so use it at home. We just posted the link there to the attendance for everyone's convenience. On the set, just give a couple more minutes to critical mass, and then we'll get underway. Okay, so welcome! Good day everyone! I'm just gonna go up here to the attendance, see if there's any updates. Okay, looks like we got a few here. All right, so before we go forward, is there anyone that's able to step in as a scribe to take meeting minutes today? Any takers? All right, I'll just monitor the chat and see if anyone is able to step in as a scribe for today to take the meeting minutes. Until then, we're just gonna go through our general check-in and the usual workflow as per normal. Just gonna see if we got anyone here from any working groups or states. Does not appear to be the case. Thank you, Stephen. Are there any members here? Is there anyone in attendance today from a special interest group or from a working group that has any updates? No updates, but the policy working group, I think, is meeting this afternoon. Okay, thank you. So then we'll just move on to our general stand-up here. We got a... Matthew, I have a quick update. Sorry. Balmings? Balmings. So, kind of debated whether... How much I wanted to share, but I just wanted to sort of put it out to everybody and, you know, techniques that I haven't heard from Sarah and JJ for a few days. I'm a little bit worried. Sarah was stuck in Boston and beginning to get ill, and JJ is stuck in India and beginning to get ill. So I've been trying to coordinate a couple of things with my co-chairs and, you know, they've been largely offline, you know, since they are in sort of non-traditional places. Anyway, so the upshot of that is if you are waiting for anything from either of my fellow co-chairs, I haven't heard from them either. I do hope that they're all right and I don't have any further information, but if you have needs that aren't getting fulfilled, you can reach out to me and I'll continue to coordinate. So, I apologize for the whole craziness around this and I will share back in Slack, you know, in the security channel once I hear from them and have any sort of general updates on how they're doing. Thank you, Dan. I do have an update as well. So the harbor assessment and things like that are beginning to start. We have the people and the statements have been signed off on. Everybody has gone and done that. I've just gone ahead and approve the reviewer conflicts, although Dan, I'd feel more comfortable if you just looked at it. I have the most non-actually plausibly even listable sort of conflict in that I, in that notary is used, uses tough, which is used and notary is used inside of harbor. So therefore somehow technically I've worked on harbor but not really. So I think, you know, but I don't think there's any real cause for concern. So just maybe take a take a glance at that. And we're looking forward to to kicking off that assessment. So thanks to everybody who's volunteered and started to do part of that. Thank you. So we have that check in from you, Justin, as well as from you, Dan, and then we have two additional ones, one from Roberts and one from what I saw if I got that correct. Robert since you're at the top list, do I go forward it's GHI number 273. Yeah, so this is the Falco assessment. I had to join the Falco community meeting this morning. And the project team there expressed high interest in continuing with the assessment process. So much so that they, they actually will gate their 1.0 Falco release on completing the assessment. That showed, I think, pretty strong commitment. That said, the flip side is they don't have a resource identified to step in and actually be the project point of contact. So, you know, I'd wanted to raise the issue, maybe later in the discussion here. There's some mechanism by which CNCF might be able to sponsor or provide some sort of, you know, project based granting or, you know, internship or something. But, you know, I don't know if there's a way that CNCF can support projects that are, you know, have high will, but low resources to complete the process so that just maybe we can take that up later in the call. I don't want to comment on that. I don't see Amy. You know, I know that in other Linux foundation, foundations that I'm a member of, you know, that's, you know, much more consideration and in yet another, it's unthinkable. So, I'm actually not not sure where the CNCF lands on that one. Also, it would be hard to do this like you can't just throw an intern who's. Yeah, that's what I was going to say it needs to be someone who knows the projects I didn't see how that would really be feasible and I think that, yeah, there are potentially one off. Bits of resource for things with the CNCF. I just not sure how what the request would really be for that would be meaningful. Right, you know, I could just put sort of names and project a bit in this. Chris Milva who doesn't have, you know, sponsorship, but this is available to, you know, wrap things up would be conceivable. I can't see, you know, a unidentified resource, you know, coming out of left field, you know, getting sponsored and basically hired into, you know, doing something like this. Is there any sense that an individual is already available and it just requires funds for the individual need to be found. No, I think the problem is that the individual does not exist. And I think that the kind of the notion of like you say like an internship might be to go out and kind of broaden the recruiting effort and say like, you know, kind of Google summer of code style. If you want to work on this over, you know, a summer three month project, you know, you could really dig in. I mean, there is a is an open summer of carried CNCF right now. So potentially that I don't know when the period where the students choose the project is I guess it's not yet so that's possible. Okay, and that maybe the maybe that's the avenue to try it's just literally the Google summer code. I mean, you can do this, but I feel like. Yeah, once again, like this isn't like a lot of TPS report sort of thing. This is, you know, this is like somebody who deeply understands and is deeply involved in the project needs to give us information so we can do a meaningful assessment. We can do a huge amount of work, like time wise. So yeah, we're taking like very conflicting statements. One is that they care a lot about this and are waiting to hold up a release for it. But the other is is that no one wants, like feels like it's important enough to do it. I don't I don't know that I don't know if they don't feel that it's important enough. I don't think any one person there felt they could be a good representative. But I can probe deeper into that. I also think that they are, you know, one, one thing that came up is that they're, you know, they're actively fixing existing security issues, some of which are not Falco issues per se other kernel items that Falco in a sense has discovered, but that they have to create some problems or whatnot. So I think there's, there is that and that folks that probably do have the security chops to participate are actively working on high criticality vulnerability or other security issues. So it's kind of this trade off of, you know, do I do more security review, or do I fix the things that we already know about. The area is more important than security expertise. So we would be perfectly happy with someone who understands the system well but isn't really a security guy. I want to jump in on this because I very much agree with, you know, your concerns about, you know, not just resourcing now but you know this is a signal on, you know, project participation moving forward that we shouldn't listen to as well. I can add my two cents. So you can hear me. I'm going to, I'm in a different room. So I thought of them echoing like a ball. So the background is, I obviously work from open source project itself. We do a lot to think like this as well sort of in our own dimension. If a project can't rest or muster the resources to facilitate a review, typically we just assume that it won't be able to muster the resources to maintain maturity enough to make the review worth our time. So for my team, Turner, it's a self protection mechanism. There are lots of people who would like that credential for credibility. But unless it comes with sort of opposing resources to us, we found that it's typically a black hole or, you know, a flip flop, right? They say things, we say things and it just, it's a low friction drag over a long term. And as far as sort of pulling in GSOC or we do another thing like GSOC, that's not really a force multiplier. In this sense, GSOC people are typically, you know, one to three years of experience and they're maybe really educated, but they lack depth. And you can take three month engagement and really make it something that you could accomplish in four to six weeks tops. That's the general rule of thumb that we operate under. But that four to six weeks of additional manpower, you're going to subtract maybe two to three of you. And the whole GSOC engagement gains you two weeks of a person with one to three years experience, which is farm, right? If you're looking to further things and I don't want to go down that road for further than to say, that's not much of a value add here. I would just pause the whole thing. I mean, my thought on it is, I don't know if we're flagging graduation based on a particular commit. And it's like, we're not right. Everything is more or less rolling release. But if they aren't at a stopping point, where they're ready to walk through the process, then you sort of multiple vectors of lack, right, a farm shorter. This is a graduation. What is it for? I'm sorry. Because I'm aware they've not applied for graduation. I think someone mentioned a one point over release. I don't think they're intending to apply for graduation at this point. I was just assuming that's the sort of mountain top, right, whether they've put on their hiking boots or not. But yeah, I would, I would just pause the thing until everybody's ready. That's my thought. I am also thinking that maybe they want for us to fix their existing vulnerabilities as you said, Robert. And they feel that, I mean, it's normal that you want to clean your house when you invite someone who inspect it. And my second thought is that maybe it's good to not push them and just wait for them to approach us when they want to and when they feel that they're ready. I mean, I think because this is a cooperation and I think this is the right way to do it. I agree. It should definitely not be a push from us and I tried to articulate that on the discussion this morning that, you know, we weren't pushing any agenda or any timeline or anything like that. So I agree with you, Martin. Is there any more on this one if nuts will move on to place law and pardon. I have one additional thing, which is for our sort of foundation internal assessments of this ilk. We have a thing we call a requester service. And you have to basically have all the information ready to go in order to initiate the process. So I agree. There is not a common understanding and sort of where they need to be in order to step through, like request or service would involve a designated liaison and if there is one then there is no request or service. So I've seen what is submitted sort of, but I don't know, maybe I need some additional information or some additional sort of copy or introduction to that expectation. Well, it was delayed. I think, because it was originally going to take place earlier and then there's for reasons it was delayed so I think it maybe there's some. I can see there's also leeway with people not being available so I'm not sure that that's entirely fair but yeah let's just wait until they are ready. Sounds good. Thank you. Moving on to law and ghi number 372. Yes, that's just for awareness. So, it's Wednesday. I lost track of time. Yes, yesterday I opened again submission of kick off the CNCF as a sandwich project. Together with this I opened the assessment issue for security for those of you who remember the whole story kick off originally applied like June, 2018, we were scheduled to present to to see October, literally two hours before the meeting we got dropped on the agenda, the feedback that's to see three thinking the whole submission process. Then we finally presented April, 2019 if I recall correctly. And then it was moving slowly than six surfaced and actually back last summer I was emailing with Sarah and I even created the initial self assessment documents. I even scheduled at some point with Brandon step stepping up to lead it. But then IBM acquired red hat and Brandon took a step back because he had conflict of interest to lead it. And then we knew that the whole process gets re-engineered and the elections are coming so I kind of put a halt on all of this. As processes new processing is in place. And elections happens. I want to re approach the topic with some, you know, refreshments improvements. So I created the issue of the new template and I created the self assessment documents. The interesting factor is that also last November we had a proper pen testing happening sponsored by one of the end user companies. So not even by red hat also linked to the report. So yeah, I know there's a queue and there's a set of priorities so just, you know, mentioning that that's, that's, that's addition to the queue. I think the process a lot more seamless. You know, I, I know that this process hasn't always been smooth and things in open source where they are so believe me, you know, it's, you know, there's there are other people that I felt a lot of frustration to with different things, but of course we're all just doing the best we can. I'm joining those meeting not all not only because of the clock, I want to engage and even if people files. So I witnessed what's happening and how it's rolling so I appreciate all the efforts for everyone involved here. I fully understand all the whole context and all the hard hardness of this. Okay, thank you. Any other points on this topic. Moving on, we have next up presentation by Mark underwood on scale that gel framework operator guidelines. Mark, don't seem as muted but I don't hear audio coming through Mark. Yeah, I forgot to hit the button out on me. Let's see if I get the screen right here. Looks good. Okay, is it showing the navigator screen or the good screen. It's like a top navigator slide and slide. All right, let me see if I can switch it. But now that better. There we go. Okay. All right, so I've embedded a fair amount of background on scaled agile. Should we take a vote on whether everybody already knows this and I should zoom by this. I'm going to assume you don't unless you say otherwise. Okay, it's a safe assumption, especially since we're recording this. And, you know, for folks who may come back and review this, you know, having that context will be helpful. Great. So scale agile is a open source but commercial framework that's got its roots and a number of other things. So the point of this really this presentation was to consider kind of a sub sub aspect of the scaled agile framework in which we were mainly considering the issues around opera operations and the specific notion of operations that I focus on here is the ops that we run into in dev ops. And it turns out there's a lot of supporting information to share in this presentation that is going to seem like it has nothing to do with that so bear with me while I try to go through this. So here's some disclaimers I'm ripping off slides and illustrations from the scaled agile people so apologies for that. Also, I put a little music symbol next to the stuff that's my opinion so you can separate the stuff that I'm borrowing from the scaled agile people, because that's intermixed in here with quite liberal self permissioning. So the way that I see this scaled agile is built from, and they describe it this way to lean agile and dev ops and it's all really pretty good mix of that stuff. They mentioned model based system engineering but that's pretty lightweight. They don't mention composable services but I think that's really a major part of it that ought to be considered and of course, the bigger legacy probably going back for decades is object oriented programming and a big lacing over all of this of the quality movement with the plan to check act cycle some people know this is the Deming cycle and institutional practice around ISO 9001 which goes back to the mid 80s or so. So this is the only time I'm going to show this diagram although we can zoom back to this there's you can see these on the public website for them. There's what you see up here in these various tabs are even more detailed implementations of this process in which they try to explain how they implement the agile process which you know the core thing is daily stand ups, the scrum process, trying to decompose the work into small pieces and working with small teams, writing stories and trying to educate one another about how to collaborate across teams so in the scaled agile framework there's a bigger process of how to orchestrate across the smaller teams. So you see this in this thing called the art, which is their agile release train. So that is a sequence of multiple teams working together on projects. And yes, you should be asking yourself when you see this, wait, how does this work for CNCF. Also, I think we're talking about here which will see on successive slides is the notion of enablers so I'm interested in where security and privacy fit into enablers when people in sprints are trying to do system demos so that constituents can see what interim products are going to going to produce. I have probably more slides and we have time for so I'm going to go through this with some speed. The principles that they that the architects of this thing which is a company based in Colorado by the way is kind of interesting stuff and if you look at it is kind of not software engineering, which probably tells you a lot it's an attempt to try to import concepts from outside of software engineering, but apply it to what is really a release driven notion now that's a comfortable thing for most of the people on the call here, unless you happen to work in the ops community. So the ops community could be somebody in the data center. It could be somebody in our jsock, you know, in my day job company. It could be a person doing 10 testing, it could be just about anybody who's not got clear milestones and sort of product release centered work. So that covers actually a much bigger part of our world of automation than most people think, for example, in in hospitals right now, a lot of the stress on systems is just scaling up existing business as usual, you know, admissions workflow of prescriptions deployment means that they're not releasing any products, believe me, that's, that's not the main concern right now, but there is a lot of stress and use of processes like the ones that safe is trying to address in systems like that. So you see it in government a lot. All companies have these kind of BAU processes that are separate from that. And one thing that Dan and I've talked about in the past is, you know, maybe for some of these things, the cyber liability engineering is the better approach but on the other hand you miss out on some of the things in safe. If you bail out of the safe principles and try to solve problems there so that's something to come to at the end of this talk when we get there. And just to give you an idea of kind of where this approach is maturing into. I've only certified as a sort of simplified semi dummy Agilist, and actually on 4.5 which is the last release of this product. And you can know a lot more about this obviously by you know seeking out these other certifications which deal with different facets of it so it's probably debatable for people on the outside of safe. Whether all these things are necessary but it is necessary if you think of if you buy into this which is a lot of big companies have you need to kind of specialize in different facets of this so I think this is kind of a little bit outside of the ops question unless you say wait a minute. Aren't implementing this thing and you look at the things involved in this it really is not developer stuff so maybe it is related to ops in some respects. So the core values they like to talk about are worth reciting here. So by alignment they're trying to say you know align the principles across the teams integrates fully across all of the aspects of the platform. Their idea of transparency is to have artifacts that are produced as part of the same process that let everyone see what the goals are the artifacts share things fully across the teams. A lot of what you'll see here is about trying to reduce cycle time. Expose what the features of new releases are going to be and the evaluations of those things so different teams can see what each people are what what other teams are working on. And also recognize the feedback from customers because the supposedly you get frequent feedback with customers with much smaller increments of effort than you would get in a waterfall. Execution and then the way that you execute the program may have very specific notions about this. Which I kind of boil this down to one slide which is kind of unfair but I do it anyway. So not this is really not a safe thing but I try to they try to make the point of what the focus of safe is by by contrasting what they they prefer so they would prefer individuals and interactions of individuals over processes. And tools what kind of ironic right since this whole thing is a tool or process. But they're what they're what they're trying to say there is if the customer says hey wait a minute you got this feature all wrong. Everybody needs to stop what they're doing on that team and recalibrate what they're doing in an agile way and turn it around instead of having everybody go in the wrong direction with that so they're saying that piece of information is going to surface that individual interactions. Then you let the process take over what you're going to do with it. So working software over documentation OK that kind of reveals the developer preference over the ops preference so you know that's an issue to consider. Yeah contracts are going to be secondary to collaboration with customers. So customers may not like that because they want to tie you to an SLA. But an SLA in this scheme of things is really not possible because what you want to be measuring is what you're able to produce in the increments according to what you agreed to in the program plan. So what those things are is kind of out of the scope of this conversation but it's the preference in this manifesto. Oh yeah responding to change right that's sort of a truism. Let's see. So this I'll let you to skim through this but if we look at this from an ops point of view. I think changing requirements that really affects operations considerations so I mean I think if you look what's going on with COVID-19 across a lot of enterprises today and I could just use as an example is going on. At synchrony we're having to put a lot of people in bring your own device settings and do it in a hurry and this is having to happen in a on a scale of you know 5000 users over a couple of weeks time. A lot of these people are non technical folks that need to be taught how to do that and to be able to operate from home on short notice. So the operational challenges around doing that are considerable. I think you could use a developer like model for that, but it's debatable whether safe really what supports that in a direct sort of way so we leave that as an open question. I think the face to face emphasis when when I was going through this year and a half ago. I was a real skeptic of that so I think I mentioned to Dan early on. It'll be interesting to see where the safe people take this now that face to face is meaning a virtual face to face thing because they really do mean people in a meeting if in a physical meeting if you see the pictures they they give you in this in this presentation it'll always be a bunch of people in a room. Now, not necessarily for the sprints and the stand ups, but for these product owner release train settings where the requirements are getting defined and where the relationships between the teams, which involve security typically are more prominent. There is more of a focus on sustainable development. Here they mean sustainable, not in the ecological sense, but I think it's useful to try to insert that dual meaning here so I always do that when I see this. Okay, so there's a little more to these principles. The one that I think there are a couple of these that I'll highlight here for for the purpose of this discussion. And the two that I would highlight our number six and seven so this thing about visualizing whip and it sounds a little technical, but there's really pretty good quantitative measurement support and support in the field for what they're after here and really the benefit over waterfall isn't just this kind of breakup of the serial left or right process or the challenge of, you know, trying to define everything in advance what you get with waterfall. It really is this problem of trying to reduce lead time and have the smaller tasks be more manageable. You know, it's basically this thing of having smaller pieces that can be recombined recombined and reconfigured dynamically gives you more flexibility and efficiency. So how you do that, you know, in and address security, how the security overlays fit into that specifically is what I try to deal with in later slides. So this is one of their slides. And it kind of begs the question, you know, if you look at these silos across, you know, here on the right side of the slide. This is a challenge right these silos are real operations is sitting there in a silo by itself. Here we are security, we're in our separate SIG, right? How do we interact with each of these silos? How does safe address this? You know, they tackle it, whether their success or not is really the question we want to ask. So the Deming Cycle is this plan to check access the way I tend to remember it, but this is a real basic capability, what a process underpinning for the safe process. I think one of the things that really get right about this. On the other hand, think about how do you do this with your whole enterprise or your whole product line? You know, like we have our retail card operation or our commercial bank, you know, we want to change a whole checking application. This could be a thing that has, you know, really tight integration with third parties with, you know, some of it might be mainframe based stuff. Some of it is CNCF products that are vertically integrated with other tools. This whole thing exists as a working artifact from an operator's point of view. How do you do plan to check act for this gnarly mess? The other thing they postulate is this notion of the program increment planning. So we have these things, it happens. As you can see, they'd like to depict this as a physical meeting with, you know, people writing stuff up on whiteboards and, you know, having face-to-face discussions about what requirements ought to be, teams taking responsibility for different things, challenging stories that people are offering for how long they think story points ought to be represented. Stakeholders, you know, need to be put into what, not direct conflict, but at least direct negotiations, you know, that if you try to do that in email or in, you know, traditional requirements writing through technical writers, they argue that can't happen. You really want them in a face-to-face setting where these things can get challenged and negotiated. You know, I think they're on to something with that, but, you know, I have some questions about it. So the questions are on the right part of this slide. Stuff in italics is, you know, requirements engineering depends on story fidelity. Do people know how to write them? That's a big question in my mind. Story fidelity is more art than design patterns. You know, can you really use the design patterns you get? Like people on this call, I think, are really highly proficient with design patterns, both from being coders themselves and, you know, from this combination of academic training and being good practitioners and learning from mistakes. But writing a user story, that's another thing. How do you get better at that? How do you recognize a good one? And really the whole thing drives from this story fidelity problem. And there's the other problem that security is kind of a bit player in all this. Privacy and compliance maybe has a bigger role to play in some applications, but it's more to say on this particular issue in later slides here. SAFE has this notion of an architectural runway. You know, you could argue that CNCF, you know, has its own kind of architecture runway, which involves trying to leverage existing, you know, CNCF design patterns or maybe, you know, guidelines for how to move from incubator to, you know, full acceptance into the framework. It could also mean, you know, there's a de facto habit about which, you know, already fully embraced tools, i.e. popular tools that are in the CNCF family, you know, that are part of the runway. So that could be seen as, you know, either features in which you import the tools and the APIs from things in the landscape. Or you could be, these could be inside the enterprise where features are being brought to you by practitioners that are advocates for particular applications inside your own enterprise. So, yeah, what this means, this is kind of another challenge, I think. They also have this notion of what the value stream is. You know, I think I touched on this delay based optimization. So this matters for operations people, right? They're mainly, when you learn about this, they mainly are going to give you the analogy of developers, you know, waiting to get specifications or developers waiting to get feedback from testers, or information from other teams that these delays, you know, introduce problems that you can only address by making the queue sizes and the work in process quantity smaller so that you can manage it. I think there's something to this, but the question is how do real operation issues like latency, you know, scalability, robustness, you know, team constraints like what you do at the end of shifts and how do you manage when you're doing migrations or tool upgrades, which we see this a lot in security tooling that when you want to do a rollout of the new version of a tool, that has a big ops impact. So how do you model these things in safe is sort of a question. Continuous exploration is, you know, as you see in this chart depicted as, you know, one of the roles in this, but I think it's a little fuzzy and safe how you integrate this. There's a role for, for R&D, you know, like going to our friends at NYU or going to the professional associations like IEEE or to NIST or to the consortia to figure out how to do, you know, better ways to do this. Maybe you look at, you know, other CNCF tools and say, see, how did they solve this and tough or how did they solve this in, you know, one of the telemetry projects that, you know, it's got a solution space. It's slumber the one I'm taking on. I think this is a little fuzzy, but this is kind of a key thing for ops because I think one of the more powerful principles here is the design patterns. So cadence and synchronization is the other thing I want to call out here. I'll let you skim this, but cadence and synchronization, these are really important for operations. You know, it looks like a fancy expression here to call it this, but sometimes it's like, you know, the shift boundaries for when people come to work and when they hand off work. Sometimes it's incident based considerations like, oh man, here's a zero day associated with a struts vulnerability, and there's a whole bunch of CNCF projects that have this in it, what do we do? You know, now you've got the fresh involved in it. You have ops people who have to be taken off their current roles to address that. So the cadence and all this stuff, you can you mean that you can implement safe to try to do some of these things, but it's really different from a release driven planning session. So we're, you know, in the sec DevOps world, we are a little struggling still with how to have security teams implementing safe to do this. It's doable, but it's not always a good fit. So if I had to pick one thing out of the whole safe thing, it would be this, iterate toward the sustainably shortest lead time of the best quality and value. So I think, you know, stick to this principle, try to apply it to ops, this is probably going to be the most help. So I hear this a lot, you know, should we do security first. So this is me in the slide. I really feel strongly about this that one of the most pernicious things that gets said is security needs to be designed in at the beginning. I think that is so upside down. The thing about safe is the stakeholders in applications are the people who need the applications, the customers, unless you're building security tool, don't want that. So I really wanted, you know, turn that around and say, look, it's, it's the applications that rule and in the ops world, it's whatever the operation is that you need to support. So if right, if right now it's running the, you know, the supply of ventilators in a hospital, it's the, you know, the workflow for that, it's the queue management, you know, it's the staffing related to running those operations. You know, it's the shift handoff, it's the security around, you know, around new processes. So it's those applications. It's, it's not the security. So not to say security isn't important. It's just that it takes on a different role than what we're used to. So operations, you know, maybe jumping to head to the last point on this slide, operations can work as applications, but it's not a one to one mapping. The point really here is that security comes later. Let your, the point of this is let the, the, the teams, the application teams say what they need from their customers and the stakeholders bring in the security and privacy aspects of this later on through the architecture enablers. So, so my reality check on this is, you know, 20% of the defects remain after you do static and dynamic scans. We don't know how to produce bug free software so get over it. The goal of trying to produce secure code really subtracts from our more important objectives, which are sustainability, manageability, risk, usability, maintainability, and what the customer needs from the application in the first place. So that's where safe says we need enablers. Let's see, we got 12 minutes or so. I'll wrap this up in five minutes so we can talk about it. Okay. So how yeah, go ahead. No, sorry, just degree understood five minutes and we'll call time. Got it. So how does safe support this frequent iteration automated testing shorter sprints trying to left shift test development. Really trying to make security be part of the quality dimension. So that's really the movement. So this is this is me touting this. Most I think this insight is external to safe. Some of this needs to happen at the sprint retrospectives. The enablers and different enterprises have different life cycles you may have to bring these into your own projects. Some of this is, you know, well, what is cut and paste for security? And what is the role of domain specific languages and, you know, bringing that into play into your security teams. Test engineering, you know, the new way to think about this is security is test and vice versa. Frequent demos should be including test integration. This means usually left shifting to developers trying to build this capability into the integrated developer environments probably clips more tagging and annotation. I'm going to skip this one in the interest of time. I think we tend to underestimate the securities of specialization. It's not covered well in college level software engineering. It, you know, we really ought to think about it the way the rheumatologist thinks about a neurologist. It's an adjacent specialization. They're both in medicine, but neurologists don't pretend to know what rheumatologists know. What that means is, you know, your average developer is not going to be a security specialist. We shouldn't expect that. So likewise, from an operations point of view, we're not going to understand what Akamai or Palo Alto firewall specialists are going to do. Also, you know, some hardening is going to require aggressive red teaming. That's really kind of a new concept. I don't think safe is baked that in. It offers a paradigm for doing that, but they don't really give us a good roadmap for it. A key thing for ops integration here is how to figure out how to mix legacy and software defined data centers under the same framework. So I'm going to rant here against teaching toys like Zero Trust and trying to do what Google can do and, you know, even PayPal will call him out here. You know, our synchrony, you know, companies of a certain size can do things that most organizations can't. In fact, most CNCF projects probably can't operate that way. So part of what we need to be thinking about our security tooling as performing for us is supporting decision support. So what we mean by that is, you know, being fully part of partners with quality writ large providing telemetry into support decision making. Sometimes that'll be for security incidents, but they could be other kinds of things. It could be that, you know, we need met a models to support our scalability or simulation goals or it could be to teach how things work so people can learn how applications are designed. We may need to be able to integrate risk in a more systematic way into decision support tools. So to do that, you know, there are operation support tools that are out there, you know, a building an integrated integration with that with our, you know, existing CNCF applications is maybe something we could think about on a more systematic basis than we do now. You know, I got a whole other comment presentation about security and big data from my work at NIST. I think we need to start thinking of application data secondary to security data that, you know, for most applications these days, if we could we would be gathering more data than the application itself is accumulating. So the question is how do we do this security analytics moving closer to data science than just purely, you know, sending log in log off and, you know, resource consumption logs off the splunk. There are big issues on human computer interaction for operations. You know how we use repos how do we discover what's in repos, what the design patterns are for operations. You can do these things in sprints but the problem is they're often not introduced so you know ask yourself how does the CNCF community get brought into the ecosystem. Do we have an ops repo is this is this even a teachable thing. Hey, Mark, I'm just going to tell the hard stuff. I just wanted to follow up on a question I posted in chat. Does anyone have a right to want to bring up for the open floor. Yeah, yeah, let me just, let me go to the last slide here, so that we've got the context stuff so I'll put this up on slide share at some point, but we can stop here fine. Thank you, Mark. Thank you for coming back to this. Are there any open for topics or does anyone have anything that you want to bring up with with respect to Mark's presentation. So this is Vinay here I had a question. Mark, I think you're the story that you're trying to convey is well received so what is the next step I mean what are you hoping to achieve I mean how do we incorporate this is something for CNCF to try to think how do we develop our projects is that what you're trying to convey or what is your broader strategy on evangelizing and also from an adoption perspective. Yeah spot on and I don't I don't know the answer that what I think is, you know, this group in CNCF is thought leaders, you know, both by deed and by actions. We'd like to see us take some of the principles, you know, it doesn't need to be the whole thing either right. You know, find a role for CNCF projects some of them in particular, you know, like tough have a role to play in this. And, you know, but maybe have some artifacts, and you know maybe periodic FAQ kind of presentation sessions. You know, people who come through this process could review some of this and say, okay, I see how I can contribute to, you know, decision support or making better user stories or integrating, you know, safety aspects into future standouts because, you know, the review that we get for accessioning of, you know, through the CNCF process that I'm aware of anyway is only going to touch on some of the security elements but, you know, I'm, I don't have the whole answer to that question. What do you think Dan. Yeah, I mean just just tie back, you know, how we got here to have discussion. This this came out of, you know, discussions that we had around operators and extending, you know, the work we're doing in the CNCF and especially, you know, getting to help and facilitate in the journey of some of these leading edge. You know projects that are in the CNCF. You know, many of them are created in a greenfields environment they're, you know, built in cloud native by distributed teams. So, like, you're seeing organizational and systemic advantages. Out of the box and some of those things and then you, you know, try to apply those practices to large organizations that don't have that luxury that they come from, you know, situations where you can conceivably get everybody together and you know, do some hard negotiating to work out all the differences. It's an extraordinarily messy process asynchronously or synchronously even, you know, it's exacerbated, you know, asynchronously. I think that, you know, any leader that has the opportunity to take the simpler route to deliver a more high fidelity answer will take the simpler route. And that is totally, you know, that that in person, you know, quorum. But, like, it's hard, even inside of an organization these days to, you know, gather enough folks to drive meeting, you might get one meeting, unless you have, you know, huge pressure from top down that everybody has to push through it. And, you know, from, you know, one of my opinioning hat, you know, the reality of my experience is it takes more people effort, not less people effort. And that's the existential challenge that, you know, we're seeing in moving to broader more distributed virtualized environments is, you know, we're looking for easier solutions with less resources. And unfortunately, in my experience, the answer is it's more time, more effort with more people to, you know, get the right answer in that distributed environment. And I haven't seen the leadership see changes with leaders that know what a distributed reality looks like and what success looks like in the distributed environment for me to, you know, really, you know, see that that existential challenge pressure coming off of it. You know, folks are going to keep pushing to the easiest solution, you know, in person meetings while, you know, other forces are pulling in the other direction and it's great to have, you know, context, but, you know, we're going to be battling this for, you know, quite a number of years. And thank you both. Yeah, I agree. It's amazing how we battle these things every day and it's just so hard to organically bring that security mindset and change. Yeah. Right. Yeah, I think, you know, they, the thing is about these composite whether we call it, you know, sprints or composable services. They can bring small things into these meetings as long as they're doable. So, you know, some of this is as simple as, hey, if you look at the API and point to a CNC F project and say, you know, here is a design pattern for solving that. And what you've done is sort of bring in a whole paradigm as part of the solution. Yeah, I think Vinay, you're asking the right question here, which is, you know, how do we do, because it's a teaching enterprise as much as anything, you know, how can we leverage what the knowledge base implicit in the software products that are in the suite here. But at the same time, learn from the people who are using it right to let them use these big frameworks like scaled agile to for deployments. There needs to be a catalog, you know, of resources that like big security could host or, you know, catalog, I don't know some combination of those maybe. Right. Yeah, I think it's, you know, great to put these things into perspective and definitely want to balance out our sort of future looking. So work with some of the, you know, hard reality applied on the ground understanding of how we put this into practice. So it's been a great perspective mark. Thank you. Matthew, our facility meeting facilitator had to drop. So I'll close this out for today. You know, during the meeting, I did hear from Sarah Allen. So happy to hear from her. She was in the process of going and evaluating, getting COVID-19 testing, you know, for better or for worse, she was too healthy to get tested. You know, a number of grumbles with that, but that's where we're at collectively. So, you know, she's, you know, healthy and doing all right, you know, stuck in Boston, unable to, you know, return to San Francisco with the rest of her family. So I again apologize for any delays that we may encounter in managing through everything. Our first priority is keeping everybody healthy and safe. So with that, I'll wrap up for this week and I hope everyone stays healthy and safe and see you next week. Thanks. Well everybody. Thank you. Cheers. Thanks. Take care.