 Hello people, welcome to open and secure lessons learned from tax security assessment. My name is Andres Vega I'm Justin Kappos We're both part of the CNCF technical security advisory group and we're gonna talk about the lessons We've learned what we extrapolated from them over the last five years of Doing security assessments across different projects in the ecosystem About 20 minutes ago We filed the PR for a book we set self-published Justin did all the writing and collaboration with some of the other tax security members The book encapsulates a lot of the structure of an assessment. How do we go about it? How do you participate? What are the expectations and how can you effectively and efficiently? Collaborate with us to assess your project. There's a lot of value in these for one a Well-explained project it's a stance at security properties and guarantees help lower the cost of Running the software and getting the software integrated with other projects as well as help users better understand What is it they're consuming so it helps? Accelerate traction and proves the adoption of the project and ultimately helps to safeguard the ecosystem as a whole There's a lot of real-world examples. There's a step-by-step guide on completing these assessments a lot of lots of References to resources and tools very opinionated takes from other experts That you can leverage and take into account when you're going into one of these well, we really start off with security by and large is misunderstood for a number of reasons but Following a certain set of heuristics and an approach you can be Methodic about it and you can achieve Expected results at least on how to have more meaningful conversations using common Language and vocabulary to get buy-in and acceptance for the consumption of the project with that Justin is going to take us through some computer programs. He just wrote in preparation for this to help illustrate that point All right. Thank you, Andres Okay, so, you know as a professor at NYU One of the things that I do is I get to teach and so I'm going to teach everyone Some easy ways to think about security at least the way that I teach it to my master students and others who Come to learn this so Failures are random events. So a way to think about this is here I've written a chess program that what it does is it makes random moves So there's no logic in it. It's just literally doing whatever the heck it wants And so if you think about the real world you think about the probability that it's going to rain or the Probability that your car is going to break down or other stuff like it. These are random events. They can happen They cannot happen. Yes, some things can influence them, but it doesn't you know, it's it's a random process however It's not too hard to come up with strategies that mitigate random events It's actually quite easy to do this So here I wrote an algorithm called furthest that plays the black pieces and that algorithm by the way has already won The game is over with here, but it's really simple. It just says if you have a move that's checkmate do that if you don't do that And and you have any pawn you can move move the pawn that's furthest down the board that you're able to move forward and If you don't have either of those things true then just make a random move and this is 26 lines of Python code and It wins overwhelmingly against the random adversary and does very very well. So it's a very simple program Okay, and yes, this is not like threats in the real world. This is an effective way to combat randomness okay So now the difference in in computer security is that you're not facing a random adversary You're not facing the weather other stuff like this There's a cunning human mind behind this that's devious and trying to figure out ways to attack your system It's studying it is thinking about it is going after the thing So let's try to look and see what such a cunning adversary might look like in the real world So this is my nephew Dylan He's five years old nine months height 311 42 pounds favorite food is hamburgers and He learned how to play chess. He knows most of the moves in chess not all of them But he's gotten most of them down and importantly here He knows how to anticipate his opponent's strategy to some degree Okay, and so now we we see what happens when we place this intelligent adversary Dylan here Who's playing as white against the random algorithm on the left and the furthest algorithm on the right? So in both cases here Dylan is going to have no problem checkmating and beating these programs Right even you know this strategy that was overwhelmingly good against randomness. He can defeat it No problem super simple for him Easiest game of chess of his life, right? And so this is why when thinking about an intelligent adversary you really have to think about things in a very different way Okay, and so when you do things like a security assessment like you're trying to assess the security of a project You have to understand many different aspects about that I'm going to go through these in a little more detail in a moment But you have to understand the scenarios like the effectively the board on which you play the game You have to know what the pieces are how the pieces move what causes someone to win What things you don't care about and other aspects like this? So for a scenario standpoint It's really really important when you think about anything computer security or anything else To think about the intended use cases of the system like when is this supposed to be secure under what? Circumstances and so on so if I say to you my car is reliable What you think is you think the car can get me to and from work day after day without having problems But if I go take that car and I try to drive it through Lake Michigan And it fails to make it across You would not say oh well that's because your car wasn't reliable. How could you have lied to me, right? A car is not a submarine So it's not expected to perform well underwater or on the planet Mars or something else like that Actors in a in a system, especially when we think about cloud native systems actors are The different things in your system either system administrators or pieces of software or other things like this that go and perform You know the important things that are going to be done in your system And so these are analogous to the chess pieces on a chess board They're the things that you need to worry about and you need to think about like it basically What these are and who they're controlled by And then you also have to think about actions These are the moves that chess pieces make in a chess game in cloud native systems You need to think about oh, you know well My identity management system talks to this database and it retrieves this data from the database and then it uses Some function to write out statistical information To a log file somewhere those are actions those are things that are actually happening with the system And so it's really important to think to understand how your actors Perform actions in your system and what in what ways these happen And in another key part about this too deals with things like how isolated your actors are from each other When they're performing actions and doing things like this because if you can compromise One actor and that allows you access to others that can snowball and present all kinds of security problems Also when you have a system you have goals you mean for it to be secure in certain circumstances You want you know in chess you're trying to prevent your king from being captured from being checkmated And you have to think very carefully about what your goals are for a system in something like a cloud native context You likely have very a very large number of goals And some of them are more important than others and there's ways You know once you think about this a little bit to go further and reason about how you prioritize What you look at and what you worry about but you know in general you want things like you don't want your kubernetes cluster to be taken Over you don't want your user data to be leaked. You don't want your you know ML models to be Exfiltrated whatever you're concerned about in your case And then you also have non-goals like in a game of chess, you know, you're probably Not worried about making the prettiest pattern you can as you move your pieces around the board in you know in Cloud native systems you also have things that you're just not focused on you just don't really care if You know someone goes to your login page, but can't log in and they just keep sending You know like I don't know just browse your login page or look at something like that You might not worry about that as much And so given this now I'm gonna talk about Two different ways that are very common in our community that people use to assess security We've talked about the chess board the pieces and stuff like this now We're gonna kind of put this in context and talk about different mechanisms that we can use to You know actually get actionable things out and there's two primary ways that we use in this community One is a security audit and the other is a security assessment And I do want to say that these terms are not universally used in the way that I'm going to use them here But they're used like this by most people in our community and are also pretty commonly used like this in practice So I'm going to use them in an opinionated way so an audit is where you have a Group go and actually try to attack a system and try to exploit it and try to come up with Vulnerabilities and other issues that directly impact likely a specific deployment of a system or a way in which a system works and So you know this is very much a real-world example of can Company X get into thing why and if so what are the ways in which they're able to do this Whereas in comparison a security assessment Is looking more at higher level aspects of your security architecture? So it will point out things like for instance that some of your actors aren't well isolated from each other So this would make it easier if there's a vulnerability for one to move Laterally in your system and compromise more sensitive data or that your code review practices don't seem to be very good And so you're likely to have many vulnerabilities and problems in the future because you're just not doing a good enough job looking at that aspect of your system and As a result if you kind of look and compare them And by the way, you know, there are groups that do some of both of these like the best audits will have some assessment aspects And the best assessments will some you know touch on some audit Aspects, but I'm kind of drawing a little bit of a line here for concreteness where audits find and identify demonstrable concrete issues and They're directly related to exactly the tooling and tool base that you're using So if you have an implementation in a language, it is going to look at that And if someone is writing another implementation in another language, then basically none of that is very likely to apply And also the results that you get will very much vary between different audits you can even have two groups that would audit the exact same thing and They each might find you know about 10 issues, but only five of them might be common issues Right, so you don't really know, you know when to stop or if you found all the sorts of things That you might need to so security assessment Uncovers higher level problems that are causing these lower level problems to occur in general And it tends to be a very long lived like long-term thing You don't need to redo security assessments every few months It tends to be the kind of thing that if you're doing the right practices and you're doing stuff like this That as long as you're continuing those practices that assessment can still be valid over many years of time And so it really provides you like enduring long-term guidance and One negative aspect though is that sometimes it's difficult to go to management and say look like there's really value in us making this change Because you're not pointing directly to a CVE in most cases. What you're doing is you're going and you're saying Hey You should change this thing because there's huge classes of problems that can come up. We can't tell you, you know What they are We haven't exploited any but any problems here will cause you problems or any problems here will cause you problems and this addresses it So now to continue the rest of the talk Andres will come and talk a little bit more about some of the tooling and things like that that we use for threat modeling and other aspects of the system so when embarking on a Security assessment, it's important to follow a methodology to enumerate the attacks There are a number of frameworks out there Lyndon stride rapid risk assessment. We find stride to be a useful baseline. It is important to know that Well across across all methodologies and why via the importance of the virtue of threat modeling is security researchers know that it's a good way to elevate security concerns and It's hard to miss the the obvious risks now when looking at things like stride, which has been used for a long time Not all of its primitives may translate well to cloud native or modern distributed systems For instance, the notion of escalating privilege is better thought as lateral moves. So when Justin was talking about Isolation across cross boundaries. That's what you should be considering if an attacker gains access to a particular component Are they able to get access to another component from there? So a failure to sufficiently compartmentalize one component to another would lead to such Privilege escalation, but more so it's a lateral move in practice And there's other properties of systems that are API driven use in tokens where if you think of spoofing may not apply in a traditional sense, but anyhow it serves as a useful baseline Another great resource to count on as part of your toolkit when performing a security assessment Is that of an attack graph and a track attack graph is a very helpful visceralization To understand what is it that the attacker is after what is their ultimate goal and from their backtrack trace What are the potential pathways that we used to navigate to get there? So it is Good to think like very carefully and focus on if you see on the key and in the diagram If this were the highest of a bank opening the safe as the root node You will have leaves or branches that are the different methods of attack the attack vectors and the relationships Which are potentially a lot pathways for lateral moves and How could those could be disrupted? Now when when you start reasoning about risk Not all not all compromise is the same in a system. There's certainly a lot of things you can Consider within your threat model But in many instances the attacker will be limited to Just access to one component and others they may have entire control There are exceedingly rare events if we use the analogy of a meteor hitting planet Earth It'd be catastrophic, but it's highly unlikely But even if it did there, it's not much we could do at the same time There's there's events that may be very probable Higher your currents a kid taking more of it more the candy that they are to take from the trick-or-treat Or a customer taking taking yeah free giveaway candy at your store So you need to think ultimately like the ultimate impact is it monetary as a reputational? What will be the consequence of that action having occurred? Often it's hard to calculate or quantify an exact value for it But you can try to service. What is the cost? What is the total cost of a key leakage or data acceleration from our system now? More on more in calculating that that risk a useful formula is Risk is probability plus impact or risk times the impact It is important to focus on once again plausible scenarios over rare catastrophes or inconsequential events An example yet again a critical vulnerability leading to a widespread data breach say a reverse shell Where a lot of data could be vacuumed out as part of the assessments we've done We produce extensive attack matrices that capture a lot of information About the actors so these are from a spiffy inspire Security assessment one of the earlier assessments that were done. You can see the actors They are servers agents containers and a deployment you may have an ha deployment, which will have a number of servers What does that mean? What's the relationship from the agents and the containers that those agents are deployed in and how what are the dynamics between all those? different moving pieces and That leads into helping visualize the trust boundary which in turn informs the the attack tree ultimately and you can Assign vulnerability scores or criticality scores, which yet again can help prioritize fix fixes and your security roadmap All of this is captured and the book that was published just now in the security repository GitHub comm slash cncf slash Tag-security there is an assessments directory. You will find it there Justin Took the initiative of capturing everything that's been learned on the assessments and how we can continue to iterate upon it Draws in the collective wisdom and experience of many different people Raga also part of the tax security Andrew Martin Ashutosh Narkar Dan Lawrence Jack Kelly and Marco the Benedictus The book is available as early preview. We were just like Doing it in adobe and designed the last couple days. So we would love to get your feedback on it things you can expect to find in the book are How to engage in the assessment process and what's the way to go about it? So the structure to it is it starts off with the project team. I see the Argo folks in the room You would start off by Riding down to the best of your knowledge the security properties of your system and their architecture we have a template for it and The guidance around the template leverages a lot of the frameworks we've discussed now to this point From there on you submit it to us Justin and myself will work along with the community to identify security reviewers that will jump in and ensure that there's a good cross-examination of the material There will be a period of back-and-forth. There's a lot of refinement during the process You you may make a claim and we may ask you a question Such as when does that does not hold true for instance to understand the different failure modes or the corner cases? It's an iterative process your response will be merged back in and to that assessment document So that the question doesn't arrive and doesn't arise in the first place to begin with Once completed and signed off we share that with the technical oversight committee With your project constituency and your end users and it's available for posterity and the tax security Repo you may also decide to check it on your own project repository Now if we look at it as a fairly well-exercised process where there's a lot of lag and We we have typically deaf difficulty to get going is in the initial phase and Helping the project understand How much do they need to produce to what level of detail? We have a number of assessments and the key right now that that's where we're seeing the most attention is required fortunately, we're kicking off an initiative and collaboration with New York University Justin has over 110 students that will avail themselves To work alongside the projects to produce that self-assessment draft draft The general plan on how we're gonna go about it is you connect There's a preparation stage really quick set up getting to know each other point them to the right resources They'll perform about one to two days understanding the intricacies of the project and Where does the project fit in the ecosystem and relationship to other projects from there on they'll take a stab at Filling out the template and producing all the information we solicit And then they will go back to you and ensure that It's tight and that it's enough and it meets the bar for what tax security expects and from there on well We pick it up and engage with the rest of the assessment process It is launching right now with 27 different projects Which is almost all of the incubated and graduated projects that have not gone through a security assessment to date We are always recruiting a Participant participants and other reviewers to come along pitch in and help us with the assessment It's relatively a low bar of entry Let us know you want to be part of it Indicate so in any open issue for outstanding or pending security assessments You do not need to have participated in prior assessments Just let us know you like to come in and shadow and observe how is it that we work with And what is like how a stringent and how a scrutinous the process is All questions all interactions. There's a section of like there's a face of naive questions There's no question that is dumb all questions are valuable and helps us help us surface and elevate Information that should have been written in first place. So it really enriches the whole process as a whole You should expect the commitment for about two or three weeks Total work time about two to three days. So something like 16 to 24 hours and from there on having having your badge or having your bit that you were part of the assessment bit You likely be someone that we look into the future to help us lead one of these assessments So, yeah, we would love to have your participation and from that Let us know about ways that we can improve it and that we can make it more impactful more meaningful Typically maintainers love it We get a lot of praise from the project teams. How much better we help them Articulate the defenses of their project or the dynamics of the project. It helps them position the project and come deliver talks at kubecon or Assist their end users with security reviews We do not hear from end users all that much though And we would love if if you're an end user in the audience Making use of any project that has already gone through an assessment We would love to learn how that assessment has been leveraged and my own personal experience with the spiffy project Is something that aided us when there was perhaps some friction from the platform team Convincing the security team so equipping them with a material that can ultimately inform controls hardening guides has been extremely beneficial It often also helps convince the business of what is the total cost of ownership if it's a product project That is fairly watertight or bottom up in terms of security And it also eases the short ends the time from getting from zero to production with with the project So yeah, I would love to hear from you engage with us It's hard to state or hard to quantify how much it Races the awareness it raises the defenses and it's also you can you can acquire skills from Threat modeling that you can incorporate back into your day-to-day job and also future projects Here's a list of the ecosystem projects that have published assessments build packs cloud custodian harbor and toto key cloak Coverno open policy agent pixie spiffy and spire Huge heartfelt. Thank you to the maintainers that have worked with us to help us achieve this Milestones and everyone else who has contributed along the way it's been truly a monumental effort But it's as neat to see this assets that are a great resource for everyone in the community With that Justin if you want to come back on and if you want to add extra color in the book or pointers You're shaking your head, but I'm certain you have something so it's great I'd be happy to take any questions if anybody has any. Thank you. I guess your terminology of assessment and Audit does that match CNCS usage like I've found these on github and they they appear to be assessments than mostly match your terminology Which are typically not point in time, but I think kind of are so there's a couple questions They're like how is this handed off to the the projects to carry on and Right completely separate than an audits. I think I've seen that are cnc Associated right. Okay. Yeah, so there's there's two questions. I'm gonna repeat and try to disentangle the the first is Is this terminology of audits and assessments completely uniform across CNCF and things I think the answer is possibly not, but we've tried to push things in that direction Assessments are point in time as you pointed out and then the second question is what do we do as these things evolve over time and For the most part we've been focused more on trying to get everybody to have some baseline at some reasonable time since these Take quite a long time to change But we have had discussions about Refreshing them when we get to the point where all the incubating and graduating projects have them then we might start to reach out To projects that we know would benefit some projects. It doesn't really matter Even if it's been five years some projects had a major redesign after a year Or or specifically any of the projects if they picked up the assessment and like maintained it on their own I Don't I can't I don't know that any have made substantial changes I think we've have had some questions about it But I do not have the best memory in general and don't recall exactly an instance to tell you hey in an organization the software you're building will evolve over time and So so what would be good trigger points to reevaluate your threat model and to update your model Okay. Yeah, that's a very good question There's a bunch of them. One is is that you're adding new actors You're adding wildly different scenarios You're adding new actions into the system those types of things might require some work, but it really depends because for instance, if you're adding a You have a system like tough and which supports RSA dsa ed 255 19 signing and you add in New form of like crypto signing algorithm. It's probably not that important to assess it But if you're radically changing parts of the design and use model to support different cases, then it's a lot more warranted So it is somewhat subjective about how to do that Also, if you did something like had a new Product team using new ways of doing things using new new tool chains and stuff like that You want somebody to look at that aspect of what it is you're doing So if a new implementation comes along Having folks focus on just that little part of the assessment, which is is really small. It's only usually a couple paragraphs is valuable Thank you and also If you if you have a huge system that you haven't threatened before How to go about that because and I've been working in in organizations where we have been developing systems for four years and and they They have become quite large. So Yeah, and and and the idea of starting to fret model something like that is, you know The first things the first thing that comes to your mind is that it's gonna take ages. So so how do you go about? It's it seems scary so you can do it at different levels too, right? You can think about like what are the big components in the system and how do they work together and what do they roughly do? And even if you just kind of write out your actors your actions your goals and on goals that initial part in the scenarios You'll often spot things yourself like oh my gosh You know if there's a compromise in this 80% of our system. We are totally in trouble Right those sorts of things are really good to know and then going through the more detailed parts of it where you get you know the second half after the self-assessment is that Experts from our team will go through and really poke at things a bit and give you stronger recommendations and more opinionated takes about how you can get to a better place And writing things in a level of detail. They can understand it might take more time So but at least doing some initial kind of self-assessment like work yourself will probably like you can kind of find your own mistakes Through doing some of that Thank you. Sure Hi, so are there any plans for a physical copy of the book you've made and will you all sign them for us? I Will have to see about this physical copies. I will print one out for you This is my PhD student Marina, so Yeah, I don't know of course I'd be happy to sign copies of the book and do things like that But yes and one more thing I want to ask you as a tag security chair, which I'm not I'm a lowly tech lead is It might be a nice incentive for projects that complete their self-assessment to get like a speaking slot at Security unconference or security hub or security, whatever in upcoming cube cons. What do you think about that as an idea? Yeah, I think that's a great idea. I think definitely highlighting the The successes of these assessments and the projects that really do a lot of work to make them happen. I think that would be great great Okay, well, that's all of our time. Thank you everybody