 All right. Hello everyone. Welcome. Thanks for coming. I know we have I think five or six competing supply chain security talks So thank you all for choosing this one Or people online. I guess we're coming back to it So yeah, so I'm gonna talk today. I'm Marina. Sorry by the way, I'm a PhD candidate at NYU I've been working on tough for Five years or so and a little bit of in total as well My co-presenter listed on the schedule. Unfortunately could not make it today. So you just have me Yeah, and I'm gonna talk to you today about totally tough Some new tooling we made for simplifying And and secure a supply chain secured. Sorry. That's secure twice, but you know securing a supply chain So in case you have missed it these supply chain software supply chain attacks are a Thing that we've heard a lot about Software supply chain one definition here is a collection of systems Devices and people which produce a final software product This is the steps to making the software An attack on the software supply chain Is when one or more weaknesses in the components of the software supply chain are compromised to introduce alterations into the final software And we've seen recently a large increase in the number of these attacks in the world The report for 2022 Released by sonotype had a 700 and 42 percent increase of these attacks And I'm sure when they release the 2023 report soon this number will be higher. So it's It's yeah, it's been happening a lot some examples From the I'll go through these very quickly just from the from the news Seeing these a lot compromises to your repositories to Source code to Updaters etc And the cncf actually has a very good list catalog of software supply chain compromises With a bit of a focus on cloud native obviously, but it's a nice kind of general overview of Some attacks we've seen in the past 10 or so years There are many solutions that many people have proposed to the problem of software supply chain security We categorize them here into kind of three general Categories the first is evidence gathering These are projects that focus on you know Looking at what's in the supply chain now and doing so you can do stuff with that information So it's projects like salsa or projects like the The espom efforts where you look at you know, what's what's going on in the supply chain is information um discovery So it's evidence gathering information discovery is um, you know getting more more different types of attestations more information About the software supply chain. This is things like Sixter adding signatures things like guac kind of combining lots of different information sources And finally we have policy and validation. Um, this is taking the information that we've discovered and seeing what You know making making choices about what you should be using what what you should be doing with the stuff in the supply chain and one common thread That you'll notice when you look at all of these is that in toto can kind of link them all together so in toto can be used you can take this evidence that you've gathered and kind of put this into the in toto links and and um, you know collect that data in a In a like in attestations in a way that you can can prove You can then get those in toto metadata into your information discovery systems to learn about what happened in your supply chain um And then use in toto as well as a policy language to say this is what should have happened and then compare that to The steps that were actually taken in your software supply chain um And as an end user of course you can then use in toto so you have all your evidence that you've gathered On the the far right of this diagram. You have a layout that lists what should have happened This is your policy piece and then the actual package itself Put that together into a final product that the end user can then You know user not used depending on the the policy One piece that's missing from this big picture Let's make a quick in toto summary is this idea of distribution. Where do you actually get all of this information in this final product? And we want to make sure that not only that you know, they're getting all of this But they get the correct layout the correct policy at this point in time Um, yeah, that it's timely So you don't get a replay of you know a policy that was valid on one day Might have a compromise that's discovered and then a week later that policy Is invalid and we don't want to be using it anymore How do we know that this this information that we got came from trusted users? How do we know who these trusted users are? And how do we ensure that this whole system is compromised resilient so that even if anyone Anyone thing goes wrong anyone server is compromised even when he is lost. We don't um You know Compromise like everything in the system right we minimize the effects of these compromises So i'm going to introduce an idea for how to solve this problem called tough The update framework or tough is a um, it's a solution that was originally designed for secure software updating It's this idea that you can have a compromise resilient software update delivery and and updating um It's designed, you know designed from the ground up with this idea of compromise resilience So that we can reduce the impact of a compromise And if you can't reduce the impact you can at least allow secure recovery in the event of a compromise um It's designed with a few Basic principles, you know, it's defining it. Um, these are response responsibility separation multi-signature trust explicit and implicit revocation and minimizing individual key and roll risk It'll go into each of these so First of all, we have this idea of responsibility separation And the way this works in tough is that you delegate you have a different roles in tough and you delegate specific responsibilities to specific roles And so if a role is compromised only the responsibilities of that role are compromised not the whole system Um, and yeah, there's these different roles have a couple of different purposes There's the idea of signing that many folks are familiar with where you just are attesting to the contents of an update Or really any other piece of information And then this other idea of timeliness where you're testing that You know, that's that idea of replay attacks that we were talking about earlier, right? You can make sure that the software You're getting the metadata and the packages that are valid today and not the ones that were valid yesterday The next one is selectively delegating trust. So if you have a um, I think this is part of this. Yeah, this is part of separating responsibility. So you can say, um, you know, I trust Bob but I only trust bob for the alpha package that he's a maintainer of and that's the only package that he should be signing If there's another and then there's another user who's in charge of a different package They should only be signing that package just because like a user Maintains one project doesn't mean they should be like signing every package that you install necessarily and so yeah tough has um Basically method to say specifically what should be signed by which entity in the system Yeah, so so yeah, bob can't do food. You can do probably you can do alpha um We can minimize the individual key and roll risk so if Yeah, so if any one key or role is compromised you can um We use the impact of that compromise um, this is like a quick equation for the you know, the academics in the room how you how you calculate the The impact but the basic idea here is that um, if you have really high impact role You can use more secured keys and and you can use these high impact roles slightly less often do things like delegation So then you can highly secure the keys by keeping them stored offline use physical like UV keys and other Other technologies put them in a safe make it that hard to compromise and then have lower impact roles or roles that needs to be used more often use online keys So that you can use them frequently, but then make sure that you can then Have that all backed by these more secured keys Next we get this idea of multi signature trust So any one role is not just signed by one user with one key You can actually separate that even further So you have multi multiple different keys that have to be used to sign a single piece of metadata And so for something like a route of trust you can say we want actually five different people to have signed this before we trust it Um, you can have a threshold of the keys. So you can like three out of five or or something like that for multi signature Um, yeah, so you have a threshold of two signatures and make sure that like so if only one key is compromised There's actually no risk to clients because you're requiring that two different People or two different keys sign sign something And next we have explicit and implicit revocation of trust. So the implicit revocation comes when So like, you know periodically, right? This is like a the time of this idea, right? So after all keys in the system are going to expire you have to make sure that, you know, both um The the key, you know a lost key can't be Used in the long term for a compromise and also It's a nice way to make sure that that people who have keys don't lose them So in the event that something does need to change they have ready access to these keys And the explicit revocation is built into all of the roles in the system so that, you know If something is lost if someone leaves a team if something something happens where you no longer want to trust a particular key You can explicitly revoke that key and then through those timeless properties. We make sure that this is immediately evident to users Um, so yeah, so nobody so glad there was a very quick background on what tough is when in toto is So how does this all tie together? So tough, I think we talk a lot in in the in the tough world about using tough to distribute packages securely But actually it's a broader mechanism than that. It's just a mechanism for a secure distribution of stuff of Um of artifacts, I guess but um One thing that tough can be used to securely distribute is these um pieces of in toto metadata So these in toto layouts that include your policy as well as the keys that should be used to assign in toto layouts And the attestations themselves So basically you can use tough as your root of trust for the whole system distribute the both your artifacts your packages and all of your in toto metadata and then the user only has to um Get trust in this one tough root of trust and they can use that to establish trust in the rest of the system Including your policies, which will change over time And yeah, so all these layouts and the keys that are used to sign the layouts are distributed from more secure roles in tough These high assurance roles using these offline keys And you can do the attestations which change more frequently and have those be distributed using online keys more easily within tough Um, this has been used in practice. This is a diagram from a blog post Um in 2019 about a datadog integration of tough and in toto um It's a little bit in the details, but the the basic idea is there's four roles in the top left That's that's tough And then the the roles in the in the kind of bottom middle That's in toto and so you can see that the tough stuff points to the in toto stuff It's kind of the big picture how this all works and the stuff on the far side is the actual artifacts that are being um signed and distributed Now it's demo time So, um with the help of that's not working. There we are. Um, sorry, I didn't get that So with the help of some some students, I um Put together a quick demo of how these technologies work together And this is really where that tooling piece comes in. This is kind of a first step towards making this easier Because I think in practice right in theory, we know these two tools work together. It's been it's happened in practice with this um datadog Implementation, but what does this actually mean for um for folks today? And so the the idea of this effort is to make it so not only is this possible, but you can actually just do it automatically with open source tooling So we start here with um This project called repository service for tough. It is a new project for creating a um tough repository more easily and so We have you know, we stood up a um repository for that. Um And this is the you know, the the base level tough roles So if we look here at each of these roles, these just contain kind of the the tough metadata So the root role delegates to the other tough roles. So this is our root of trust This is the thing that's backed by more secure keys and revoke anything else in the system um And then we have targets, which is currently empty um, and we'll get to that in a minute we have um This is the time stamp. That's our um our timeliness role Make sure the heartbeat. Yeah, so no targets currently in that bin and um, these bin roles are just One level of delegation from targets. It's this efficiency mechanism for really big Repositories that's built into rs stuff. So we just Are those repository doesn't get that big but we're using it for um simplicity All right, so the first thing we're going to do is define a supply chain layout So Alice is our supply chain Root of trust like she's Alice is the person In charge of our supply chain. So she's going to define this supply chain. She's going to define some steps We're going to create um You know that the product we're going to build it and then we're going to have some rules about what should happen at each of those stages This is just the intodo metadata that describes that that pretty simple supply chain Okay, then we're going to so we're going to make that layer. We're going to generate it and we're going to upload it to our tough repository So if we refresh this we see that these bins instead of um Being empty now contain this root layout. So this one contains a root layout um Yeah, and then this one Contains Alice's public key. So now we have in the tough metadata, which again is back by list things we have The Alice's public key and the layouts things you need to verify the intodo metadata All right, next we're going to actually run run the workflow. We're going to create the project. Oops What's happening here What's going on? Sorry Okay, sorry. Um, we're we are yeah, so we created the project And then we are going to build it. They're following the same steps in that layout that we saw Um And upload it to the repository and you can see, you know, we're doing each of the steps in there Okay, so that is uploaded. Yeah, we're going to look at the new version of the metadata see that in there Um, so now let's see and the the the different versions for the So this is just again three is the version the current version of the metadata, right? And that that snapshot will indicate that so now in addition to This root layout we have additional information So we have the build that happened and we have the actual project itself As well as some of the links that happened during the creation like during the process of Doing this and now we're going to be a client. We're going to actually verify That the supply chain happened properly Yes, we're going to so we've verified each of the things and as you can see it, you know, it actually runs the verification It actually used the um in toto layout to make sure that all the steps happened And it says okay. Yeah, it's our past all verification And we're going to now download and install it and we run hello world Okay, now we're going to clear some stuff up and see if we can change Um, something in the file and make sure that this still Works I run through this one a little bit faster So yeah, we ran it again Kind of cleared stuff out. So now we don't have targets Empty Okay. Yeah, and so we have the same layout. This is the same people are to do the same things push that And then Bob's going to check it see that Yeah, so they have Alice's public key before And It's not the same file. I think I think there's a mistake in the demo But you it's the same as before you have the public key and you have the in total layout um now now stored there and then we actually do the um make the change to The project the person who's allowed to do it does does the change And then we can build it upload it and That will propagate in The direct in the repository because we made some changes now We have a version six and that version six includes the new build of the project okay Should have done this part a little bit faster just showing that you can make changes to it Um, and so then this one when it's downloaded Um, you see that it also also passed verification So what happens if it if something goes wrong or what happens if You know, not the correct person uploads it or someone someone, you know The wrong machine is doing the build or something doesn't match up between the different steps That's what we're going to look on look at next. So now we have an arbitrary adversary Who does not have Alice's private key who wants to tamper with the source code? so And this is an interesting especially interesting demo right because this Adversary tempers with the source code, but the build step actually still happens as expected. So um, this is kind of showing that even if You build system like remain secure even this this can detect a attack further back in the supply chain so anyway, as you can see here the Hash of this package Is actually different here than it was in the previous version right because that's because the Attacker has tampered with with this step in the supply chain. Yeah, it starts with BE instead of 09 So then I when the client tries to download and verify this they see that Um, you know raises an error. They say, oh, no, this didn't actually verify correctly. So we verified this stuff. This stuff looks good And then we find that um Yeah, if they found this disallow and it says, oh wait Um We didn't match any of the allow statements. We last to the disallow therefore this is going to fail That then we're going to do some some demo cleanup there, but that's that's that's basically it for that Okay, actually Great, um, okay, we're gonna go for like going to go like this um So yeah, so that's the demo And in summary basically what this shows is that you can use tuff to distribute the inota metadata to get end to end security um software supply chain integrity So if an attacker it manages to tamper with you know the source code like was shown in the demo or the build step Or anything else. Um, this can be caught because it doesn't match the rules in inota And the inota metadata is securely distributed to the user through tuff. So even if um So yes, you can't all you also can't tamper with the distribution of the inota metadata So you can't just change what that layout says and try and give that to a user to trick them That's what that's what tuff prevents And the layouts and the root level public keys are signed with offline tuff targets keys for additional compromise resilience so in the demo that we use online keys because um It was a demo but in in practice right these two are are the more um more important more um Like high assurance roles and those can be signed with offline keys change less frequently And we can get get assurance that they are are correct And the tuff snapshot and timestamp Prevent replay attacks of all of these different pieces of the system This is the beauty of putting it all into a tuff repository is that we get tuffs protection against replay attacks Not just on the package itself, but also on all the other pieces of supply chain metadata that we're distributing this way So I think we're ending a little bit early so but um we um Join us so basically the um the goal here is to show That these projects can work together and that we're working on improving the tooling for making them work together what I think my call to action really is that We want to know how this would work in your supply chain and how to make it work more easily in your supply chain Um, we have you know, we've been working with with various folks to make this more usable But um, we want to actually talk with the people who are going to use this open source tooling get some early feedback on how it's working um And make it work better. So um, there's some things here to the different piece different open source projects kind of related to this talk There's tuff. There's a specification itself implementations in go and python. You also have um a couple other language implementations um That people have written are as tough. This is that repository service for tuff that we used in the demo as a repository server Um, this is kind of this is a newer up and coming project that I think is going to be a really important piece of this usability idea And you can also contact us on the cncf slack to get pointers to these and other pieces of the tuff ecosystem and the intoto project Has a specification also open source fully available And the attestation framework and intoto is kind of where some of this work has been happening There's this um ight or intoto enhancement ight 2 that kind of defines this combination of tuff and intoto and um There's a lot of work happening to make this tooling more easily in that project And so that's kind of the place where this will eventually end up when it's like fully finished and kind of upstreamed Into that the the demo is also open source But I think we're trying to make this really like cohesive tooling not just a demo intoto is also on the cncf slack and um Yeah, please do chat to us. Please ask any questions now and also be chat to us Um on the internet and we'd love to hear more about if this is working for you and any other software supply chain things that we can work with So do they even have any questions? Yeah, so the question is about kind of the relationship between tuff and s bombs and kind of yeah How we get information about the supply chain and I kind of see them as complementary. I think that From my understanding s bombs are really about looking at what's in your software today And I think that this is more about Securing the steps of what's in your software. I think you can distribute s bombs with tuff I think that you could there's a very clear kind of linkage between intoto attestations of the supply chain and then salsa And then s bombs too many s is in the software supply chain. Yeah Yeah, but yeah, but yeah, then the s bottom actually says um in the end product what was in the end product So you can see a pipeline and there's this project called um S bomb it that's actually looking at this is how can you use this information from intoto that you have about What happened in your supply chain and use that to then like generate an s bomb say in summary This is the stuff that's in it and we know that because we have all these attestations about what happened Um, and I think they in some ways they serve a different purpose, right? The intoto can be can be verified to say What steps happened and then the s bomb can be verified to say but what went into those steps Definitely overlap. I think oh, but I don't see it as competition so much as opportunity for collaboration Um, I would say I think that we're definitely interested in in s bombs in at least I am in these communities and I think that They're nice like summary format to say, you know, all the stuff that's happening in total metadata tells you a lot about what's happening But sometimes that's too much to actually then um Be human readable and be actually looked at by people. So I think there's like this Cohesion where you can say, okay, this is everything that happened This is you know the the exact artifacts that went into it or something like that focus around doing Yeah, there's a lot of focus around like putting metadata about software and like The things around software into s bombs But in the end you still need infrastructure to do that and this is just a framework around securing that infrastructure So I don't think there's any competition I agree with both of you. Um, so for example, um, we distribute s bombs In what we do. Um, and internally we have we have this attestation framework. We're designing that that is based on toto Um, and you know for end users who don't necessarily want to verify, you know, these big complex pieces of information Yeah, um, we we have our own sort of we do the in total we the in total style We're getting there We do the in total style verification internally and then we that becomes a diet assigned digest That that that goes into the s bomb, right? So s bombs are kind of a place to me, right? They're a place in an organization mechanism for lots of for lots of key information um, and in total and tough together provide like Uh, you know a trust architecture that allows us to know, um, you know that that feeds that right It's more information for the s bomb to communicate Okay, explaining for for the one that is being in the beginning when no one was talking s bomb when we started Those things they're in the group in Germany People are looking at s bombs completely wrong most of the people that talk and even the components here talking completely wrong about s bomb s bomb is the end of the So the information after we have everything is just the declaration of data that you can Collect during the parts. This is one type of data that we can collect There's a lot of of data, but there's nothing to do about this bomb is providing information to s bomb S bombs is just a declaration of what is coming in the end There's nothing special about that what people talking about magic No, it's not the magic it comes about the information that comes before s bombs is just a result That's going to be bad could be good, but this depends on the information that collects That's the difference. So you are correct. You can actually give data to s bombs, but there's nothing to do about that All right, I think we have another question Oh Yes, uh fantastic presentation. Uh, I especially liked where you were detecting a malicious contribution without having to sign individual commits and store that in the commit metadata One problem with commit signing is that depending on people's workflows They might rebase there might be a word merge commit They might force push and then someone overwrite someone else's content I'm wondering if that's a sort of problem is addressed with in toto and tough Um, that's a good question Yes, and yes, and no, I guess is the answer right because I think we don't solve that problem specifically for get I think that there are efforts um working on specifically like the get signing Question and I think in toto can take in information for example Get signing and you can create a policy in toto that says, you know, you're committed to be signed by these people um But I think the other thing that you can do is just in it within toto is just kind of bypass that and look at The end piece of the um source code and get that to be looked at and signed in in toto, which is what happened in this demo Um, and I think that I I believe in the in toto community. There's work on in github actions to create To make this happen more automatically Um in toto, sorry, which my my co-presenter was here here I have a slightly better answer about the current work happening in in toto, but I do know that they're thinking about it Yeah, so Will this help against phishing attacks? So what happens if like an employees of an organization? What happens if their Keys are stolen. Yeah, so, um, I'd say yes. I think it doesn't solve phishing attacks Obviously, I think that you know the key will be lost or whatever if it if if it is phished or like you can create fordure signature For example, um, even with things like tfa right there've been attacks shown that you can forge signatures But things like thresholds and some of these kind of tough protections thresholds things like and revocation Can really help both reduce the impact and maybe even like get rid of the impact of a particular compromise So if you have just one key that's compromised, but you require a threshold of two It doesn't really matter, right? You want to revoke that key, but in the short term, you know, you don't have any any damage done Um, and that's that's exactly the benefit of these of these thresholds and this idea of built-in revocation Um, that I suppose you could also use tough to secure your messages Internally in the company, right? I don't know if that's practical, but hypothetically, yes, you could do that too Right, so but this is just this is a trust architecture that you can use the bgb keys you like right and The bgb keys for the emails with tough, so you know, which which email should be signed by which people? It'd be a fun project for someone to take on All right, I think I need a question. Oh, okay. I have skewed this before but I wanted to get it on tape So what if I'm too much of a dummy to understand tough, but I still want to get involved Contributing and still learning about the project Yeah, I think that there's definitely used for that. I think and again, I think one of our main Efforts in both tough and in total right now is improving the usability of tools So I think it's especially important for people who don't understand tough to come in and try to use our tools and tell us Where they got stuck So I think that's a great place to come in Similarly with docs. It's always nice to get outside perspective on docs things that that make sense or don't make sense And yeah, I think those are the main things and yeah For for folks who don't and then you can also once you do that if you start to get more involved You can start, you know, writing the code writing tests doing all of all the other classic contributions I'll be there Oh, yeah Maybe fun question Um, can you secure a google docs photo with tough Actually, it's a good question actually So, um, I don't know that anyone has secured a google drive photo with with tough There are actually surprisingly diverse uses of of Tough that go beyond even software. Um, there's this thing called the archive framework or taff that actually uses tough or a variation of tough to secure Legal documents and get like a historical record of those that can be securely given to people So yeah, yeah, I really I think that's the the power of it is you can securely distribute Any artifact, you know, Sigster uses tough to securely distribute keys and their root of trust So I think you could very easily just kind of, you know, and an image is just another type of data So you just put that data, you know into some kind of You know condensed format and then just put it into a tough into a tough repo As a target or whatever All right, and I'll take the last couple minutes to do a couple of thank yous. First of all to Adolfo here for being my ad hoc q&a Q&a guide. Um, also to Aditya who couldn't be here today But it was very helpful in putting this together and to um a couple of master students at both nyu and purdue Yonri and alan who helped make that demo that that you saw today So I want to shout out all of them and thank you all also for coming