 So Do you want to talk through these things? How do you want to go through this? Because we made some suggested edits that I don't know whether they're after it They were just intended to be clarifications or tell us that they're not like Or do you want to start with the questions I Was thinking of the questions maybe because this might also help a little bit If you go to the top so I don't think we Disagree that steps themselves can be compromised. I I don't think there's any You know like we make sure that the steps that you say you did you actually do but you know just like if you go to a hospital and a licensed doctor could go and I Don't know amputate the wrong foot or whatever. You know in all cases that People have sometimes seen that happens But we make sure it's not like a somebody pretending to be a doctor in there with a bone saw operating I think it's just It wasn't immediately clear with some you know, probably 15 20 minutes spent looking at the docs not a Like we didn't use it right that like Is it clear? What these steps actually are? right like Is the compiler? Executable and input right the version of the compiler is that is that how it's set up do people understand What is a step and what's in total? You know, of course like the compiler can be compromised But if it's installed on my machine, you know like you know that has physical security behind it And maybe I'm not worried But if it's a web service, you know, like there's if it's a reproducible build then What is my step and how am I specifying that? Like if everything is an exercise for a reader, then it's it's sort of hard right that this is doing anything Right, so I can answer this Santiago do you want to do it or do you want me to? No, go ahead. I just had a question Is this a like Boss preemptively trying to make clear for the reader or is this more like I don't think I follow what you guys mean Right. So basically, I think I know what you mean, right? So I think I could figure this out I'm trying to like so Justin and I got into a conversation with like is this a problem with the documentation not being clear enough and quickly successful for somebody who's doing the cursory Or is this that it is So generally specified that people are likely to misuse it to the point that they Implement something that they think is adding security, but it just adds complexity without actually have Right. I kind of think it's the former But like I couldn't defend it with things I looked at, you know, I was like, oh, well I think it's the former I poked around a little bit and they it couldn't point to something that says see no here This is exactly how it's, you know defensible that the docs could be improved, but You know, it's clear how to use it Right So does that I like I couldn't really defend which way it is on the Yeah So you can do things like say this exact version of this compiler with this hash and everything like that has to be used in the system You can make those kind of statements. You can also say things like a Compiler that you know, so let's say for a moment. Just just just say for a moment that you trust that This system can be used to produce a single piece of software that went through the right steps Just just suspend disbelief for just a moment if you believe that and then Since it can you can have things that point to other Layouts that went through that step. You can also say things like well. I don't actually know what the correct hash of GCC is supposed to be and maybe I'll be updating it more frequently that I want to update my own layout But I can say that as long as the GCC folks their in-toto supply chain says That this is a valid version of GCC then I trust them to know it So you can also do things like that where it's like an in-toto supply chain can refer to another in-toto supply chain in addition to doing literal hashes of things and so on Right, but I guess the question is how do I know like how do I exactly know that my so I'm doing builds, right? I'm doing like a you know, suppose we use the simple workflow where developer checks and code Release engineer decides to make a version change to kick off a release You know the we want to make sure the tests get run they pass and the artifacts gets inside, right? So in the the artifact gets built by some compiler How do every time I run this right? It's different code. So the artifacts going to be different How do I know whether my compiler has been compromised? Like how do I like how do I know that the steps? like I Couldn't articulate how in a specific instance. I I Actually know anything Other than yeah, I trust a total and it did some things. How do I know I set my lia like Yeah, how do you verify that I've set this up correctly and I'm using in total correctly Okay, so we validate what Effectively you tell us to validate So think of it this way like How do you know that the aspirin you get in a bottle is Actually aspirin and it's not going to hurt you Well, you know this because the FDA did some checking and you know that the the plant went through a series of steps And you know that those procedures were followed now Does that mean that somebody working at the plant couldn't have gone and substituted something in or that? You know the FDA couldn't have made a mistake when drug testing the drug and so on you don't know that You you have no idea what could happen Okay, but you get much better assurance that that the right things happen You know that these certain steps happen and that gives you trust and that's sort of what in total does so you can if you as a As like, you know a software vendor are really paranoid about things in your build process and you have some special hardware that is preloaded with You know it has a TPM and is preloaded with like an OS image and a specific version of a compiler And you want to make sure that all those things work exactly that way You can make that all part of your in total layout and have that all be verified Okay, if on the other hand You don't do anything like that and you just like compile with whatever compiler you have locally and it use the developers say Yeah, you know whatever compiler I have on my laptop, which I update all the time this compiles it You know, I'm I'm fine with anything You can specify that in in total it in total isn't meant to Force you to take a certain series of steps in building your software It's meant to let you capture and verify that whatever sort of steps and protections you have That you take that those were actually done correctly and no one circumvented them I think I understand that I think the question is if you provide zero guidance on What are appropriate inputs to the steps what are things that one typically does Then it risks that you are basically Closing the door, you know, like you're like, oh, well, what I do is I curl a random string That uses that's wide open to the internet and then I publish on github that I'm doing that and I but I've locked down all my steps, but what they're doing is fundamentally Blot, right? I'll be have to take responsibility for something and you have to define the edges It's just that without examples that are realistic and The answer being well, you can do anything it Risks that people will Feel like they're being really secure when actually they're just locking the door when you know locking the windows and the doors Okay I'm still having issues coping this is this a in total shoot Should from now on say like you can only have layouts that look like this Or is this like the document needs to mention this examples or is this like because in total was assigned So you can actually do the random string if you think that's a run thing We're not pushing you to think in any way. So yes, so this is so my guess and Before writing a detailed summary, I'd want to have like be able to point to things But like my guess is that it is a documentation weakness Not an in-toto flaw however There's a risk right that like Instead of saying well one can do anything you figure it out To say here is a common pattern Right. I have C++ code in Get That these are the steps that I want to do These are the risks if I do it this way or don't do you know like you have you've got like details about supply chain hats Right like you want to set it up. I think ideally I'd like to see like some documentation That is like this would be a supply chain that would be secure in these ways and not secure in those ways Where it's not just everything outside of in-toto is up for grabs because I think that people are having trouble Understand like the people I've talked to they're like I want to use in-toto, but I'm not quite sure Where would be suitable to insert it into what I'm doing? Total project or is this in the scope of the write-up this is basically In that the topic is What are the risks that a Average programmer like a better than average programmer somebody who deeply understands their release pipeline And their software right would pick up in total spend a lot of time integrating it and actually Not significantly change their security while making their release process more complex How big is that of risk? Right, so that's really the question, right? It's it's it's something that could be a oh well Docs need to be improved and like But I mean i'm not making that judgment i'm trying to like describe this concern In a way, uh, I agree. I I think it is uh, I think it's a valid concern The what i'm trying to figure out here is Uh, is this concern something that for example, we would like to have address from write-up Or is this something that uh, you say hey, you should improve the overall state of the project by having this Somewhere in the I don't know specification or in the website or in the x y and z The write-up having some, um Clarifications, right some of them we put suggested edit in some of them like it might be like like this, you know like obvious like this is this contradicts the The statement that there are things outside of the scope of intoto, right? that the Like the like to just scrub through the write-up to Scope things according to the assertions you're making about us as intoto and the ones that you're not And to clarify like if reproducibility is something that intoto offers Well, then what are the preconditions in how you do your layout that would? Allow you to do Reproducibility, which it doesn't have to all be addressed in the write-up, right some of it. It might just be like for example something right And To just try to tighten it up a little bit in terms of setting people's expectation when they're reading it about how much other work They might have to do in using intoto Yeah But the main thing is like it's In terms of next steps, this might be a recommended next steps to have a stronger use case spelled out Yeah, I think is my my feeling was it's it's really important to With security projects to let people know exactly what's in and out of scope in a really really clear way because people often assume too much Around, you know, once I've done this, I no longer have to do anything else kind of thing and so Spending some time clarifying this is is kind of important as users will misinterpret it because They absolutely, you know, they've they've done that with With notary I've seen a lot Okay, uh, yeah, I just wanted to Because I'm taking notes on this side and I wanted to like make a A list of things that I need to work on and uh To meet the viewers that the answer is both we should need to clarify this on the on the write-up which is like point Uh that point fixes and then probably uh, we should elaborate a little bit more on the metadata examples and content on the website and Documentation in general. Yeah, I think so. I mean, I think it um Yeah, I mean, I think even as I mean is I think it's important to have it the specification be as clear about what is in is in in scope as well so and I mean because then it gives people and Pointers to things that cover other relevant pieces are useful because they Help you understand what you do about the bits that are not in scope for this And I would be fine with having it addressed by Open issues that are like well we've gotten this feedback and here's our list of open issues for improvements the docs or the examples or You know What have you? Right. Okay. Uh, yeah, I'll uh, I mean, I'll have an issue either way. I think it's always good to keep this things track Uh, but it's something that we should be working on the near future. We we already have in the roadmap Documentation in terms of like having read the docs repository and having launched More grounded content that people can just like consume this year Um, well, we're on this topic about the documentation and use Something that came up to me was that um, the way that the project uses the update framework seems to be A little ambiguous from what I've read So I'm what the update framework top Yeah Oh So, yeah, so it seems like that that part of it is Um, actually important to the verification of the entire pipeline um, but it's not Um, clear how to integrate that with the existing system at least from the doc the whatever I saw from um The assessment and the the docs in the repo so, um So it is not mandatory to use top. We we had some integrators use tough as a metadata delivery mechanism and to Essentially ensure that a layout is fresh. Well, you can do it without without top, but uh Basically, they play well together in terms of metadata delivery now, uh Something that we were going to have soon and that was actually something we'll be working on with uh data doc people is Uh, we have this uh extensions repository the ids. Um, where we host the They're similar to Jenkins's chips Uh, the idea is that there we will have an actual recommendation document that says how are we going to How you should use tough and in total together? Okay Um All right. Yeah, that sounds good. It's I I've my my um, kind of field of it was that um The actual storage and the signing of of these things is um Kind of critical to to the entire pipeline Yeah, but the way that we designed in total is to make it transport agnostic, uh, that's why we have like The document there's like total five rafias and for example, that doesn't tell if it's using graph rafias as a as a metadata storage um We've had for a little while a control thing was storing it on redis Server that basically was expiring the metadata and memory Then it moved on to uh to like a to the To the mission controller and it was stored on disk and submitted on the fly But uh, all of these things can happen. Uh in my very personal opinion, I think tough is the way to go But uh, but for example, we kind of mandate, uh Debian for example to adopt tough just for verifying total metadata You you don't Have any's I mean, there's nothing in this about what the requirements are for metadata storage or What's better or worse? And why you might want to use that for why or what kind of metadata storage you need? Uh, well, that's a good point. That's the kind of with the we saw the the demo the demo example, but there wasn't much that was Concrete about you you need a metadata or this is the properties it's required to have This is the security properties that may be useful for it or necessary or essential or optional Okay, uh, I think we can elaborate on that. Uh, I would have to Reflect a little bit on it and think of what's the best way to put it. This is on the scope of the of the write up, right? Well Your doc should say You need a metadata storage period Here are your options Right Since it does seem to be a requirement that you store the metadata somewhere Well, the it you don't really need a metadata store. I'm sorry. Go ahead Santiago. You're about to say it Uh, I don't remember the docs were notary saying you need a metadata store Well, naturally provides a store tough Tough this back is more um I mean it does Talk about threat models to the store. It doesn't specifically say Much about the requirements for a store, but it does talk about Compromising things at the store. So it's kind of I mean naturally specifically provides A store because it's an operational product. So it's kind of It's kind of an implicit in that That's what I was getting at. Uh, I mean to me To me feels not much different really Uh, why why do we have a mysql store? What are the properties and the security property the security characteristics of mysql store compared to an in-memory store well, I guess that Not being deeply familiar with tough and notary Um, I it wasn't clear to me that I need to provide a metadata store Or whether it will just do it in memory unless I provide one and then it will be persistent or What Yeah, just what the threat model for the store is does it does it matter if the The store is compromised on what is more what I was thinking of It's it's not go ahead I can't review this but uh Uh, I'm I'm not entirely sure In how much value it would add I mean as long as the metadata is there and you can verify it Which is basically what tough also requires you to have You need to have the metadata get to the client at some point And that's what you need so the specific question that that I had about this was um With the x exploration check, I know that like this update framework does some of these with the time stamping uh Is it's a guarantees that you're prior providing like the the time check and exploration check. Is that reliant on That mechanism or is this in built within in total itself? It's uh, it's built in within total. Um, there's some question expiration. They can lay out Yeah, you get some consistency benefits with combining the two but it's not um Like but in total functions Just fine on its own Okay. Yeah, I mean, maybe you just need to say that you need to provide a metadata store The security of it is not as long as it's available is sufficient and um um You might want to integrate tough with your metadata store for the following reasons Uh, okay. Uh, yeah, I can review this I don't think there's a harm in trying to add more information. I'm just uh, I'm just trying to figure out exactly what I mean, it came up with with sarah asking Do you need grapheus because all the examples have grapheus? So it was a kind of mora Clarity and what what you need and what you don't need and what it has to do Yeah, so it was clear from the write-up of the example that grapheus was an example What wasn't clear to me was If I weren't going to use grapheus What would I do there? Or is it like like in it's just because like I know what jackal is I've used it I've never used grapheus. So it wasn't clear to me whether Oh grapheus is just another step. It's another thing. My pipeline is doing but I don't think that's true It's a it's actually something that is needed by in toto But I could do something else But I don't know what that other something else would be if I were to do it Sorry, I'm gonna run again. I'll be back Okay I'm saying is I am representative Of the next person who wants to use it in toto So take this information and yeah, the write-up should We should clarify what the requirement is in terms of you know, what's the threat to that? But I think the the other thing is just like it's not clear how to use in toto without actually using it And that's you know, just data for you as a project and Um You know, but I think the bigger question is well if data if the metadata store is a requirement right to have some words in the assessment of What um What what security considerations there are in choosing a metadata store? Uh, no, I agree. I think uh, all of this is valuable. I'm just trying to I'm just trying to follow and uh, like exactly know what the What is what I need to address? um So I'm like taking notes and I'm trying to like I don't want to run off and like do something that doesn't actually work or help So I think there's like documentation, right Improvements, right, which could be you know open issues and then there's um, there's also the security implications in choosing a metadata store Is the in the assessment this is more goes with this um You know finishing with this one So we can refer to it So yes, I want to Make sure that when we wrap this up We end up with In here Yeah in the roadmap So my guess is that some of the things that we've talked about in terms of grounding The introductory information about in TOTO in real use cases and having people understand Responsible for are probably more important than the CII silver badge items Although I haven't reviewed recently which ones are missing And so that's kind of the stuff that we're trying to get at in Writing up the summary is like, what are the most important next steps from our recommendation from security? Yes So I wanted to be here for the what happens when a layout changes or maybe you can just You want to just talk about what happens when a layout changes? Is that like is that something that you that was over here in Goals, I think this is a really important question. I'm not sure whether The this could be for we just want to clarify it and then understand whether it's future work or it exists now and Uh, yeah, let me let me read the so So, uh The layout does mean space for projects It it can expire And so it needs to be replaced and it can be replaced to for example revoke functionary keys and Or like add new steps or change Or yeah, or change thresholds and keys or whatever it Basically, it's map to one single application and it's the mechanism for revocation layout changes configuration changes Or enrollment of new functionaries. Does this answer the question? Oh, or Well, I guess the question is really like what are the risks when change happens to your process? Right like is it that well the project owner rate controls the layout itself, right? And so is it that you know, like Is can there be more than one project owner? Probably it's a roll, right? There can be more than one project owner, but I mean they the as long as they're allowed to sign that layout they can be The project owner Right. So like if you have new project owners, then they um the project Like one project owner can add another project owner and so forth presumably and then like When that like are there do we want to like To what extent have you considered risk the risk of change itself, right? and and Are there any additional considerations? in terms of When the layout changes, right? So, I mean my quick answer is the worst that can happen is You unintentionally Dost yourself because you updated the layout, but there's not enough metadata for window of time Because the functionaries haven't updated their operations but uh But that should be about it. I can I can probably work on this and try to phrase this in a clearer way and And add information about this on the write-up and probably also on specification. I think we do have an FAQ. So I I know I can add it there too because it's a I think it's a question that may come up often Yeah, so I guess there's um in the security analysis. I wonder Whether it would be clarifying whether it would help reader future readers, right to maybe there's some sort of like Different aspects of What users might do that could create problems, right? um where From my understanding of in total though, I feel like this There's a low risk of this being a big issue Because it's kind of just like creating a new policy And so if it's a misconfiguration the policy, you know, this This would be for example, if we've learned to Having a miss like a bad conflict file, which in most cases On the expectation is that it just wouldn't work I don't think I don't think there's any things particularly special besides The additional verification checks in place Uh, yeah, it's uh, I think that's that's about it. You update the layout and the worst that can happen is that verification fails because You haven't propagated all of the links or because uh Or because of your policy being wrong now Um, I think that's about it. I think you're right So presumably the layout is itself versioned or One can do that In the normal way that one versions things Yes, uh, that's for example, uh Again, we don't force stuff, but that's one of the properties that stuff does provide to you You can snapshot the layouts and you can Follow a verification path all the way to the most recent layout So I'm trying to think of how to capture this in a way that would adjust This address Justin's question when he's not here to speak for himself I don't know what happened. It's not Is he coming back? Uh, well, he he didn't say that he was leaving But he just took a step out I'm thinking I'm I'll just write some notes here, which is um So the um, are these build cycles ever, um Like are they all happening on like like how like How do I ask this question? Do you ever have to worry about like multiple versions of the layout being multiple places and being in sync? That that like are you assuming a single? um A single machine doing a single Um like executing this pipeline uh That goes back to the Right and so that if it changes there would be no Pipelines executing at the time of the change Uh, no, I mean it can happen and uh And I and that's that's okay The layout does need to Like functionaries may not even know what the layout is They just need to execute their steps and provide uh At the stations that they did what they did Right, but if if a functionary is say Preparing release 7.9 of a piece of software, right? I'm preparing with a 7.9.1 And um, I just that certainly could affect the integrity of 7.9.1 that could uh At worst make it so that the to essentially pay obligation given a service Because you didn't follow the layout that or the new layout um If the layout at the end is different from the layout at the beginning that verification will fail good it It's not a guarantee though fail. I can for example add a new layout Uh at the in the middle of now seven That that version and then uh in which I add like a new static analysis step And as long as that metadata makes it on time there Uh at the time of verification, uh, you should be fine It uh as that step there, uh, didn't need to change anything. I just added more steps to the layout Oh, you could do also kind of like a Red, blue, green kind of deployment as well, right? You could have Two current layouts, which are valid at the same time before you roll it over Uh, yes, uh for that you may need like a little bit more infrastructure outside of the open total, but you can totally do that Right like if I created a step at the beginning Your risk would be that You would get to the end of your pipeline and the thing wouldn't be present, right? But then It's possible. Yeah so, um I've added a note. We'll see Justin said that he would Try to come back So, uh, so yeah, I think that is kind of the the set of our questions Um, I have a few questions that I don't think I'll talk that I've put in a section or but I'll just say them out I think such a fun one. Thank you, Brandon. Sorry Um, so this is around the attacker capabilities in the write-up Um I'm not sure whether we should the scope of this includes the verification client as well And in that case, I think that um a potential risk that should be considered is that verification can fail Sorry, I was muted. Uh, so uh I may be missing something but if the verification client is compromised, I think that's game over Right. I'm not saying that it's it's um that we can't trust it. I I just saying that um, I think it should be written down as a risk so that If There are multiple other ways to control the risk, right? You can take the verification client and put it on a note that it's isolated and so on Um Actually, what do you mean? Yeah. Okay. Uh, we we can we can clarify that Is there any other questions? Uh, is it the one that you have put on the block in the oh, Brandon Yeah, could you put a question in the did you put a question in the main doc? No, I don't think I could It's just on that doc I might have not added you Is it the is that in total safe assessment notes? Um, I can just add you to the doc No, the the write up Um, because brandon joined after we started this Epic photo thing um Just made you an editor brandon. Sorry about that. All right and um We just made it so that um It's view only so that we know what you can't comment if you're not known So that people have to ask There's no way to say allow comments if you're logged in Okay Um, so So while you're putting those comments in I think that the next steps Um, are Santiago for you to go in and like resolve all the suggested things or correct or whatever, right? So Um, and then just put in links and whatever like there's some nits in here. Um, and then And then what I'd like you to do is adjust this roadmap based on our conversations, right? Of what you think and like we can go back and forth Um And then uh, so we can kind of align on what we actually think It's going to be the Important things and that anything that you think is important it has a issue in one of your different Okay, uh, I like the idea of tracking it tracking things as issues in the project, right? And then this becomes alive Um And see some issues will have been addressed or changed or responded to or whatever Um Does that make sense? Uh, yep, that makes perfect sense. Um There's going to ask something What's the what's the timeline that you like in terms of like the time window that you are thinking This should be addressed in what I'd like is for if I can corner just informer Have him and me try to write up this summary later this week um And that can be done in parallel with your I can just Make assumptions that you will have we're writing the summary as if you're The changes you're about to make have already been done, right? And we can Run that by you And I don't think it will say anything contentious But you know, we don't we want to make sure that you have the opportunity to correct anything or make, you know propose alternate ways right now if you think that You know, like I think we're we're kind of making up the first one of these like some of this So, um, so one and the the goal is that the process that we've described is that You would present it to the cncf Coc which could be an async or like because things are pretty busy before qcon They might just ask that the quote presentation be a note to the cougar um But in any case we want to make sure it's something that you would feel comfortable presenting um, or you know I think we're having a separate parallel conversation about What would it mean if the project completely disagreed with us, right? That I don't know how we would handle it, right? So that's a process point that We hope and expect is not going to happen within todo That we in fact are coming to the same agreement of how to articulate its security posture, right and where you are in You know, yeah some things we've planned to address in the future and that's fine um, and so we come up with some kind of description of that and then Which hopefully we'll have like a draft end of this week, you know Maybe we can go back and forth quickly. Maybe we have to meet again early next week And then um, and then we'll run it by joe and lia's to see if it matches what they want to send to the toc um My guess is like after we all get to the one place of like this is a good description Then they probably will be like, yeah, it looks great. But they might say, oh, but what about this thing? And then, you know, then we'll regroup but yeah, um, certainly I wouldn't want this done before um Keep on if not, yeah like As soon as reasonably possible without Craziness I am flying to England on tuesday so I'll just go visit Dustin in cambridge and not let him leave until it's done Um, but yeah, my hope is that that we have in principal addressed all of the issues so that we can be wrapping up documentation of those questions and concerns and You know, like just next steps And then um, and then the rest is kind of like formatting this summer Okay, sounds great. Uh, I'll try to address this as soon as possible. I have another thing That's also time-sensitive, but I think I can juggle both and I think By end of this week, you should be seeing At least All of it and tickets At the leg at the least Great, yeah Yeah, okay, and I'll give you I'll give you a shout out if it seems like Justin's schedule is super constrained But I'm gonna try to see if we can get together in this week and just draft something and run it by y'all uh, also apologies. I I tried to To like add or resolve most of the in-line edits that you did But I think I didn't do all of them And uh, like it just slipped through my hands. So I'll I'll get back to this and try to have this Uh, you can care of the since possible super And then I'll find make some suggested edits with our new sake brand She has to edit this sake security We have new name Yeah, it's uh looking at the date in there. It's also like It's been two months almost Yeah, and we're not gonna do this right now. I'm just gonna leave this for now. We'll come back to that Okay All right Super and I'm gonna try to I like the actually Justin's capos's Um point of like having issues for everything. So I've been trying to When I have questions that are clearly like the docs need to be improved like putting things as issues rather than in You know, they don't necessarily have to be addressed in the assessment. Just a bit of questions. So Right Yeah, and thanks so much for those. I I know that other people in the community have been saying like, oh, who's ultra sour. She's helping with the issues You know her Super all right any other last notes, Brendan? Oh, no, I'm good for now. All right. Thank you Justin Last comments Nope, uh, none from my side. All right. See y'all Later Yeah, see everybody and through most of you on three minutes on the safe call Next call