 So welcome to the afternoon session. I hope everybody had a good lunch. And thank you for coming back, I guess, to a session. There's about a half dozen really awesome other talks that are happening right now, so I'm gonna try to make this one worth your while. So my name's Erica Dretzka. I am with the Department of Defense. It's a very large employer and entity, so I'm within CDA, within DOD. I am at this Chief Digital and Artificial Intelligence Office, so that's the CDAO. I'm gonna try to spell out all the acronyms I use because there are, I mean, I don't think there's enough dictionaries in the world to spell out all the acronyms we use. So yeah, I have two co-authors on this. I have them enumerated at the end of the presentation, but one is Dr. Nat Fuller. He is contract support to CDAO in my shop, and he is also an adjunct professor at Purdue for data science. And Brent Smith, who is a senior executive technical advisor at CDA for the Advanced Distributed Learning Initiative, which is out of our undersecretariat for personnel and readiness. And you'll see as we go on why we included PNR here. Of course, we're tightly coupled to CIO, to our research and engineering, to our Department of Test and Evaluation, to acquisition and sustainment. We're working with all of them, but PNR really presented a unique application of X-bombs that we wanted to demonstrate to really leapfrog over the, I don't wanna say the banalities, but the things that we're all used to seeing when it comes to bills of materials. Okay. So, that was really loud. Okay, so first of all, we all know this bombs are not new. They've been around for a very long time. And even for some of our lag in timing, even DoD has been tracking these for a while. And just a quick going through a timeline. So up here on the far left, you see in 1955, we actually issued our first DODI on bombs. And a DODI is a Department of Defense Instruction. It is not enforceable. It is not policy. It's really just a statement of, well, and we have DODIs on like literally everything. So we have a DODI for instance on, how do you enumerate somebody's deployments? Which is a far more complex thing than just saying that person A was gone from their duty station for one day, right? There's a lot that goes into it. So anyway, you can imagine these are some pretty extensive documents. That didn't really take off. You can imagine this was probably a hardware bomb. I think it's still available if you guys wanted to actually read it. Software, honestly, I think we all know was not really a thing back in 1955. Also, we all recognize how manual those would have had to be to be produced. We reissued another one in 1972. So just a few years later, less than two decades, people realize these actually aren't, we do need them. Again, that failed, that didn't take off. So break, break, there was like some little semblance of something happening in the early 90s. Now we're starting to get to where software starting to like creep into the asset classes that DODI is procuring. I mean like obviously we had some before, but it starts to be a little bit more material. But then why do we think this actually matters now and why do we think that we can actually pull something off now? And it's because while we had this Executive Order 14.08, May 2021, it came out. White House said federal government is going to be requiring bombs from our suppliers. So this is no small feat. There's a lot that goes into this. There have been a couple of really awesome talks in the GovCon track. This is in the AI track for a reason and I can talk to that in a second. But we're trying to figure out what does that look like. CISA, the Cybersecurity Infrastructure Security Agency out of Department of Home and Security, they have Justin Murphy is here representing them. He did a great thing this morning at 10 o'clock. So they are working with NTIA, that's the National Telecommunications Infrastructure Agency to first of all define what are the minimum S-bomb elements? And the EO 14.08, just so everyone knows, is focused on software and software alone, right? You'll see why I, well, you can probably, you guys are all smart. You can probably figure out why we're saying X here. But this is really saying, I think they came up with like seven data elements. They're not, I mean, they are the most basic elements that you're going to require, right? They are not going to help us illuminate this supply chain if we only collect those, right? And they recognize that they're actually revisiting that, but they really were going for the lowest common denominator on that. Again, CISA, they hosted an S-bomberama amazing name in December, 2021. And that's really where I started to get plugged into this. So they have I think like five or 12 different ongoing working groups, if you guys are not involved. You can shoot me an email, I can put you in touch. Justin Murphy and Dr. Alan Friedman, who's the director of CISA, they are extremely hands-on and they want all the industry incorporation that they can possibly get. So they are like so like, you know, man-on-man when it comes to trying to work with industry to figure out what this looks like. And you know, they're looking at the federal government wide, DOD of course has its own uniqueness to it. So anyway, I got plugged into this in December, 2021. Again, I'm from the digital and artificial intelligence office. And so I'm gonna give you a brief background to that because I think that is critical to why we're in this track in what we're actually hoping to accomplish here. My background is actually data science. It's not software development. So when I first heard about S-bombs and I attended some, it was like a vendor showcase or something, it was from Cyclone DX. And our chief software officer said, hey, Erica, why didn't you attend this? And as I was going through it, I was like, yeah, okay, I don't do software. But then like the more I learned about it, I was like, hold on a second, like within the half hour I was like, hey, Jason, this is data. Like this is literally just data. It's, and it's data that is structured in a way that we can perform advanced analytics, remedial analytics, just pure insight. And so I was like, can we do this for hardware too? And he goes, well, yes, we can. I mean, obviously that we've been doing that. It wasn't obvious to me at the time. So I spent the last year actually coordinating with the five principal staff agencies that I mentioned before, CIO, R and E, ANS, DOT and E and CDIO to issue a memo. Memo is not enforceable. It's really just a formality, but it's like a bat signal to like DOD and people that we actually care about this and we're gonna try to make this happen. And that's really where you guys come in. So let's first understand X and this is one of the major misconceptions as we were putting this out there. X really is, as we would do in programming, it is a variable. It's not the extensibility of the content of a bomb. So we're not trying to say that we're gonna try to shoehorn hardware into the software bomb. That is, that would be absolute naivete to do. Rather, we're saying that we're going to maintain the fidelity of each bomb as it is. In fact, we're not even going to try to step in and say that we're going to use SPDX and everyone else follows suit. We recognize that we work with vendors from all over the place and we need to have these things be able to talk to each other. And with minimal disruption to the people who are producing the things. Because at the end of the day, we want the thing they're producing and this bomb should really just be a derivative of that. So anyway, we do recognize that all of our assets, however, are very interdependent. So I'm just gonna do a nominal walkthrough here. You can kind of see it on the right over there. And let's pretend that we're running an event and this event is going to be an exercise. And we do military exercises all of the time. We probably have a thousand a year that we do. And we do them so we can test out our vulnerabilities with our allies, where they're active, they're using real ammo in many cases. Lots of modeling and simulation because you have to simulate what the red forces would be doing. But anyway, we have this event, right? We have people who are running it, planning it. That event is using hardware. So number two is hardware. And we could say that that hardware could be anything from an F-35, which is arguably flying software to a server that's sitting back at headquarters. It could be the boots that people, well, not the boots, maybe that's not hardware. But you see, so we have these physical assets and that's a bomb of its own, right? Those assets, especially nowadays, you think about drones and such, they are, there are a lot of software inside of that. So you're already seeing how we can't divorce these two if we're actually going to be putting ourselves on the stage with some very advanced adversaries. So we need to be able to tie hardware to software. So that software is pulling from data. And data would be up on the upper right there. So databases. So it's not only pulling from data, it's also populating data in real time, hopefully if we're doing things on the edge and stuff. So now you see we need data bombs. So there's data cards. We know Google has data cards, they're pretty effective. Are they 100% effective? And so these are all just the questions that we're posing to the community with no judgment passed on what's out there because everything is being done as everybody thinks is most responsible. So that data is being fed into, I was saying AI and ML here, it could be just a dashboard, it could be, I don't know, it could be a lot of different things. But you know, let's say that that's like a GPS router that's advising the plane, it's an F-35, a pilot inside, you know, there's a gray cloud ahead, there's a stuff happening down, I recommend that you circumvent the cloud. And then the pilot can override that. Say I disagree based on my 30 years of experience, I'm going through the cloud. So you can see on the bottom left, we have a talent bomb or a competency bomb. So you have people who are also assets. And while that may sound callous, we've often referred to people as our greatest asset in when we talk about companies. So there is a way to do that. So anyway, if we can actually pull this off, what we're looking to do is create these, you know, these statistical patterns to make these, they could be symmetric, but most of these are gonna be asymmetric asset classes and how can we actually make them work together without disrupting them in their own. Okay, we frequently and DOD will talk about things in terms of operational, sorry, strategic operational and tactical. So at the strategic level, it's not like just a bunch of people with like, you know, cigars and cognac and their snifters swirling it around having some big thoughts. They're actually integrating a lot of our budget constraints, our policy guardrails and a bunch of other things that, you know, can't always be thought of at the tactical level. So I'm gonna speak to the strategic side of this and then I'm also gonna talk to the operational side of putting this in place. The tactical side I think is why we're here and we're here to really implore the community to help us make this something that makes sense for everybody from industry to academia to government to everybody else. Okay, so let's really break down what this looks like. So we have these dispersed supply chains. We encourage them to be dispersed and divergent because that injects constant, let's say, challenge to resilience, support for resilience. It keeps us as fresh as possible, you know, so we can keep incorporating best practices and not get stale. But that does, you know, that distributed supply chain does present a huge challenge as all of us know here. So we are kind of boiling it down to metadata. So if we can capture the metadata and you could say the bomb is metadata, right? So if we can boil all of those assets down to their metadata, then we can actually achieve what in the DoD data strategy we call Valtis. And this is a set of, I think it's six goals. It's acronym meaning that all data should be valuable, accessible, understandable, linked, transparent, interoperable and secure, that's seven. So seven goals. So if we can actually break everything down into its metadata, then we can actually accomplish that. But in order to do that, we have to encode it into structures, which are, you know, we veer away from saying standards. We like to say patterns or structures. It's a little bit more fluid and allows us to not be, you know, so didactic about the way things should be done. So we need to encode all those metadata into these structures. And if we do that, then we're going to be able to actually like, cluge all of those together when needed, as needed, you know, per use. So they're not always going to have to be clugged together. We can tie them together for whatever event or reason is needed. So we imagine this is going to result in a weighted directed tree. And we can, you know, have all the weights according to whatever is the type of, the type of weight that we're looking for. Okay. So another way to look at this is in terms of the actual like physical graph build out. And I think people in this audience are going to be like, yeah, why are you boring me with this? But, you know, just for sake of conversation, we do frequently look at things hierarchically. And DOD is probably one of the more rigid in that respect. I mean, if we look at our organizational structure alone, we tend to say, you know, this battalion belongs to this command and that's it. But in reality, that battalion could be parsed in two for two different events or as needed. Or it could for a specific event be assigned here and to a different, you know, command. And they have dual assignments so that every command is like based geographically, but where are they actually? And for whom are they working actually? So there's a lot of these like multi-relationships that we're having. Just the organizational structure. So in terms of where bombs come in, we think, okay, so I'm gonna point out two here because in the next slide you're gonna see things kind of shift around. So here we have product one, it's called a plane. It has four components, one, two, three and four. Component four has two subcomponents, five and six. Two has, you know, two subcomponents, you see it. Realistically though, in that C6 is actually, let's say it's a panel, and that panel is not only on C4, it's also on C2. And so you start to have these multi-relationships like we just said. Okay, so now let's re-render this out into this graph structure. And now you see that C6 goes to C2, C4 goes to C2. So you see how there's, now we're starting to render out how this, you know, this is a little bit more complex than that hierarchical thing that we've been looking at. Okay, now let's add in weights. So I think we all know how this works, but of course, you know, in a DAG, you would have a direct-to-day cyclic graph, you would have weights on your nodes and on your edges. And I think, like, we could talk about the weights for a second to say what would these weights be? So the weight could be the quantity of the component, we have 15 screws and we have one panel. The weights could also be the cost. So for a budget constraint question, I remember going back to the strategic questions that people ask at the strategy level, budget constraints are always big. And I mean, especially looming as we are right now with what's happening on Capitol Hill, we have to make those trade-offs. And then, yeah, so feasibly, your weight could be your cost. It could also be a risk weight. And so I've done this in, when I was in the data science side of the house, did this in like looking at, you quantify the risk for a geographic area. And you can do that, it's based, I mean, we just use like stacked random forests for that, but I mean, you could do, that was what made sense for that problem set, but you could feasibly create a risk metric for what each one of these would look like. But really at the end, we now have a better picture of that plane and its components, where the risk is in those components. And we can all build this off of that simple JSON file that is a set of bombs, or those because we're gonna have to be able to colluge all those together. So we envision that each bomb is a graph with weighted edges. We would like to insert a semantic layer, and so you'll see that in the next slide, but this is where we do have a couple of efforts going on that would be awesome to hear what other people are doing. For instance, there's ontologies like NIEM, and I don't actually know what that stands for, N-I-E-M. I should know it, but it's a, I think it's an IEEE or something like that, but it's widely adopted by DOD. But it still has ontologies captured inside of silos. So you could have the word red in one silo mean red light, the word red in another silo mean I'm flying red forces, which means in an exercise you have blue forces, red forces. And I mean, those are two different things. They have similar connotations to them, but how can we make it so that they can still have their red forces, they can still have their red light, but we can make these things talk from one ontology to the next. And something we've been ideating on that is, and we're actually building on a microservice for it too, is we're calling it canonical controlled vocabularies, CCV. There's, anyway, if anybody has questions on that, we can always talk about that afterwards, but it is basically like your meta ontology, so to speak. Another thing we're looking at is universal identifiers. And so of course, when you have bonds from all over the place, you need to be able to understand who they are wherever you place them. So what can we do for a universal identifier that, so for instance, like I'm an American citizen, and as an American citizen, I have a driver's license number, I have a social security number, I have my EDIPI, which is my DOD number, and the list goes on, right? And so if I lost my driver's license, would you be able to find me if you were DMV? That sort of question. Okay, so yeah, basically the semantic layer is gonna allow those node linkages across all those layers. Let's see, manages all nodes context of data, yeah. So it also allows, so the context, like the red forces and the red light that we talked about, it's gonna make that possible. Yeah, industry 4.0, we think, of course that's like a manufacturing thing, but duty does pretty much everything, manufacturing is part of it, like we are like our entire economy inside this little capsule, so you wouldn't believe the, I mean, even like the commissary, where we have food and we're serving soldiers, you know? It's pretty massive and you wouldn't even, you probably couldn't, like we have actuaries. Didn't you think we have insurance actuaries because of the GI bill and all those expenses that are long term? So we really are like this massive, we have everything going on, yeah. Ontology management, I think we talked about that. Okay, so now we have this weighted bag and we now see that this enables us to, let's say that plane is product one, we also have a tank, we also have, you know, the DISA, that's the Defense Information Service Agents Agency, they have, you know, server farms all over the, not all over the place, but in specific places. So these things all have to kind of work together and this way we can actually get that understanding at, let's say, the exercise level of how these things pull together at the end of the day. So yeah, with the, I think we already talked about everything on the right. Okay, oh, that semantic layer, we talked a little bit about that. If you guys have other ideas and things that would be useful inside of that semantic layer, you know, we've only been tackling the ones that are at the lowest hanging fruit right now, but there's gonna be, I'm sure, so many handshake things that we have to think about in there. So I know other people have thought through this very deeply and probably have solved the problem, but you know, there's, we can't just always adopt a solution as it's stated. We have to, you know, figure out how that works inside of our messy, messy infrastructure. Please. Can you talk more about how you came to that conclusion of what that actually is, the structure and layout of the data? Yeah, so what I could do, I think it's publicly releaseable. I could probably share the paper. I'd have to check to make sure. It's okay, I just wanted to, in industry we have, I think of some more concept where we talked about, where we say I have two different things, they were developed by two different groups of people, therefore they have two different views of the world. So when I transition between one product and the other, I have to have that as a dictionary to say, as you said, red, what does red actually mean? Because it meant something different to give it to different groups of people. So I'm trying to understand similarities, just similarities between CCB as data advancement and our concept of ubiquitous language for product. Does that make sense? Yeah, yeah, sure it does. I'm not sure I could speak entirely to it right now, so my contractor is the brains behind this operation. But no, it basically, we boil it down to every, let's say that you have OSD, Office of Secretary of Defense, and you can imagine the hierarchical work charts. So you have Army, Navy, Space Force, Air Force, Marine Corps. And each one of them, I think like the last digit is indicative of which one. And then there's a whole lot behind it. So that wasn't very much of a clarification, I'm sorry. But I promise we'll make that work. Yeah, no, no, it's cool. Yeah, if you have any questions, I got a mic here. No, that's fine. So it sounds like you're looking to create a universal management framework for S-bombs instead of directing how bombs should be created within each of the defense organizations. But with that universal framework, what do you envision the storage capacity, assuming that you would want to run different analyses or other machine learning aspects on it? What are you envisioning it as like a standardized, like a database, a relational database, or is it more than ever? I'm curious, mostly I'm a program manager at the Linux Foundation. I have two networking foundations and I have two AI and ML foundations. And when it comes to S-bombs or creating bombs to be in compliance with industry standards today, I've noticed particularly when it comes to networking, we kind of group open source software and software bills of material as one, but network open source software is vastly different than machine learning open source software. So our considerations are security controls, parameters, secure coding really needs to be different based on our software type. So I'm wondering what we can do as a project and as a foundation to assist in the collection of a bill of material, particularly for capturing regression and other data science aspects. Oh yeah, and maybe this is a conversation that we would have to further it because this, so like I said, we got a memo out and it basically itemizes and it's intentionally vague, but it does itemize, because that's the way you have to do things in DOD. You can't just like come up, people like here's the answer, right, they'll be like, they'll tear it apart, it'll be dead before it even walks out the door. But that, so we itemize that as one of the considerations that needs to be thought through. So that would be amazing if we could actually like- Partner? Yeah, partner. I think it would be too. Yeah. Yes. So I'm wondering if you've had a chance to look at what we're doing with the next graph of SPDX, SPDX three, because we are effectively setting it up so we can set these semantic structures up as well as capture the AI and the data set information and weave that in. And so I'd really like to have further discussion to see how we, if there's an idea. We've also put extension points for the things that can't be public. Okay. And so we've been working with MITRE and some others, but I really would like to say, I was very curious about what you're doing here. And we'd love to have further discussions and see if we can get things to coordinate and build on as opposed to reinvent. Oh yeah, for sure. It would be great. You know, we have a regular call with Cisco. And when I say we, I mean, there's representatives from CIO, from R&E, ANS, DOT and E and myself. And so if we could always like set up a regular touch base, we are tracking a lot of what's happening in SPDX. Not myself personally, but a gentleman in CIO through aerospace is, he's been putting together a whole like, hey, these are like basically a side by side for all of these, because SPDX is certainly one of the, it's one of the, you know, big gorillas in the room, right? Other people are gonna love Cyclone. So we got it, but I did hear that you guys have an AI bomb that you guys are working on, which is like very exciting. And we separated AI from the data, whereas sort of the machine learning ones are trying to put them together. But we separated because there's cases where you just want to catch the data and you want to catch the provenance of the data as it emerges. Oh, yes. And so we were sort of seeing that as a use case. And so I would like to say, if you all put me in touch with whoever is working on this comparison, I'll bring them up to speed because we're doing things all really fast right now. Okay. And I don't think a lot of this stuff is terribly visible unless you're in the room, if that sort of deal. Oh, yeah. It's going out to GitHub Repos right now, but I don't know if people are waiting through GitHub Repos or just reading documents. That's true, yeah. And I don't think we have the time to do all of it. So yeah, let's definitely make that a touch point. So that, by the way, is what we're all hoping to get, what my team is hoping to get out of from today is like that sort of thing. So please, you know, keep the questions coming and the connections. Can I ask one more question? So when it comes to specifications such as Cyclone DX, obviously at DODA have gone very far when it comes to managing the data or categorizing it. So we in the industry in insurance, banking, et cetera, is it too early to align with a spec? If we align now, obviously this sort of analysis is going to follow, right? So if we think of Cyclone DX as this big JSON with everything dumped into it, do you see us being able to transition to this sort of analysis down the line? I mean, there should be some sort of a tool that's going to look at the spec and transform it to something like this. Do you see that evolving or are we on our own to do that? You mean with an industry? Yeah, with an industry. Well, I mean, this is where I think like talking to Linux Foundation and you know, Cyclone DX too, like we have to, and Dr. Friedman out of CISA is he has his active conversations. So I would imagine that there's going to be, well, I'll say this, we are never going to say it's going to be this standard. That will never happen for us. We will say that when you give it to us, we need it to be translatable into whatever pattern, right? So we have the same question right now. We have SWID, SBDX, Cyclone DX, all buying to be the next spec. I think the industry doesn't really care about the spec. What they care about is, can you get value out of the metadata like you said? So I think everybody's shy to align with the spec at this time, but the more important thing is, can you translate it to this weighted graph that you're talking about? Yeah, and this is where that semantic layer right there. That thing is, oh, you guys can't see my little cursor, can you? The semantic layer is that, I think that's what you're asking about. And you know, honestly, this is in CDAO, like everybody's talking about the mesh, right? We've been talking about it for a while. Nobody really knows what the mesh is. I personally, if you ask me, I would say this is how we get to the mesh, right? It's all those handshakes, right? It's not something that you can just buy because it has to be like for an insurance use case. And I actually, I spent 10 years doing data science inside of an insurance company. So I get it, right? But it's a different area, right? But can we build those modularly enough so that we don't have to have, retool this every single time for every industry, right? So it's an excellent question. And maybe this is something like, I don't know, maybe we could team up and start to talk about this cross industry because there's lessons learned on all sides. Absolutely. Yeah, and this semantic layer, obviously it's, if it's coming from DoD, there will be rules around it. But do you see the industry evolving to build this? This could be more important than the spec itself. Oh yeah. Yeah, I think so. Yeah, I think this is something that the industry is going towards. And everything that this lady was saying is exactly right, right? And I think this is kind of what we're getting at with it. But can we build it so it's modular enough, right? And therefore we don't have to retool it. I mean, wouldn't that be beautiful? We're constantly retooling things, by the way, that's... So one of the other hats I wear is I'm one of the co-leads on the tooling working group under CISA. And so I'm basically seeing both sides very clearly and what we're going after. And certainly being able to have interoperability, at least the minimum elements is something that I know that the SPDX community has focused on. And we actually put a release out last year for 2.3 so we could do that. 2.2 is the ISO standard, 2.3 is a few more fields on top of the ISO standard, okay? But 3.0 is where we're really taking this up to another level. But modularity and the types of S-bombs that have to come in here, the information you'd store when you deploy something is quite different than if you're scanning a source file. And being able to have one referred to another, referred to another, is something we're very focused on so we can work towards these measures, which is why when I saw this talk, I went, okay, I'll back them up here and figure out what you're going after. And the question becomes, what semantics do we wanna keep on the relationships? And that's what I'm interested in understanding. It would be great. I also came from ZODO as a data scientist at JSOC. It would be great if we could use the S-bomb and metadata from the S-bomb to help change the ATO process, particularly when it comes to assisting small business and getting their products based on open source up, you know, up the chains and then to air-gapped environments. Like that was such a struggle. It was just a straight struggle. If there was a way that we could use S-bomb metadata to assist in a pre-appearable process or some sort of anything within the change management boards so that it would lessen the burden on the J6s all around. And I'm very sorry, I have one more question. No, go for it. My second is how you anticipate S-bombs affecting multi-domain operability between organizations. So I actually, this is how I presented it to our director is that S-bombs actually start to break down silos. And so I actually think if we make this the information or the data information worthy and allow it to be transparent. And this goes back to those Valtis goals, the visible, accessible, understandable. If we do that, we actually break down those silos and not everybody wants to do that, right? That's part of the challenge, right? So that's why it's so critical in this memo being aligned with CIO, RNE, ANS and DOT and E because those four entities, in addition to CDAO, especially acquisition and sustainment, you mentioned the ATO process. And just so everybody knows that's like the nightmare part of acquisitions is, well, one of the nightmare parts of acquisitions in the federal government. But do you have an authority to operate is what ATO stands for? And yeah, if we, so not if, I think this will break down that visibility and it will allow us to see. And so we're actually working with our acquisition of sustainment right now to demo out can we acquire some of this stuff or at least the bare minimum upfront so that we can do exactly what you said. And we will start breaking up silos across the place and it has to happen. Nothing happens in a silo, so. Okay, so I think we have a few minutes left. Okay, so ma'am, I'll point this to you. I really like that you guys are splitting apart data from AI. We do, I coming from a data science background, I'm very much keen on the fact that these are two very, very different things. So really assets on the left and the green, those are just a small enumeration. Certainly it's not exhaustive and it might not even be representative but assets could feasibly be anything that DOD has value in or an insurance company to your point, sir. We have a few of these listed here. We've talked about some of them. I'm gonna show you a demo of something we have going with the people side of this in a little bit. But then people always ask, well, why do we need these? What's the usefulness of this? And so on the right we start to say, what are those analytic questions that we could ask when we do render this out in a graph? So scrim, supply chain risk management, cybersecurity, naturally those are low hanging fruit here. AI assurance, getting that clarity into what's happening. Responsible AI is it is the Achilles heel of making AI deployable in an operational environment. People don't trust it. And you have people who have been trained for 20 years to do this thing. First of all, I'm insulted that you think a bunch of code can do this. Second of all, but then you have to help that human machine teaming element which kind of brings me to the mod sim side of this. I work really closely with the modeling simulation community as well. And they're always using, you know, in order to, you have to make digital twins. You have to have your metaverse and all the big buzzwords that we have right now. But, you know, they've been doing this for a couple of decades, some of them. In fact, one of the co-authors, Brent Smith, he is, he was pretty seminal to the mod sim world down in Orlando which is kind of like a hotspot for, it's like the gravitational pull of mod sim happens down there from my understanding. Command and control platform. We have the joint all domain command and control, JADC2. It's been vast, has vast literature written on it. I'm not sure. Honestly, I think they need X-bombs to make it possible. It's not gonna happen without it. I'm trying not to be as like, you know, in their face about that, but they are starting to see that that's the case. I was talking to our director of T and E at test evaluation the other day and once he heard that, he was like, oh, now I get why we need these. So that's good. And then training. And so I'm gonna show you a quick example of training. You know, the personnel life cycle that is a supply chain in and of itself. And so let's show you what the, this is what ADL, the Advanced Distributed Learning Group has been doing. They have a couple of IEEE standards out there but they basically are building out this Enterprise Digital Learning Modernization, EDLM. They've been doing it for multiple years. Everything they do is open source and they're just a killer team down there. They're out of Orlando and this is where Brent Smith is out of what the co-author. So I know you can't say this. It's not that you're not able to, it's not that we're not able to show it to you. It's just that it doesn't fit on the slide. But yeah, this is just basically there, what they've built out is their infrastructure. Lots of, you know, microservice sort of stuff. You know, everything from event driven to asynchronous and all the stuff that you would expect in a pretty advanced space. They're not, they're not operating in air gap environments like so many people. So they have a little bit more latitude but they've developed this architecture. It's just called total learning architecture that is actually adopted by most caveat, many of our NATO partners. So this is something that's been pretty widely adopted. So definitely they have a lot of literature out there. I think if you just go to like ADL, I don't want to say ADL.gov, but if you look up advanced distributed learning, you'll have enough to keep you busy for a while. So how do they actually do that? Cause we have data on every thing. I mean, once you put on a uniform, once you enlist, we, it's amazing the amount of data that is behind the PII and PHI walls there. They are appropriately protected where we do our absolute best to make them appropriately protected. And they're capturing everything. We try to bucket them here into learning experiences. Like you have formal and informal learning. Like right now, this is arguably an informal learning environment, but I think every single one of us has learned a few things today at this conference already. But yeah, so there's, you know, what are your credentials? Do you have a certification? Yada, yada, your individual experiences. So when you, let's say you are a military, you get deployed, that's an experience. And how can we quantify that and make that digital? Performance assessments, mentorships, all those things they're starting to capture because we have that data available. And the talent management, you know, how are you progressing across your workload? So it's, they're actually doing this. And, you know, frankly, these things all tie into all the different types of Xs. So if you're a software engineer, so we have people who, this is gonna kill some of you guys, they'll be like a gate guard on base, right? And they're like talented software designers. Like they actually know how to code and stuff. And that's like how they're spending their time. So that's a large part of the success of the software factories, if you guys attended that earlier, they're actually starting to find those talent and bring them in and train them up even more to say, hey, you can do more than an import statement. You know how to do, and let's show you how to do that in real, and then pair them up with actual exercises. So like if we can start to do that, you can see the traversability from a person to like hardware to software to all these different things. So yeah, I think there's one more slide. This is more just like a call for thoughts of what people think we should do with this. If you disagree with it, that's a perfectly reasonable response as well. The right is just more of a visual, but we are very interested in the transferability of these analytic questions. So VEX, yes, VEX is a software thing, but hardware has vulnerabilities. Humans have vulnerabilities. If you broke your knee, that makes it, even if it's healed, like that might be a vulnerability should we deploy you to Siberia where it's super cold and that gets to your knee joint. Supporting infrastructure, we are talking. I think a couple of you guys asked about CCV. So what are all those like other things that we need to be thinking of? Of course, privacy is massive here. We have a lot of our vendor partners who their entire profit revenue base is based on this and we wanna make sure that everybody maintains their competitiveness and such. Revision or recall, somebody asked about this earlier. It must have been in a different session, but how do you retain like currency, sorry, current state with your materials and then should you need to go back to a prior version to understand what the delta or such, like you guys all get this. And then just analytic techniques. So there you go, that's Xbom's. You guys have all of our information here. I highly recommend Mr. Smith is awesome. So is Dr. Fuller of course, but yep, hope you guys enjoyed it.