 Hello and welcome to the aerospace village panel discussion on building communications across the aviation ecosystem. Hi, let me introduce myself. My name is Katie Trimble Noble, and I'm the director of product security incident response and bug bounty at Intel. So I run the bug bounty program as well as researcher outreach and engagement. And I've been with really in the ecosystem for many, many years. Prior to coming over to Intel, I worked for the department of Homeland Security, where I was the section chief for vulnerability management coordination. So I've been doing this for several years throughout my career I've coordinated and disclosed over 20,000 cybersecurity vulnerabilities. So, I really want to jump in right now we don't have a lot of time today and I want to go ahead and introduce our panel. So we're going to ask everybody to go through and introduce themselves. So Randy, can we start with you. Yeah, hi. I'm Randy tally. I'm a senior advisor with the DHS and cybersecurity and infrastructure security agency been working in aviation for quite a while. As a, as a pilot doing aviation security for DHS, and have been on the, excuse me, the tri chair, the DHS tri chair for the aviation for the aviation cyber initiative for the last two and a half years. Awesome. Hi everyone. My name is Sid Gadji I'm, I work at FAA. I'm a manager within the Office of Information Security and Privacy. I've been with the agency about 14 years and in a variety of different role. And I serve as the tri chair for the FAA on the aviation cyber initiative for the past two and a half years as well. I am glad to be here. It's my first year at DEF CON, lots of people in the room, you know, virtually, and I look forward to learning from you all. So glad to be here. Thanks. Welcome, Sid. Welcome to DEF CON. I hope this experience is awesome for you. You get a little bit of a different flavor this year. So, John. Hi, my name is John Craig. I'm the chief engineer of cabin network and security systems at the Boeing company. I've been there for around 34 years. I've worked in all sorts of systems groups on commercial airplanes and my current role has me working all the cabin systems, the networks on the airplane. I was the development of connectivity links and I'm responsible for products security for commercial airplanes. I'm the chairman of the aviation ISAC and really like to encourage people to look up that organization. It provides a great sharing of opportunity and aviation. Thanks. Yeah, welcome. Jen. Hi, I'm Jen Ellis. I'm the VP of community and public affairs at rapid seven. And I am probably the least of the aviation experts on this panel when I say probably I mean definitely. But I represent the voice of I think the security research community might my job is to think about how do you leverage security research and insight and expertise to create social change that makes a more secure safer world. Awesome. Awesome. Jeff. Hi, I'm Jeff Troy. I'm the president of the aviation ISAC. I've been in that role for about three and a half years. I also work for General Electric. I'm on the staff of the CISO General Electric Aviation. And I'm on the board of directors of the National Defense ISAC so very engaged in the information sharing world. And after that I was with the FBI for 25 years and left there in the cyber division, you know, work in the cyber criminal and national security cases. Glad to be here. Thanks. Yeah, happy to have you, Alan. I'm Hal Burke and I'm an associate deputy director in the Air Force is cyberspace operations and warfighter communications. And I'm also the Department of Defense chair for the aviation cyber initiative. I come out of the airspace and missile defense operations community. And my primary focus in the ACI is that I'm proving cybersecurity. So I'm going to talk about the objectives and resilience of derivative aviation capabilities, where the DOD interagency and industry objectives intersect. Awesome. Awesome. I'm excited to hear more about that later. So now that we have all kind of introduced ourselves and we have good idea who we are and where we come from you can see we have a just jam packed panel full of awesome aviation cyber and researcher professionals here so I'm really excited about it. So today in this panel we're going to talk a little bit about some current activities between government industry security, the security community the responsible disclosure community, some information sharing and some improved collaboration and coordination across the aviation sector so really jam packed. I want to step back just a little bit and explain and go through just a little bit of the groundwork on what the aviation cyber initiative is and so Randy can you talk really briefly about what the aviation cyber initiative is in the try chair. You're good you love to aviation cyber initiative actually started probably around the five years ago the 2016 timeframe. It was it was originally focused on on aircraft only there was some testing their DHS S&T folks were doing and and was was of interest to everyone. And we've we've taken that and a lot of things has happened since that initial surge or testing, if you will. The national strategy for aviation security was published by the White House about a year and a half ago we're getting close to two years ago. And it specifically was a was an update the NSAS needed to be updated was dated back 2016. It was old, and it didn't include anything about cybersecurity nor did it include anything about us so that was a big upgrade. It also defined the aviation ecosystem, which, which really helped us out and kind of defines the swim lanes if you will for for aviation security. So, so those those six swim lanes as defined include the aircraft obviously but also talks about airports, talks about airlines, airlift, airlift being the cargo derivative or equivalent of the airlines talks about actors, which can mean anything to to third party vendors on the airport it could it could be anything has to do with people, and it goes into aviation management, which is all the infrastructure necessary to run the aviation environment. So, so all of those things are swim lanes it's a little easier for us, the aviation cyber initiative to to to talk about this when you're talking about the various swim lanes as opposed to a big amorphous blob. So, what is the ACI. Well, I talked a little bit about it initially, a year ago we were chartered by the Secretary of Homeland Security, the Secretary of Defense and the Secretary of Transportation to to be a tri chaired organization. So I have my DoD colleague, Al, and said my FAA colleague, and we lead this effort across the whole of government to include industry and to include anybody that's really looking at aviation cyber security to pull them in and try to reduce air risks and increase their resiliency across cross aviation. So I'm very proud to participate in that that gives you a little background I don't want to get too much in depth in it but I think that they should answer your question. Yeah, really good. Thank you. So, I want to talk a little bit about some recent situation that I kind of really feel was a really good example of a watershed moment within their research or aviation and public sector communities. So this happened about a year ago. And Jen Rapid7 made a pretty, it's a pretty interesting disclosure on civil aviation last year. Can you walk us through that and kind of give us your perspective on the process. Yeah, sure. Absolutely. So we have a researcher, Patrick Keiley, who has done transportation in the past. He has done a lot specifically with automotive. He's also a pilot and he's building his plane as one, apparently. And so, as part of his sort of, you know, passion and enthusiasm in this area, he was investigating various things. And he heard about a technology and used in small aircraft that is well known in the security community as being sort of quite trivial to exploit. So, the technology is Canbus. And he heard that it was being used as a way of sort of connecting avionics. And the avionics, you know, being the parts of the control plane, he was like, oh, that sounds quite dangerous. And he was well aware that a few years back there had been a lot of noise made around about Canbus in automotive. And there had been very widespread discussion in the automotive industry and as a result, many automotive organizations either moved away from using Canbus or introduced additional mitigations and protections. He had not heard of such a discussion in the aviation space and actually on talking to more people who work in aviation. He found that, you know, generally security people seem to believe it was an issue but said that they were having trouble in having this be talked about at the right levels and get enough attention to really make a change. So he did some research on that. And our goal with the research from the beginning was never to sort of shame a particular vendor or embarrass a vendor. This wasn't a, we're going to take a specific vendor system and uncover new vulnerabilities. It's much more of an architectural issue and an unknown issue. So the goal was to look at a few different systems verify that Canbus was in fact being used and then talk about why that's an issue on a sort of strategic level and try and stimulate some discussion in the aviation community around it. As a result, and also because we were dealing with aviation and, you know, we recognize that aviation is a different space. It, you know, there is a sort of life and death element when you're talking about things to do with aviation. We were very cautious with how we approached it. Our typical disclosure process, which is documented and on the internet is normally a sort of 60 day process sometimes a bit longer. This process took a year and a half. And the reason was two things. One, we very much did not want to cause hysteria, you know, as somebody who flies a lot. I know how it how easy it is to get spooked on this stuff and I, and the rest of the team were very keen not to have that be the case. So we wanted to be very thoughtful with the approach that we took and we wanted to be very balanced and as neutral as possible. And the second thing is that we really did want to try and stimulate this discussion that was our goal. So we wanted to try and involve as many people to participate in that process as possible and really sort of like immerse the community as much as we could. It was a somewhat mixed process. You know, there's not, there's not a lot of tried and true ways of doing this in aviation, even for an organization like Rapid Seven have done a lot of vulnerability disclosure, a lot of vulnerability research over the years. Aviation, you know, is every sector is a little different and aviation is certainly also a little different. And what we found was that generally people, including many of the people on this call were very, very generous with their time and their insight, their expertise. So we had a lot of people who are willing to talk to us, which was great because we were worried at the beginning that that wouldn't be the case. But we also found that there was a bit of a flavor in the discussion that came across of, you know, people sort of pointing out that we weren't aviation experts and that physical security would take care of this. And that, you know, we weren't understanding how pilots work. Now, here's the thing is, certainly I'm not an aviation expert and I was very glad to have input and have expertise from people who really are immersed in this space. It was a great learning for me. But in general, I think anybody who works in cybersecurity is always going to be a little leery of any sector that really leans hard on physical security as a response to cybersecurity challenges. And while we were very aware, always very aware and acknowledged in the report that physical access was required to exploit the system. We're also very aware that these systems stay in place for a very long time and that motivated attackers will find ways, particularly when you're talking about smaller aircraft, which might not have quite the same degree of physical protection that, you know, sort of larger commercial aircraft might. So that was a little bit of that. And then I think, I think Patrick probably felt a little bit affronted every time he got told that he didn't know how pilots worked since he may not be a commercial pilot, but you know, he has flown planes. So it was good to get the feedback. We were super grateful to have to have the ears and to have the feedback. And we certainly wanted to challenge our own investments. But what we did learn through the way is that, yes, it's always, I think, with any coordinated fundamentalist disclosure and we are big advocates for coordinated fundamentalist disclosure. It is critical and key to build empathy empathy to listen actively to take on board feedback into challenger and assumptions and to try and build trust like I think above all trust is where you want to get to But you also have to know when to hold the line on your like, yes, challenger and assumptions, but hold your line. Once you've done that and you've verified that your findings are as you think they are. So it was a good learning process for us. And I think in the end, the disclosure went well one thing just before I wrap that I'll say is I think part of the reason the disclosure went so well was because of the role that DHS is a played. I think that having the vulnerability management center part of which Katie was kind of running out of time. Having you guys independently verify the research findings and then decide that it was a significant enough issue to put out your own alert to coincide with our report. That to me was a pretty big deal. It was actually kind of a game changer, I think. I can't say whether we were about to pressure if that had not been the case but I'm certainly very grateful that we didn't have to make that choice so thank you. And thank you to everyone else on the call who helped educate us along the way and gave us that time in their feedback. Yeah, it's definitely a complicated situation I think a lot of times it's difficult when you think about every sector is different and you know your ICS systems your embedded systems your safety systems. It's really complex and doing that coordination up and down the hardware stack is a little different than you would necessarily see in say a digital services sort of platform or in a general typical traditional software platform so it's definitely learning I think for everyone and there there are some ups and downs and that I think that when you look at it you have to take it for we're going to look at climate versus weather you know if we look at one each individual disclosure it's pretty tough but in the end we learned a lot so can you talk a little bit about the DHS got involved in this disclosure can you talk a little bit about how DHS got involved in this closure and what role DHS took. Sure, you know we have the vulnerability disclosure program that DHS runs great group of folks as a matter of fact I think J Angus is going to have a panel at the aerospace village on vulnerability disclosure. So I invite you to go and listen to that great great folks they approach me great folks including you Katie approached me and said hey I've got an aviation thing I need to I need to get it to you and they don't need you to understand and we need to know kind of where we go from. So I was able to get the briefing from from rapid seven actually got to read their final draft report if you will. And you have to realize an aviation aviation is a big thing so I've got a big background in aviation but the canvas where it's installed you know different aircraft vendors. It's a it's it's a huge try thing to try to go oh yeah I got this. So, so how important is it well it did require physical access so I can stand down a little bit from that I think everybody understood that. But, you know what, how does it affect commercial aviation how does it affect small small aircraft or the general aviation folks. So, so I said look we need to get it to two places we need to get to the FAA they're the regulator they should be aware of this, because they need to make a risk assessment if it's if it's a big thing. We also need to get the aviation isaac aviation isaac has their members that can quickly get it out to aircraft manufacturers and and actually the folks who are building systems and determine hey, what can I do with this or what should I be doing this. In the future, I think this was the first time since the ACI has been stood up that we've had a vulnerability disclosure come to me. It's not the last time but it was the first time. It was handled very well I think by rapid seven I think I think they understood going in you know the physical access aspect was a was a big deal, the acknowledgement of that. It wasn't the skies falling it was, hey, this is an issue and we need to address it. And I love that part of it. I know our VDP folks the vulnerability disclosure folks were very interested in hey look we need to send out an advisory. It's going to mirror what we did on the cars years ago, and it's also going to say hey it does require physical access. So then once again not a, not not the skies falling we found a vulnerability but a, this is something you should be aware of when you're when you're architecting a system. So, so it was very, it was very good for me. I enjoyed getting involved in it to this level. And then the focus for that vulnerability disclosure is is, can we get, you know, how do we approach a mitigation if we found a vulnerability. How do we approach a mitigation, how do we close that down and make the system more safe and secure. And I think we achieve that in this particular case. Yeah, it's, we go back to that physical security and I think overwhelmingly one of the big things that was very different from say this disclosure to a disclosure that we might see somewhere else is that this is more of a architecture it depicts a lot of different things versus say one particular product and one particular version and so just the impact was very inspiring and we didn't know where things were so we really had to rely on the subject matter experts and we really had to bring in a lot of different people to ask and find out and so said can you really quickly tell us about your work and how you were brought in and talk to some of your colleagues at the FA and the safety that went involved was involved in that. Yeah, sure. So, you know, Randy kind of brought this up last year. And I think I agree with you Katie I think it was a watershed moment it's a good way to describe it. We had a researcher that came forward with the vulnerability rapid seven came came to us and they say hey look this is important, you know this is an issue. And so it really helped kind of spur a set of conversations within the FAA, both from our ABS which is this aviation safety organization within the agency, the CISO was on the call, and it helped us, I think, you know really connect different offices. And so granted it's a physical security issue. And I believe the other thing that happened also was TSA was notified. That's the agency that's responsible for all of aviation security in the US. So it really helped us connect with different stakeholders and and really lay the groundwork for a process. There's really a lot of back and forth in such situations between the safety engineers, the regulators, the, the researchers, and that's really in my mind very, very good, and it's healthy and we want to spur those types of conversations. Everybody's not going to agree on what what exactly how big the risk is. It's going to get a little difficult to come to get to the same page but I think having those conversations is really important and I think this event helped us do that. So thanks to rapid seven thanks to DHS and the ISAC for or take the lead on this. Yeah, yeah. So that actually leads us right into Jeff, can you talk about the ISAC and the ISAC information sharing and so if you really want to affect change, you need to get the information out there you need to make sure that people understand so can you talk to us about your your involvement in this. Thanks Katie. And so, first off, we're just really highly appreciative of rapid sevens approach to this and allowing us to be invited in. Similarly to Randy and the rest of the folks at the aviation cyber initiative for being so inclusive, let us, you know, kind of get this information and be able to pass it out to the members so like has been mentioned earlier. You know the aviation ISAC role in these security disclosures is really a connector. And, you know, Jen highlighted how important it is for communication to be happening during these events. And typically when they start it's really hard for a researcher to find the right person in the industry. Also mentioned here though that this was a little different. This is a technology that's broadly used across the industry and it's actually something a lot of other folks plug into. Many times when a researcher has a particular disclosure they want to make it's, it's about one type of product there's one vulnerability in some design of a specific component. It's a great hold of that company frequently they're calling us because if you want to get a hold of somebody in the industry, it's a lot harder nowadays actually to find the right person and we just happen to have them all connected in our community. But this one, this one was different and it was a great learning experience for us all, because there was a little bit of a shorter window on the industrial side but as we look back at the event really made a lot of sense. And there wasn't somebody that was one particular company to take a look at this this was more of everybody saying hey wait a minute we're all plugging into this, we really need to take a look at this as a larger issue and as a systems issue. In the end, we did, and it's been mentioned as well, like there was a physical security component that is one component. But in the whole concept of layered security, you know physical security is just one layer. And when you find that there is an issue inside of any layer. The objective in a really good layered concept is to harden that layer. And this was a really good example of that it's like it doesn't matter if if it's embedded in a lower layer let's get every layer as hard as it can possibly be, and make it for anyone to make that that complete penetration. So, I would agree with everyone I think this was a really good example of how to do it right. And I think what was really impressive for us to as the trust gets built across the engagement between the researchers and the industry was really good to see, you know, what Jen talked about was the perspective from right from the start, hey, we don't want to get people overly excited or hysterical about something we really need to understand in the whole equation of risk, you know how big of a risk is this and you know how does that risk get managed, and their approach that they took that cautious approach of really trying to get that understanding and validation was really critical to the successful outcome. So, I think this is a great opportunity and it was a great example of pieces that didn't ever, you know, typically work together and who were new to each other and maybe didn't even know that the other existed, and we're able to really pull together and there were some ups and downs, and in the end, rapid seven put out their vulnerability information DHS put out a complimentary security advisory and I think it sort of laid the groundwork so now we kind of talked about that I really want to pivot to the currents. So there are some really awesome progress being made in the aviation and researcher communities, and it's, it's hard to even pull a couple topics when I, when I sat down to kind of go through and say what can we talk about here. It was, there were just so many topics out there that the community is, is moving so rapidly and I think even the aviation village itself is something that 10 years ago, we wouldn't have even thought of, we could pull together and do that and so there's so many wonderful things. John, Boeing is getting involved in some really awesome community outreach and community engagement. Can you talk a little bit about the tech council. Yeah. I'll start off with you know, aviation is unique space, and we have a very strong safety culture. We have a unique development process and, you know, to some extent I think we view ourselves as being special. You know, Jen kind of mentioned earlier that when people come from outside that community, the natural antibodies kick in and we find reasons to kind of discount the feedback. I've been working in this space for 10 to 15 years and, you know, reality kind of kicked in on me when the Stuxnet virus was was disclosed and it broke all my, you know, stereotypes of aviation and how we're unique. You know, and oftentimes I feel like an evangelist out trying to spread the good word. Well, last year was a significant milestone and I'd say turning point for Boeing, a researcher, got ahold of some of our executable code, reversed engineered it and disclosed things that were actually quite surprising. I think we were surprised with the tools that allowed them to actually go in and, you know, kind of view the code in, you know, in a space where we didn't think was really possible. We actually spent a lot of time, several months analyzing the code we were in our lab we have very extensive labs that replicate the airplane. Quite accurately, at the end of it we actually went out onto an airplane. We brought all the systems engineers out to view it. We went through a bunch of scenarios. Some of the claims we actually went well beyond the claims. We did a pretty thorough test on a 787. And at the end of that, our response, you know, not knowing, you know, it wasn't the real intent but I think it was viewed as hostile. And I got that feedback from several folks, you know, in the aerospace village. And so after our analysis disclosure we held a meeting with a lot of key stakeholders a lot of the airlines had a lot of questions for us, you know, what do you think of this is it real. This is one of the key of our synopsis. A lot of the government folks and Katie you're in one of the meetings with us we had a spirited discussion with lots of different government people as you remember. And in that meeting, Katie kind of said you know john you really need to reach out to these folks and I can help you if you'd like. But we went internal to Boeing and we had some spirited discussions because this is really uncomfortable, you know, changing how we've operated for a long time is not the easiest thing. But at the end of the day, we said you know what we need to do this because these people are looking at our designs, they're not, you know, I don't think there's any ill will, but we need to embrace them and we need to learn from them. So, we set up this tech council, and we had several meetings trying to level set a little bit it's a little hard via the phone. And so after the RSA conference in San Francisco in February, we invited them all back to Boeing. And at Boeing, we, we kind of went over some of our design methodologies and you know met face to face and that's a real. powerful to get to know people and much more personal level. We took them into our labs showed them what we're capable of. We even arranged time on a seven eight seven simulator in fact one of the gentleman was a military pilot and we set up kind of a difficult landing conditions and, you know, low light conditions, etc. He made it down good, which was positive. But I think it went a long way to mending the adversary relationship. So, it's part of that we have part of this team. They have a some folks have a claim they'd like to investigate with us and we're in the we were in the process of bringing them into Boeing into our lab to kind of evaluate that when, you know, the kind of slow this down a little bit. But, you know, I really want to embrace this and I want to expand it and, you know, at DEF CON we actually plan on bringing, you know, real hardware, setting up some kind of capture the flag event to kind of embrace this community much more. Another aspect of this, we actually matured our vulnerability disclosure process it is now very easy to find on the Boeing website, at least I hope it is for folks I was able to find it. And you can send your vulnerabilities encrypted we provide that means. And we're starting to receive a lot of stuff in. And a lot of things that come in are shared between all the stakeholders and as the product guy for the airplane, we see all of it, and we actually evaluated surprising vulnerabilities that are in the IT space a lot of these systems are used in some form in aerospace and so we have to evaluate all those and see if we have the same issue that maybe the original claim came in with. And so we've closed, you know, saying we're still crawling here. I'm really hoping to get more engaged. You know, I think it's critical that we interject this into our designs, it's a different view and, you know, it's always powerful to get diverse opinions and diverse perspectives and those outside of aviation probably we're powerful at looking at our designs and we may be so. Thanks. I know I remember sitting in some of those meetings and I remember thinking to myself, what, why don't we all just talk, you know, why can't we just, why don't we all just talk and I can imagine that there were some people in that room who thought that I was just that crazy lady from Homeland Security with these, these wild dreams of working together but I'm hopeful that we're going to see more of that and that that has actually worked out really well. So, yeah, you know, we're still in steps here and I would like to, you know, commend Boeing because Boeing did all the hard work on that you know the heart the hardest part is getting going and and taking all of the, all of the steps that needed to happen in order to bring people in and really get down into the labs and get into the weeds and that that's that takes a lot a lot of effort and so and I can go into that. So Sid, can you talk a little bit about the FAA structure and the work with the tri chair and some of the things that you've been working with. Yeah, sure. Thanks Katie. I, I agree with everything that has been said on this great panel. I think, let me let me make a few points. You know, we kind of oversee and regulate civil aviation. So we have a unique role we are the premier aviation agency in the US. We have the authority over all the aircraft that fly in the US and all the airmen all the pilots, we certify them, we certify all the aircraft. And we also conduct the air traffic control all the airplanes that take off and land are are controlled by the FAA. So, it's a pretty big role it's a pretty responsible responsibility, I think safety is our focus, and we are a safety agency. And so, the way I see it is that cybersecurity is part of our safety responsibility. And I run into that all the time. As a tri chair for the ACI, you know it's it's really a big task is culture change we are trying to bring that culture of security and looking at how to improve the security of the whole ecosystem into an agency that's focused primarily on safety. So in terms of vulnerability disclosure we don't have like a formal program. We don't have like a way that you all the researchers can come forward and directly to us. We rely on DHS, which is our partner through the ACI. But that does not mean that we are not open to all of your ideas. I think the fact that I'm here on a panel. You know, I'd love to hear from you all. And, and, and we love to have that that dialogue about what what the risk is and what some of the vulnerabilities are. So, I want to echo what has been said earlier I think trust building is incredibly important. We all come together and like Katie said we need to be talking all the time and exchanging notes and exchanging ideas. Let me share a quick story about two years ago I was at the Pentagon. The UD has this team called Gen five. That's basically a, you know, a bunch of very smart people just like you all that we're looking at some of the vulnerabilities within a DSP, which is our system for surveillance based air traffic control it's satellite based I'm sorry, air traffic control, we are moving away from radar into satellite base GPS, you know radar control control. And so they had worked on this project. For about a year to look at some of the vulnerabilities within a DSP which is a huge investment for the FAA and for the American public. And with the CISO at the time and some other executives from FAA and, and we just thought it was fantastic, I mean to invite these researchers into the Pentagon, and have them test an aviation system. And here's some other ideas, they actually came out with a report, which was super insightful. And we took it all the way to the National Security Council. And that has formed a basis for a lot of the follow on work that's going on within ACI for a DSP so you know I told I told my boss this is something we need at FAA to we need a team like this to come in and test our systems and and bring a completely different perspective than what the agency is used to. So that coming together of the researchers of the research community of the safety folks, you know of the air traffic controllers and the IT people, all that has to happen because it's, it's really about bringing all those disciplines together to tackle which is what is a very sort of problem. I mean we are looking at risks across the ecosystem. And so that conversation needs to happen between the different disciplines. I just did a online cyber course through Harvard a month or so ago I finished it was like an eight week class. Let me just share a couple of things every organization, including aviation has operational risks, reputational risks and legal risks when it comes to cybersecurity. So a cyber breach can cause huge damage, millions of dollars, in terms of your operations in terms of your reputation, something to think about. Second is that we need a culture within cybersecurity where we reward people for being skeptical, you know you don't want to just kind of reward people who are who are sort of agreeing with what's going on you want people to look at skeptically at what's going on and, and tell your different way to do things we want to reward people who want to break things, because that can lead to more cyber secure for sure. And a lot of the challenge that we face today as organizations has to do with information sharing has to do with the culture change and trust building and so all that those are the tasks that we are all involved with. So information sharing should be happening all the time and that's what the ACI is designed to do within the government between our three big departments and with industry. I want to, the last thing I want to say is also the fact that the aviation ecosystem faces a lot of state and non state cyber threats. The threats are very real. There are known vulnerabilities within this ecosystem we all recognize that. That can impact the operations within the national airspace and civil flights. And the fact is we got to tackle those threats and risks together. So the FA issues cyber situation reports today which, which are, you know, which address specific equities across this ecosystem. And some of those are externally tailored to partners outside of FA. So those include, you know, other federal agencies and industry partners. And that's, that's what I have so thanks Katie. Yeah, awesome. It's really exciting to kind of hear some of the things that that get passed around and just really are exciting initiatives the Pentagon has I know been deeply involved in aviation for, you know, I was in the Air Force so you know, since forever and so it's really great to see these very, very established organizations get excited about working with research and breaking things that just that just makes my day. So these are really awesome initiatives and I'm really excited about them. I want to move to the future for a little while we kind of talked about the past and we talked about the current and then I want to kind of talk about the future really quick. And so I think the DOD's got kind of a really fun new initiative that I'd love to hear more about and I think it's called the, the in factor so Al can you talk to us about the in factor. Sure Katie. Hey, just as a little background before we dive right into the in factor so we talked a lot about identifying and sharing cyber vulnerabilities, sharing information, but going forward the future is that we need to work together to close those vulnerabilities using a threat informed risk based approach. And that points us to two trends, the aviation ecosystem must address, you know, like said said, the first trend is the cyber threat to aviation is real and growing. So it's highly likely that advanced nefarious cyber actors to include adversary nation states will use cyberspace to steal our aviation intellectual property and to conduct cyber operations to damage the reputation of us and allied aircraft and aviation industries to gain a competitive advantages for their own industries. And so from a national security perspective, improving the cybersecurity and resilience of our own nation's aviation ecosystem to counter this threat is key. And for the Department of Defense, we have to be able to project power, defend the homeland and protecting critical aviation infrastructure is part of that. But there's one thing we acknowledge this challenge is not something the Department of Defense, or the government can do on its own. So recognizing these two trends, recognizing that trend, you know, requires an increase in a public and private sector collaboration or a whole of nation approach. I don't know if you're aware, but Congress recently charted the cyberspace solarium Commission, and in this commission report, it identified the need to increase public and private sector collaboration. And the goal is to improve our speed and agility and addressing cybersecurity and resilience threats. And so the end factor that you referred to, which stands for the National Federation of Aviation Cyber Test Organizations and Researchers. The end factor is a great example how the aviation cyber initiative is pushing to work on this. And so the thing that the aviation cyber initiative does is, you know, we bring together that whole nation approach, bring together the cyber experts from federal agencies, state agencies, industry, our federally funded research and development centers, our university affiliated research centers, our national labs, all working together in the end factor to achieve three lines of effort. And those lines of effort are one is a catalog collaborate and connect. And so we're just at a top level. We talked about catalog, our big push is to create a national level aviation cyber resource guide. The goal of this resource guide is to be an online accessible and searchable database of aviation cyber research development tests and evaluation resources, expertise of facilities and capabilities. And so, if you're part of the end factor and you're working on initiative to counter to look at cyber fuzzing or any of these other kind of cyber trends. So if you could go to the resource guide, you know, give you a list of who's working in that space, what kind of capabilities they have the test, and what and more importantly, how you can get to hold, get a hold of them to further collaborate. And so the collaborations are second line of effort. You know, the goal of the collaboration is to create a persistent collaboration forum where these cyber experts this whole nation approach can get together, and be a spotlight or showcase capabilities, share information on projects. And more importantly, if a like let's say for example you're working a project on intrusion detection cyber anomaly and intrusion detection. So you can come to the end factor and present your project and and give us asked and in the ask the goal for the ask is that the end factor try to close those gaps. And when we talk about closing those gaps, we call that a connect. And that's the third line of effort to connect projects efforts research papers with resources to do things like test validate and move forward projects and just just a couple examples, you know, on the collaboration side. And Johns Hopkins APL did a presentation on how they were doing cyber modeling on various aircraft. And the aircraft, they were doing the modeling on it so happened, since john Craig was on. They were Boeing aircraft. And so, you know, after the meeting, john and his team got together said hey, we probably need to know more about this modeling effort and oh by the way maybe we can work with john Hopkins to make it better. And so right now that's what we call a connect. We're working to bring those john Hopkins APL and going together to talk about how they can enhance that cyber modeling. And then on the resource sharing side. We had a project come forward that was working on a cyber anomaly and intrusion detection capability was AI machine learning based, and they needed a data tab to pull data from 1553 bus. They didn't have it. And they also needed large quantities of data to support, you know, training a machine learning and AI capable tool. And so we connected them with the Air Force Church lab and provided a tool called vampire which is a aviation bus tap and ship that to them. And then john Hopkins APL again came through. And they're they've been working on the data sharing effort that's been fantastic. In terms of connecting so those are the kind of things that the end factor is trying to do. We're trying to do it at scale. And so that leads us to the, I'm sure people are saying so how do you participate in the end factor. Well, we got a couple, couple asks, you know, first off, we're focused right now on us organizations. So if you want to be a part of the end factor, what we asked you to do is three things. First, we want you to one, agree to populate our aviation cyber resource guide with your company's capabilities and resources. So that's one. The second one is, we want you to participate in our tight 90 minute monthly forum. It's the end factor collaboration forum. And our goal is to be able to showcase all our major participants as we go forward. And to participate and follow on meetings, you know, should those connects happen. And the third one is that is the connect part that if you have capacity and if you have resources, we would like for you to help connect others to serve as a mechanism to accelerate our cyber innovation and the aviation ecosystem. And so that's the kind of the three asks. I tell you, just yesterday we had a nearly 100 participants from across more than 60 organizations participate in the in factor forum, our FAA tech center, and do the guys are working with the aviation cyber resource guide by the end of August, at least the first instantiation of it. It's based on the FAA's technical capabilities library. And I would say, no in factors growing. And I think it's exceeding many of our expectations. And what we want to do is get more people to work with us. So we can do what I like to say, collaborate with a fact. So that's, that's an in factor going forward, working to address the threat, and to increase our public and private sector cooperation across the aviation ecosystem. So let me ask you really quickly, is that is this initiative specific to military aviation or is it open to civil aviation as well. Yeah, it's, it's, it's not limited to military aviation. In fact, it's not a DoD initiative. It's an aviation cyber initiative initiative. And so, you know, we have a charter to engage industry. And so we're, we're working to do that in spades and, you know, at least a couple of times a week we're reaching out to bring our industry partners and really small businesses. You know, which we think the small business businesses are real powerhouses and innovation. And, you know, when we can connect them with the larger big companies like the Raytheons and Boeing's of the world that we can, we believe that we'll be able to accelerate aviation cybersecurity capabilities. Awesome. So we have about eight minutes or so left. And so I really want to close up with some of our closing thoughts and some just kind of words of wisdom and things that we've learned from each one of the participants. So, Randy, can we start with you some what are your what are your words of wisdom things to take away closing thoughts. Well, I'll tell you what, Katie, I'm super proud of what we're doing under ACI. I think it's, I think, I think it's time that we've we've reached out across the entire ecosystem and try to pull people together and have those conversations, address those those vulnerabilities. And, and I think, frankly, I think this is beginning to work. You know, working for DHS CISA, you know, I've got that outward look. I'm looking to industry. I can pick up the phone and call virtually anybody on this panel at any time and if they're not in a meeting, which they always are, they'll answer the phone and we'll have a conversation. This is this is collaboration at its finest. And, and I got to say, you know, and I know we're limited on time so I'll stop it here. I love my job. And these are the reasons why I love my job. The people on the panel, the things that we're trying to accomplish, and what we're trying to do through ACI very proud of that. It's wonderful. It's something that I think is really going to make a difference and that's, that's very exciting. I'm excited to be proud of it or to be part of it. Sid, what are your, what are your thoughts? Well, I, I want to say that COVID-19 obviously has affected the aviation industry. It's transforming is going to transform how we travel. We've changed so many things and we're, you know, we're going through a change right now as an aviation industry, but that does not mean we ignore the cyber threats. The work we are doing here on this panel and the work I do through the ACI is incredibly important. Cyber threats are real from both state and non-state actors. It's a combination of physical security and cyber security and the information technology and operational technology, safety, security, all of that. And the ecosystem is a vast complex network, so there's a lot of vulnerabilities to it. And cyber breach can happen anytime. There are threats out there and an attack can cause millions of dollars of damage and a loss of reputation. So it can shake public confidence. It can change again the nature of flying. So we cannot ignore it. What we're doing is super important and I'm privileged and I feel lucky to be part of this team of so many smart people who are doing such incredible work. So thank you for having me. You have my contact information, so I urge you all to reach out to me anytime. Awesome. John. So, you know, the last year has been interesting and we have got much more of a focus at Boeing on this. We do daily report outs to the board of directors. In fact, we have a champion on our board. We are leveraging our enterprise to not only help us with incident response, but, you know, helping to beef up our design guides. We have expanded our product CERT team and we're being much more proactive at looking at things. You know, and then we're looking at things across board. How do you create a process to evaluate threats and risks and we're looking into that we're actually working with, you know, industry and the government on, you know, how do we do that effectively. But key is we're really starting to engage outside of aerospace and I think that's going to be the thing that really helps us the most, you know, the tech council is one but participation at deaf comms so we can build those relationships, RSA conferences, etc. So it's pretty exciting. It's really exciting right now and a little uncomfortable, but it's good. So, thanks. Yeah, embrace the uncomfort. Katie, I'm honestly kind of blown away. You know, I think I think the whole purpose of the aerospace village is to increase understanding and appreciation of the importance of cybersecurity in aviation and to do so in a way that builds trust between the security and the aviation industry and the government, which all play a very, very important role in advancing cybersecurity, not in aviation. And when I sit and I listen to my family panelists talking about some of the amazing initiatives they've got going on, and here are their attitudes and their responses to the research and that kind of stuff. I just, I feel like we're in such dramatically different position to where we were a couple of years ago. And I hear people saying that they want to hear from the research community, they want to hear from the security community, they want to partner and collaborate. And that is, that's an incredible opportunity. And I hope that anybody listen, particularly people who are participating in security research in some way, that they could love hope and optimism from this and feel that they can engage and that they can build trust. They can build empathy and they can get involved. It's very exciting. I'm, I'm the trust and the relationships I think are some of the most important factors and all of this it's, it's getting involved. So, yeah, getting involved, Jeff, information sharing what have you got, what are your thoughts. I think this has been a great discussion. And, you know, we only went through one example, but at the aviation Isaac, we've had several of these events happen over the last couple of years, then each one of them pretty much follows this pattern of building a great relationship with a security researcher, finding out, you know, what a vulnerability is that they've discovered and then going through that validation process of, you know, isn't really an issue or isn't it an issue and then, you know, working through the radiations and disclosures when those have to happen. And, you know, likewise we have found this to be incredibly eye opening. And, and, I mean, so much so with the research is we've even hired one which helps us tremendously particularly in the work that we're doing now so I am glad that there's been kind of this breakthrough and that we're seeing, you know, the bridge is being built and only think that it's going to get better. Awesome. Al, what are your, what are your closing remarks closing thoughts. Okay, hey Katie. So kind of echoing across the board. I think the most important thing is what one is we have to recognize that the thread is real. As you learn from the maritime sectors, no petra attack. We shouldn't have to wait until a serious cyber attack of our own on the aviation ecosystem occurs. And the key to, to preventing that is to embrace the idea that there's safety in the herd. You know, we need to strengthen the aviation herd, you know, by continuing to share information vulnerabilities and work together to address those vulnerabilities to improve our aviation cybersecurity and resilience. And so threats real, and there's safety in the herd. Terribly true, very true. So I guess my closing remarks then I will say that the thing that I take away from all of these things is that I go back to critic the coordinated vulnerability disclosure and I say that coordinated vulnerability disclosure is an essential part of any security research. I think that everyone here and everyone that I've met from varying backgrounds all over the world, everyone has the same goal. We may not all speak the same language we may all might talk past each other but I think ultimately this we have the same goal and that goal is protecting the end user it's reducing risk and making people safer. And so I think the last thing that anybody wants is to inadvertently put countless people's lives at risk economic situation at risk, because we published a roadmap that if that fell into the wrong hands could just wreak havoc. So I go for coordinated vulnerability disclosure because it really is a balancing act. It's a, it's a process that allows us to mitigate while we're, or to balance while we're trying to mitigate so that's my that's my closing advice I feel like all of the the, all of the progress that we've made is so wonderful. We have, we have a long way to go. I think that I, I love the fact that we're all talking to each other and we're working together. But I want to make sure we know that there is a road ahead of us. And that road to me it, the future looks bright and I'm, I'm excited for it. So, thank you guys for coming to the panel and listening to us and if you have any questions, our contact information is up. I think there's going to be an opportunity for some live chat later at some point so you'll be able to find out more information about that. If you have any questions or want to talk more to us please feel free to reach out. I think everyone here would would invite a conversation so it was great talking to everybody today. Thank you.