 Hi everyone. I'm glad to see everybody's in this room and we're gonna talk about a lot for really, really largely making all of my software, so I'm getting a little bit time jackalock right now. So we're gonna talk about a lot of DevSecOps functional safety, functional security. Okay, we are getting buy one get one talk by the way. Anybody knows buy one get one? So you're in this room, you're getting another talk as a parallel. I'm gonna pick up a lot to So focus on me or otherwise, otherwise we cannot get it done. So anyway, we're gonna talk about a lot of DevSecOps, but as well as we're gonna talk about what is really going on Automotive Industry. Anybody from Automotive Industry by the way? All this? Okay, great, great. I see three, four people out there, five. You don't think that you may not part of Automotive Industry, but in reality, you are supporting Automotive Industry, somehow. Because you are contributing the software community. So let's continue. One, two, three, it's not lit as illegal stuff. There you go. And this is me. I'm Hassan Yassar and I have been working in the software world almost 30 years, I'll say. It just keep moving. I'm a faculty member and also I'm contributing to the DevOps, DevSecOps. Anybody heard DevOps and DevSecOps? I saw somebody heard it. Anybody heard DevOps, DevSecOps? Yeah. So you're gonna hear a lot today as well, and which is kind of connecting the open source community as well as what we are doing for Automotive Industry. So I have been contributing many things. I'm also a faculty member at the Carnegie Mellon University, which is the Carnegie Mellon known as one of the number one CS department in the world. So I'm teaching over there for DevOps and DevSecOps course and software security course as well. So what are we gonna talk about today? Why you're here? So you're gonna hear a lot. What is the verification and validation for our safety? Our safety, we are in the car and our safety is playing a key role with respect to the software. So I'm hoping that after my talk, you may be a little bit cautious or comfort depends on which car you're in. You may be scared. You may not be scared. It's up to you, but I'm not gonna scare you. I'll tell you the truth, right? So let's, we're gonna talk about why matters and we're gonna talk about what does the verification and validation means. We're gonna talk about how we can solve safety and security with the verification and validation. Then we can be safe and secure in a car. So let's dive in. Why matters? Why is important? Why we are talking about? When we look at the today's landscape Automotive Industry and it calls a software defined vehicle. Anybody heard about that term? Software defined vehicle, right? So almost every car it's going that direction right now. Start from Tesla. Everybody knows Tesla. It's a great example for software defined vehicle. Everything is software. Because of that movement and things are changing from this to this and this, it's basically what that really means. Instead of using a specific hardware, ECU, EC unit, electric circuit, electric control unit for a car, we are adding more software into it. In a typical car, as a kind of minimal level software defined car, we have about 68 computers in the car for each of the dedicated functionalities in the car. When we look at the software, we are talking about 80 million line of the software that we are running on the car. That's really scary. Anybody, you know how many line of software you're writing web pages? Depends. You don't know a lot because it depends on the backend component. We have 25 layers typically. We may have a million million lines, but in the car we have 80 million lines of the code we are talking about. Another interesting note is basically when we look at the bill of materials, it's going to get more by 2030, which is we are very close. Seven years, 50% of our car will be based on software modules. Now I asked at the beginning and I said how much you are in the automotive industry, but you are. As Linux Foundation members, as the group of people you are in this organization, you are contributing somehow. You're not realized that, but you are contributing somehow. That means you are somewhere in this bill of materials. You are not working for Volkswagen or BMW Mercedes, but you are contributing indirectly. What you are producing as an engineer in your open source, in your modules, in your organizations, somehow you are sharing your information with Stack Oriflow. Anybody in Stack Oriflow? Of course, we do a lot. We are using it. We are exchanging information. It is actually contributing into the big organizations. What is the really reasons people are going in that direction? Because it is saving so much money and also the control. Let's look at why. It's another why important. The goal is the automotive industry going to solve their defined vehicle. It's saving the life and saving an environment. Because when we go in that boat, yes, we have a lot of cars at the pass. Now, when we look at the modern cars today, it is getting much, much, much safer because we have so much sensors in the car. Previous talk was talking about a lot of adaptive systems for looking for the cameras and analyze quickly, give some warnings. I'm really happy to drive a car. It has collision detection on it because it's going to help me. If I'm slipping in the car, I would like to get some alert from the steering wheel. If I'm looking somewhere else, car should yell at me. Something like that is really affecting. Now, it's much, much, much possible using a software. That software is going to help us to drive much faster and much efficient ways as well as safer. I'm sure you saw so many stories people are slipping in the Tesla car. I saw that pictures. In California, this is happening a lot. People are slipping in the Tesla. Yes, it's wrong, but it's a safety people that are believing in it a lot because software giving that confidence for the people to be safe and secure. Another important thing I would like to highlight here is the preventative maintenance. What does it really mean? How many of you stopped in the road because of the car got failures in one of the mechanical parts? I did when I moved to the first time in the U.S. 1995 and I stayed in the road many times and I got pulled and pulled over the cars. I had a Ford car. Anybody from Ford Company? No. Okay. Ford is a different name. I don't want to say it right now, but I stayed in the road because of malfunctioning. I don't know when it's going to be expiring some of the parts, which I have no idea. It's just happening and transmission is gone. Brake is gone. Something is happening. With the software, it's basically monitoring how the function of the unit is keep changing and it's giving us a feedback as a preventative maintenance. We are getting alert from the car manufacturers. Based on your driving patterns, you're going to run out some brakes and some other parts, it's happening good software defined components. Now, software is becoming our life as safety and also the helping for effective driving. Another interesting thing I would like to talk about environment sustainability, the most of the cars, it is giving us a more full efficiency, either electric versions or the gas gasoline version. Truly it doesn't matter because you see a lot of improvement in the gas consumption as well because software is becoming key elements for monitoring those, giving the right routes and effectiveness of the environment. It's great. What's the problem? The problem is this. How much we are trusting the software car is a safe and are we building secure? Two things is important. One is, how do you know that we are building? Yes, it's great. It's working, but since we're using a lot of software in our components, in our car, it's getting more and more and more. Think about 50% of an 80 million line of code. Think about the sacred spec of the car that we are driving. Now we can ask the question, are we building the safe car? Since growing a lot, you think everyone's going to do it because time now there's another pressure for the market. Market is going to pressure a lot, get more and more and more building up the software. More means we're going to do a lot of shortcut in the security. What it really means that everybody is going to have a higher demand of driving those cars. It's going to get much cheaper. It goes back to the VEP technologies. Like VEP became our life with a lot of vulnerabilities keep increasing. We have a lot of demand building a function out of this. Also another thing we can ask ourselves, are we building the car securely? There's two different questions. Are we building the safe car? Are we building securely in the car? Based on the attack, this is another 2022 data sets. You see that when we are going to more software, we are getting more attack vectors for the cloud server. What this means, we are using a lot of software. It's connected to the cloud environments such as AWS, maybe on-prem, on-prem cloud environments. Many other things will be part of the connected because it can attack surface. Now it's not the car we're driving as fully mechanical that I own as a driver, as the owner of the car. Now car has connected to many other endpoints. Maybe mobile phone, maybe the data center, maybe another car, talking another car, you guys heard about, there's actually GM is working closely, building up mechanisms. The car can talk to each other. They can give a respect of when they do the red line crossing and it can get so basic, putting more safety into the car, but also we are creating more connection for other environments. It's becoming more connected vehicle. Over there updates and Wi-Fi, it's creating an attack vector for adversaries. So that's a great side of the safety and security, but there is another side of safety and if you don't do the right job. So what are we going to solve this problem now? How can we address the safety and security? So to address these two questions, we're going to dive into the verification validations. Anybody heard the verification validation? If you are in a regulated environment, you should know about the verification. I'm just going to give you a quick feedback on what the verification validation means. So verification, this is a definition for a myTriple. I'm not going to read everything, just going to give you a quick idea what the verification validation means. And this is the kind of a standard everybody's truly following. Yes, it may be changing based on where you are in your compliances, but the source of the verification validation comes from myTriple, 10-20, 10-12 documentation. Verification is basically helping us build the product correctly. How are we building the product correctly? Then we can validate the product. Is the right product that we are building? There's a two different concept verification. How are we building the product which goes back to my questions, are we building securely? Then we build a safe and secure product, which is the validation pieces. How do we assure that? And we're addressing the vulnerability that is keeping cruising for 80 million lines of the source code, will depend on each other. So we have to use verification and validation a lot, a lot in our new generation software, because now it's affecting our human life. Yes, web application may not be affecting human life. Sorry, I'm pointing on to you. I promise I'm not going to ask a question, but it's affecting our life. If you don't do that, now as an automotive industry, as an engineer, as a part of it, we're going to think about how can we apply verification and validation in our code. Let's move on. There's another graph I like sometimes hard to explain. So we kind of look and say, like we say, are we doing the right things, fighting with malware, doing the right stuff? Then validation is, did we build up the product? It's working or not? Or malware perspective can keep in your stuff? Right. So now, how are we going to build up verification? Well, let's go into the why pieces and why is important. The successful verification evaluation is going to help us eventually build and safe and secure systems. It's going to save a time, money, and to find that problem as early as possible. Verification evaluation is not just for compliances. If you look at the compliance perspective, it's going to become a checklist. So change your mindset. It is not a checklist verification evaluation. It is the process to help us to get a better product. If we put yourself in a compliance auditor perspective, it's going to become a checklist. When you become a checklist, you do a lot of shortcuts. How many are developers in the room? How many engineers? So as an engineer, if anybody asks you, have you ever done this work? Typically, yes, I've done it. Your world is becoming a checklist in auditor perspective. We trust you as a developers, but we would like to see a result of your code as test results, as a result. Yes, we love it. But auditor has no idea what they're asking. They're asking, are we building the product? And you, as a developer, you say, yes. But as a developer, you should say, go take a look at this output. Here's my artifacts that I'm going to show you what that means. It's going to change a lot of perceptions. Let's continue. There's, as I said, verification validation. Verification analysis is based on process requirements, activities, and methods. Same thing is also true for validation. One is focusing on how are we doing well. The other one is what the product looks like. So verification going to tell you there's a process pieces, there's a requirements, activities, and that's the same thing for validation. That depends on where you are in the system lifecycle. You can apply those techniques and activities. So I'm going to show you how you apply those techniques and activities with the DevSecOps concept of open source component. Again, you can take the picture. And if you guys reach out to me, I'll share the slides as well. So don't worry about taking the picture. Send me email. You will see my email at the end. I'll send prompts to everybody with the slides. So validation analysis is about getting more testing. That's the questions auditor is asking. Did you build it correctly? Show me your functionalities. Have you done the secured analysis as engineers? Yes, I did it. I did it. I did it. And then auditor says, check, check, check, everything is good. But actually, it's validation based on the test. How can we do the test? We have to generate an artifact software and wire them so we can build up the validation as we are going through. And then record is actual product. That seems a lot of testing. The successful and safe card is recording a lot of testing. There's many, many techniques we'll talk about shortly, but it's recording a lot of product mindset. Let's move on. This is a little bit heavy graph. And it's based on how architecture is really driving our safety and security. So that means if I am building up an architecture, think about the systems level now, everybody has to think about what is my architecture looks like. I can create a specific test cases on every phase of my life cycle based on architectures. You may be thinking new modern world of engineering, it's omitting architecture. It's not correct. Architecture is the source of everything. Either we might be using a framework like open source framework. We always use some other red code. We always engineer thinking about how it can be architected. It's becoming more and more and more important because we are dealing so much connected complex systems. That means we can build up a lot of test cases when we are in any phase of design, we can build up our test cases. Because going back to validation, testing is the key point to validate our product for a safety and security. Again, remember that. If you do it late, like I don't want to be a target, somebody is testing my car while I'm driving for a safety. I don't want to be in that car. You want to be in that car if somebody is playing with your car for more testing? No, I have to do more testing as an early, right? That's what we should do. Let's continue. So this is another long list of verification and validation activities. I'm not going to go every line. This gives you an idea. There are activities listed by IEEA documents that we have to follow verification and validation activities. There are many activities, requirements, design interface, traceability, all these activities that we have to do. Now I see because I'm going to skip too much. We'll talk about more. Here's another set of activities. It's a lot. So I didn't count it maybe more than for the activities that we have to do, build up a success of verification and validations. Now that's a problem. How are we going to do all these 20 activities to address in a short time frame? So everybody's thinking engineer builder, QA builder, tester builder, that probably you're thinking, right? Because if you look at the configuration audit, who's going to do the configuration or who's going to update check? How do I know that the update is working successfully on this thing? So it's becoming very challenging when we look at those activities. Let's continue. I'm going to give a little bit detail. And again, if you look at IEEA, we'll see a lot of detail activities and subtests and how you can do. I'm going to mention two of the key activities as part of secret analysis. If you are told about the secret analysis, we have to look at the owner's definition about the systems. We have to use the CIA to react. Anybody know what the CIA? Confidentiality, integral availability, not the three letter agencies. Confidentiality, integral availability, it's the source of the security. So we have to think about our car, our systems as individual component. One is a system level. Another way we're going to think about every component, how are we addressing the three key security analysis? Confidentiality, integral availability. Like confidentiality means today's car, it's very connected our personal information, PII. We're using connecting our phone to the car, our data goes to somewhere else. Maybe somebody's when we rent the car and you're connecting your phone, all your call inside the car, your phone number is in anybody deleting your profile when you're done with the rental car? Yeah, you should. If you don't, then you should, please. Still, there's a lot of, and I was in the, Robin and I, we were in the software Divine Valkyrie Conference in Berlin. The privacy, unfortunately, it's not addressed well. That means we are not doing a lot of verification validation probably. There's a privacy concern on the growing software world in the car industry. That's a sad news, but we're going to get much better eventually. When we look at the risk analysis, this is basically every test we do based on the risk. Based on the risk, we will look how the system is going to behave under certain conditions, maybe environmental conditions, maybe software security conditions. We see a lot of hazard situation like maintenance perspective. That's what we predict if maintenance is helping a lot to solve risk analysis. These are the couple examples. Let's talk about how we can get all this giant list of activities and make it effective working with our auditors and we have to use infrastructure. That's the connection with the software development perspective such as DevSecOps and the connection with the open source community. So far, that's auditorial perspective, safety and security. Let's connect how we can build up safe car. This is a quick definition of DevOps and DevSecOps. Again, I'm going to share the slides. DevOps is a basic set of practice and process, including all stakeholders. It's not just Dev. It's not just operational. It's requiring all stakeholders. If you remember the verification and validation activities, it is requiring all stakeholders in involvement. How can I get all stakeholders in involvement? I have to change my processes. Most of the organizations who are going to the software defined vehicle, they are using agile principles. They are using DevOps principles. If you look at the Tesla is a great example as an agile enterprise. They do all automation automatically, checking all this and they are using DevOps and with the SEC pieces is about adding a secret into it. You're using it like Renault and Ford, Porsche, Volvo, all of them are using DevOps principle to achieve those automation testing integrations. Otherwise, we cannot achieve the verification and validation. Otherwise, we cannot address the secret and concern. So if you are in the situation, start to learn about it. Start implementing. Let's take a look a little bit more. I'm going to give a little bit idea, just a quick reminder when you talk about the software development lifecycle. There are four environments where we write the code as developers, where we integrate the code multiple 50, 60 computers for integration pieces. Then we do the more testing in the staging. Then finally, we go to the production that car is working in the cell. So based on that four things we do all the time, but we can look at the environment until there are four environments we work and then based on our flow. Now we are talking about software, right? Software, we write the code, go into the testing. If you get more closer to the dev environment, then we can solve the problem much easier, much quicker. I'm going to show you quickly. Another one is the key concept you might be hearing a lot as infrastructure is a code. IAC means everything what we do, what we're writing, we can script it. So we can build up automations. It's not just the writing the code. We can set up our test environment, test harnesses, build up many cases like a digital team is one example. Most of the car manufacturer are building the digital representation of the ECE unit and then using the script and sharing with others. If you're an OEM, if you're a one of the contributor to the one of the components, you can build up the visualization part as scripted and share it together. Another topic is the key as a continuous integration, which we are integrating our code, not just the code. We are adding other elements with the code which is infrastructure, the app code and documentation together. So why I'm saying it, because you will see how the verification and validation goes together based on this concept. So if I'm building my code as an engineer, I can write the script along together. I can keep the word. If I'm a vendor, if I'm a contributor to the community, most likely using Docker containers. Docker containers are a perfect example for IAC because you're scripting your environment based on the base operating systems, you're adding your configuration, you're sharing with others. If you're a contributor to the community, somebody already using your Docker container as a base image in their environment, you don't know because you don't know what's happening because somebody might be using it. This is an example for IAC. Same thing for Kubernetes. Same thing for CloudFormation template or Azure Resource Manager. There's an example of the automations that people are using it a lot. And when we talk about all the students, we have to integrate our environments with the repository management because everything is a software. We have to put in our source control with our IACs, with our integration, we can build up automation. That's what basic DevSecOps is about and integrating all the send wire elements. I'm going to go quickly. When we combine the verification validation with the DevSecOps, verification validation is really based on all activities we do. And then using the engines behind it, which is automation engines using the DevSecOps, using continuous integration, continuous delivery to help you to build up the verification validation analysis of your code that you're writing about. So we're going to dive where we can do met those activities shortly, in which phases I have to use those activities, which one is verification, which one is validation. I will show you quickly with respect to the lifecycle and also the engineering perspective. So there's a problem here. Anybody knows what's problem? Especially here. Can somebody guess me with a short break? All right. You're open source community. You're in the Linux Foundation open source summit. I'm sure you heard many, many security, many components. Let's take a look. Here's the problem. We are building a components, but how do we know that? Our components we're using because open source is the one of the key drivers for DevSecOps environment. What you see here, almost every tools for continuous integration, code review, testing, building, all or open source, like GitLab. I know you have the paid version of the GitLab, GitLab or GitHub actions. GitLab run as an example for the CI, or Docker open source for deployments, or using any parameters for a monitoring open source, or I can name it 500 tools, but it's all our open source. Tecton. Anybody's in Tecton? Tecton is another great ones for a continuous deployment, which is making it much easier to deploy. There are many tools we are using it here to build our infrastructure. So now there is a connection with open source. Yes, we can do a lot of verification and validation, but we are using a lot of open source to build up the infrastructure, writing a code in it that we are driving. So now the problem is, how do I know all this component that we are writing and using infrastructures actually build it securely? If you don't build up the security infrastructures at the beginning, you cannot do all the verification validation successfully, because attackers will use your Docker container, put the malicious code into it, your engineer, you're going to use his Docker container, you trust on him, but actually it's his container modified from a third party person, then we are putting it in our cart, and we are trying to be as safe. Is it a problem? Now, how can we do that? So we have to build up a security infrastructures, including software build of materials, look at the configuration, build up the components, we can build up a security infrastructure, then we can build up verification validation on top of it. Are we good so far? All right. Okay, let's continue. So we have to really build up a certified ECU in a DevSecOps pipeline. What that means? We're going to create a pipeline, each of the ECU units that we are building, again, each of the ECU is the specific functionalities of modules. It can be OEM, it may be a contributor, and it may be one of the environment that you might be using. So we can build up infrastructures for each of the ECU and we can certify architecturally, which is verification elevation, and also we can look at our infrastructures, eventually, make sure that it's correct. So another representation of infrastructure, these are more example for the open source, like study analysis on our cube, container scanning BME Aqua, maybe other type of scanning tools. You might be using malware scanning here, there's a lot of open source version available. You may be dependency analysis, like the dependency tracker, or the SC analysis of the GitLab, you might be using a other approval, which gives you an ability using all open source, but we can make sure that our infrastructures configure securely. We can put a lot of audit generation from pipeline. That goes back to my questions. If I'm a developer, how can I generate the validation of my result of testing the pipeline build automatically for you? As if you're auditors, don't do the check. And if your developers say, here's my result, go figure out for a DevSecOps environment, you will see the test results. Let's look at maybe this one is all together. So I'm going to give this one as another note for all of you. And when you look at the typical DevSecOps environment, open source libraries are everywhere in our software lifecycle. Open source means that we are acquiring the component from third party. We are creating the code in our base container images, and we are releasing the code based on the Docker container or Kubernetes. We might be using Kubernetes cluster somewhere else. We have to monitor third-party libraries. It's basically within the lifecycle and also part of the product. Since the software ecosystem is growing in automotive industry, we are using it to save time. We don't want to reinvent the wheel. There's a lot of standardization or modules happening a lot in the software world for specific automotive. That's the reason you are here. You're going to use some of the analysis that you presented maybe as the image analysis. We use image analysis, the third-party libraries in our code. In short time, it's available because you don't really write the code as an engineer. You just google it, go the stack overflow and copy it and use it, saving time. Great. But how do we measure all this dependence in our lifecycle? It's becoming part of our lifecycle. Where we build the pipeline also we use in the product perspective. Let's take a look how we can giant list of VNV activities throughout the lifecycle time check. We have a couple more minutes. When we look at the verification validation, I'm just going to pause a couple seconds. All DevOps, DevSecOps, it is not changing how are we building the systems. It's not changing. It doesn't say forget architecture, forget the design. Just go write the code. No. We have to have a feature. We have to think about the requirement here to understand architectures. If you don't know what the user is asking from you, how we can architect your systems? If you don't know what the future are working on, how are we going to build up your components eventually? Don't confuse the phases of system development lifecycle with the process as a waterfall and agile. The phases of lifecycle is true for years and years, but we build up the wrong processes to achieve those requirements. That's a process. The phase is the same, either using agile or a scrung as the one of the techniques or the Kanban or the save. Still, you are doing these phases in a feature-based, important features about the functionalities, features about the specific functionality you're working on it. That's why I said Tesla is an agile company. A lot of automotive industry are using the very agile centric DevOps centric, but it's focusing on the future. They're not dropping any of these phases. It's important. That's the reason if you try to drop any of these phases, you cannot do verification and validation activities correctly. You're going to do some assumptions. With assumptions, we're going to build up unsafe cars. You will see a lot of things. I'm hoping if not, but based on the trend I have been seeing it, we miss a lot of fatal accidents eventually because we don't do proper verification validation. I hope we will not, but if you don't do it, you might be seeing it. Let's take a look. Quick things. I'm just going to run through with the lifecycle perspective where we have to the verification, where we go to the validation, how the interaction looks like. I'm just going to list the activities that we think about the verification, about understanding are we building the right product. Usually, you're getting requirements and your feature requests. You can all this analysis that I listed. You can the trace it, but now you can all analysis pieces when you have your requirements as available. Because if you do analysis at the beginning as a plan, you can do all this activities eventually. If you don't plan your analysis, you cannot do the rest of the work. So you're going to plan analysis and upfront in the future request. When you get in the requirements, you can the further creating analysis of your work based on the test, based on acceptance test, based on secret analysis. So basically you are doing a lot of preparations as you are building your functionalities requirements. You can put the verification activities in your specific phases in your lifecycle. If I'm writing any cases, like using a JIRA or GitLab, issue tracking systems or anything, so GitHub, then I can think about what type of analysis I'm going to do. Let me create a potential test cases that I have to do. Test case is our driver for the validations. That's the connection pieces. When we get into the design and architecture, we can do all this test planning, test design during our architecture and design we can do. If you don't plan your test cases when you are architecting system, you're going to pay a lot of money to build up your test environments. It's going to delay your deployments. Eventually, your manager will say, we are running out the market. Let's do something quick. That's happening a lot. You're going to do a lot of shortcut. And eventually we're going to fail miserably because most of the testing, if you don't plan accordingly, you can fail on your tests because it's going to cost more money for you. Just going to say one word for agile practice. If you are following scrum or agile or if you're creating a sprint or a case that you're writing the best criteria for your case that you're writing as advocates and stories, you have to follow the criteria called as invest. Anybody have an invest criteria when you're writing the case? Yep. So invest one of the letters. T is a testable. If you're writing a story in your requirements as an engineer, you have to think about how can I test those requirements. So basically we are writing testing right away in our analysis and design. Let's continue. When we get into the development phase, now we see the validations because when we basically we can shift left from validation to product, put into our life cycle so we can do early validation for the specific components. Remember one is building the right. The other one is did we build the right? So we can look at the static analysis result on the development phases when we're writing the code. We can look at the vulnerability. Let us get some validation of vulnerability that has been addressed correctly when we're writing. Why we are trying to get validation early? If something is not right, we can go back and change easily because we are so close to the architecture and design, so close to requirements. We can go back, change easily and look at what we are doing right. If you separate those activities, you're going to pay a lot of money down the road, go back and change it. So basically we can do a lot of work in here for validation. We can create a static analysis result. We can do functional. We can IAC build up a lot of test cases, test it and test it and more. Eventually when you do all this activities, you will be creating a digital replica of your car eventually. That's what Tesla has. Tesla has a unique digital replica of their car in their car and everybody has a digital presentation or using in the cloud and wireman and you can see that. So let's continue. When we do the integration still because of the verification validation of certain analysis and risk flow, we can measure it. When we get into the final phase of delivery, we can do more acceptance testing. Now we are getting more validation based on what the verification requirements looks like. So it's all integrated. Let's make a quick deploy. I'm just going to quickly skip a little bit of time and for data, it's about the pieces we can collect data in everywhere because data is our result that we are generating for the validation perspective. Every actions you do in your life cycle generating data, either test result, test cases and specific attack vectors, specific vulnerabilities, you will be generating all this data so eventually it can feed all to your requirements and verify. So overall, we have the continuous verification validation in every phases of life cycle. How are we going to do? We're going to use DevOps and DevSecOps based on our open source tools and techniques we're using it. So basically it has to be our life cycle. Everybody has to involve. All stakeholders has to be involved. Business, security, quality, tester, name it, just a quick replica. Then we can think about verification validation of life cycle. Eventually we can drive in a safe and secure car. That's it. Here's our workflow and which is another way of seeing our verification validation workflow. This is you as a customer that's your expectation is being a safe and secure car. But this part is done by the manufacturer which is a car manufacturer. We can set the specifications, look at the how are we doing well and we can build a product. We can verify and validate. If your customer, which is us, if somebody's kind of got four of its, if something happening when we die, think about that company that's going to buy that car and then car is going to get pulled over. There's a lot of recalls for a safety and it's going to cost so much money for organization. Then it's very hard to change the brand of the company. It's going to cost more money. That's it. I'm going to say here's my information. You can download it from the website. As an SCI, we have been publishing many, many other articles. I know we have one minute. I made it. Here's my contact information. If anybody has any questions, we can talk about this. You can take and send me email and I'll send the slides copy as well. I hope that we can be in a safe and secure car eventually. Now you have a job. Your job, think about security as you're building, as you're writing the code because it's affecting our life in some of us which we don't know. Thank you for attending my talk. Any questions? Yes, a couple questions. Yes, please. Hello. Thank you for the talk. I am actually new to Automotive Grade Linux, but I've seen the pain and the friction while working in the embedded industry quite a bit in terms of certifications. And quite painful for a good reason. Everyone wants to be safe. And my question is, are there any open source standards or methodologies to testing that are out there we can read about? There are. And something we can widely follow in other industries as well because Automotive Grade Linux is the standard in terms of testing safety. Great. Perfect. So there is actually, that's one thing maybe I can think about here as a continuous verification validation means the industry are learning. And if you look at the Tesla example, Tesla does continuous verification validation, which is for certifications continuously. They're not getting certification every car, but they are doing incrementally. What that really means incrementally, if I'm building any capabilities, any functionalities, and I have to show a delta of functionalities and also the testing, instead of sharing the whole giant set of requirements, I have to share the incremental version of it. The thing about here is an example, and I'm talking about maybe this one. If I am doing some testing here, the next time, next iteration will be different. That means I have to change only the incremental pieces for a compliance perspective. We can get the certification quickly. Yes, organization does. There's a compliance as a code for open source version. Send me a lot, send it here. It's information that's available at this of compliance as a code. You can put your compliances, which is a certification into your requirements. When you look at the certification, it's about the verification validation analysis. That's exactly what they're asking, but we are getting a customized version for a specific automotive industry or healthcare. All are coming from my trickle source. So short story, we have to determine incremental, and also there is a cultural pieces as well. If you build up the culture with your certifiers, and then they will trust on you based on the incremental changes. That's another point that we can think about. Any other questions? Yes. So functional safety with optimized boot time is like two contradicting requirements. So do you have any suggestions on how we implement functional safety also having some boot time optimized? We always get requirements that functional safety test needs to be done at boot time, but then it increases your boot time. Any suggestions? Regardless, safety is always the priorities. That's kind of like we cannot trade off the safety. And better way to explain to the people about what the consequences are, but based on my suggestions based on what I've been saying it, there are some risks we can do. If you'll mind the risk analysis, I'm going to go back here quickly for a risk perspective, and I know I'm just browsing the screen, based on what things we can tolerate it. So I really suggest for a safety cannot be a risk situation. Separate those, and then you will get a balance of which one is required based on the risk. Risk can be hazardous, but how much is affecting our human life? If it's affecting human life, we cannot make an in-circification. We can't. We have to follow. I hope that answers your question. All right. Anybody have any questions? Yeah. So actually, I'm from automotive industry, and I have a question that, you know, in automotive, there are a lot of regulations, specifications, ISO 26262 and ASPAS, and all this kind of homologation, those kind of a lot of regulations. But on the other hand, I think there is a more fierce computation in the speed, or say the delivery speed. So how do you think, for the automotive industry, can balance the speed and this kind of? Actually, the automotive industry are taking a lot of advantage of the speed. They're not contradicting. The problem is, when we try to apply those certifications, we are applying as a waterfall tank. We are trying to get more papers. We are trying to get more manual process. That's actually contradicting the speed. If you're able to get those certification requirements as part of our life, that's what the verification analysis is, we can get much quicker. But usually, certification or certifier are outside our cadence. It's kind of a ping-pong ball. We're just saying, here's my product. Go certify. Get back to me. Because it's not working like that. We have to work as a team, working together. If as an engineer, if you know what your requirements look like, you will do it. If somebody tells you, two months later, you have done wrong, it's going to get close to you. What you're going to do is a shortcut. That's the problem. So if you look at the software, they find the vehicle, look at all the standards from Deloitte, to many other things. Renault has a great standard. Porsche has another one. We are getting more automation in there in a while. We can use a lot. So it's not actually contradicting. It's really helping a lot. One of the principle DevOps giving traceability, visibility, makes it much easier for it to certify it except that's what the Tesla does. I think we are done. We can talk outside so I'm available. We can talk more. Thank you.