 Hello. Thank you for joining. So today we talk about the functional safety certification methodology for the Red Hat E-Vehicle OS. So let me share my screen. Okay. So again, thank you for joining. I'm Gabriele Palloni, a work for Red Hat. Introduction about myself. I am a senior principal software engineer working Red Hat. I am the technical leader for the functional safety in vehicle operating system that indeed is a Linux based operating system. I have read that quite recently in July of this year. Prior to that, I was working in Intel Mobileye for functional safety automotive and industrial solutions. I mean, in the last couple of years, I've been quite involved in safety critical Linux. I was still the governing board chairman of the ELISA Linux Foundation project, and I also chaired the safety architecture working group in ELISA. So, and in general, you have sort of mixed background between functional safety and kernel driver developments. So what I'm going to talk today. First, we will look at the complexity of the certification for Linux and how the cake is made. How what are the different layers that, you know, constitute the Red Hat vehicle OS. What are the pillars of our safety argumentation. Okay. Then we'll, I will explain the weaknesses and remediation evaluation technique. That is something new that we have developed internally in Red Hat. Then I will explain the India class functional safety qualification flow in the perspective of the current safety standards. Okay. And finally, I will leave time for question and answers. So the complexity of the certification. The Red Hat vehicle OS Linux distribution is complex, like every other Linux operating system in general. The, the, the point was, the difficulty was, effectively, how can we find a tailored approach that would make it feasible to certify Linux. Okay. And one of the challenge was also safety element out of context versus safety. In context versus safety element in context. So, as Red Hat, our aim is to deliver a safety element out of context that can be adopt, you know, across multiple OEMs on different hardware. Okay. And across different use cases. However, as you know, the, the safety element out of context. Usually are difficult to integrate because when the OEM starts to integrate it then there are always mismatches with respect to the assumed safety requirement and conditions of use. So, how do we accommodate a safety element out of context into a specific context. So what we do in Red Hat, we collaborate directly with our partners. Our vendors on OEM, so that we make sure that when we define the assumed safety requirements and conditional use, these are fully compatible with their context of, of integration. Okay. So, and effectively, this is how we, you know, how we make sure that our distribution is working effectively for our customers. Also, we have the problem of the continuous development. As you know, Linux is continuously evolving. It is pretty common to have new features, bug fixes, and in general, over the year updates for Linux systems that are deployed on the field, and actually complex autonomous driving systems that are Linux based are no different than other Linux systems in this regard. Therefore, we need to be able to find a continuous development model that is compatible with the certification. Okay. And in this regard, we have developed internally the concept of continuous certification. Okay. I will talk about this later. So how is the Red Hat in Vehicle OS made? We have multiple layers. Okay. So the Red Hat in Vehicle OS is the result of multiple layers where each layer depends on the previous one. Each layer contributes with additional testing, additional review, additional development process practices that harden, you know, the previous layer effectively. So we start from the far left where we have the open source projects. Here we have millions of open source projects that are Linux based. Okay. And what we do initially, so we repeat like thousands, sorry, 10,000 of these open source projects and we build the Fedora distribution. And doing so, there is an initial, you know, set of testings, reviews, and the regressions that are run. Okay. And then, you know, the Fedora distribution that is freely available for desktop systems is released. Following the Fedora distribution, we have Fedora ELN. Fedora ELN is the build of Fedora, just stereotyping like the 4,000 packages that are used to build the center stream distribution. Okay. So, at this stage, we take Fedora, we pick the 4,000 packages of the next layer and we do another build. Okay. So if there are no errors, then this is the baseline to build the center stream distribution. And the center stream distribution is yet another open source distribution. This time with the target of the enterprise segments. The center stream is a freely available enterprise distribution that our customers can use to collaborate to prototype and where they can, you know, push new features backfixes in order to support the enterprise segment. This is based on center stream internally, we have additional testing, additional reviews, additional coverage figures, and we build the Red Hat Enterprise Linux. And this is where we make the money. Red Hat Enterprise Linux is a proprietary distribution that is aimed to the enterprise market. Okay. And here, real is very, very robust. We already have customers working on mission critical environment, like financial transaction where having a bug, a loss of said this would result in a, in a significant loss of money. Okay. And here we have a support, we have a product support time of 10 years. Based on real, we do another selection. And from 4,000 to 400 packages to build the certified Red Hat vehicle OS and in doing so, we add additional mitigations additional testing. So whatever is required to meet the ECB systematic capability. So, right. So in general, here the principle is that we have for this layer, every layer is contributed is contributing to provide more robustness. Okay, with respect to the previous layer. So here, there are some statistics about rail. Okay, so rail is deployed on millions of systems. They are widely used. We have a portfolio of more than 50,000 customers. And we have more than 900 engineers working on rail, more than 400 quality engineers. And we have major release every three years, my release every six months and the bug fix roll up every six weeks and security updates are a synchro. So this is just saying that, you know, real, you know, it's, it's a very, you know, widely deployed baseline, robust baseline. So what is the scope of our certification. Okay, so indeed, as we say, the target is isn't big. And we have a core runtime. That is, Marcus is it be so the current time is the set of binary packages that will constitute the executable code on the car. And we have the tool and tool chain that are used to build this core runtime. This must be qualified up to be. Because the whole run time itself is easy. An example is for the GCC compiler. We have tools and tool chain that can be used by our customers to build their own functional safety application. Okay. So, and this must be qualified. As well. And finally, we have tools and tool chain that are, they don't need to be qualified. They, they, they can be used, you know, to for non functional safety application or maybe for debugging. So, and this do not fall into the functional safety scope. So, what are the four pillars of the safety argumentation. The foundation is solid. Okay. The foundation is a very widespread, robust operating system. And we also showed the layer approach, okay, with the different layer of layers that is hardening, you know, each layer with respect to the previous one. And one pillar is that the top level safety requirements are met. As discussed initially, we, we work with our customers with our partners in order to define the top level safety requirements of the vehicle was. And our vehicle was must be is validated according to this top level safety requirements. The third pillar is following the ISO 26262 development process. Now, in this, we cannot do that to the letter. It is well known that Linux is not capable of meeting, for instance, part six of the ISO 26262. And we do that by having defined our own theory approach in partnership with the exceed that. Okay, so that we can make sure that we follow the spirit of ISO 26262. Okay. And actually, in the next slides, I will present the details of our own theory approach. And the last is the continuous certification model. Okay, so already mentioned that for us it is very, very important to be able to deliver a continuous certification because we need to deliver continuous updates over the air. So, we, for us, it is critical to have this in place in order to win the, the automotive Linux market. Okay. Okay, so weaknesses and remediation evaluation technique. So this is a concept that we developed internally in Red Hat. And as you remember from the Red Hat in Vehicle OS pipeline, there are different layers that are involved in producing the final Red Hat in Vehicle OS. And this is, this is live. So on the top row show what are the layers. So we start, you know, as the other slide so we have the community projects with Fedora, Fedora ELN, center stream. We have rel Red Hat Enterprise Linux and then we have the Red Hat in Vehicle OS. You can see that each layer has got its own strengths. Okay. And, and the strengths are the development process practices that each layer contributes with in order to make the final ACB claim. Okay. And each layer also comes with weaknesses. Okay. And what we do in telling Red Hat, we leverage the strengths of each layer to build the ACB systematic capability claim. And wherever we find the weakness, we also put remediation in place so that we make sure that all the weaknesses are covered. And in the end, we are able to make the final ACB claim. Okay. To summarize, on the top part, you have the layers, then below the layers we have the strengths. And then for each layer, the weaknesses are identified and for each weakness we have the remediation. And all of these process results in the evidence is being produced. And these are the evidence that are also referenced by the Red Hat in Vehicle OS safety case. So, okay. So, anyway, if you have questions specifically about, you know, this slide, you can, you know, you can contact me offline or ask me later on. Now, I will present the Red Hat in Vehicle OS functional safety qualification flow, with respect to the current ISO 26262 safety standard. Okay. On the right, you can see the left side. And this is the standard V-Model life cycle that is used also in the ISO 26262 standard and I guess many of you are familiar with. The Red Hat, with respect to this life cycle, you know, the technical safety concept and validation tests are, you know, considered out of the scope of the in Vehicle OS itself because here, you know, we take the point of view of the integrator if you like. Okay. So, effectively, so we start from the safety and nominal requirements. Okay. And then we go down to the software architectural design. And the counterparts are the integration test and platform test. So for these phases of the life cycle, we use Part 6 to the letter. Okay, to qualify the in Vehicle OS. Part 8 comes down to the single software unit, so the unit design implementation unit test. We go and use either Part 8.12 or a tailored approach that is mixing Part 6 and Part 8.14. Which approach is used for the single units depend on the unit complexity. Okay, we'll, we'll explain this a bit better later on. Right. So, this is the top left part of the diagram. Okay, so we start from the in Vehicle OS assumed safety and functional requirements and assumption. These define the safety scope of the in Vehicle OS. Effectively, this define which packages are ACB versus which packages are through M. Okay. Now, once we have the defined the safety scope, we go and we define the software architectural design for the ACB packages. We provide both a static view of the of the interfaces between the ACB packages and also a dynamic view of the interaction between the ACB packages. Okay. Then, based on this software architectural design, we define the safety and functional requirements that are allocated to the single packages. And in order to confirm this safety requirement of the single packages we perform an FME analysis that is based on the architecture. And also we do an FFI analysis in order to confirm the safety scope. So, if you're at this stage, if you're able to meet part six, we move on with the single package qualification. Otherwise, if there are gaps, you can see that these gaps are filled. Till at the end, we can move on. And as I said, here we are following part six to the letter. An important point is that you can see that we have for the software architectural design. We are leveraging the evidence that we can retrieve from all of the layers that constitute the in-vehicle OS. So, and this is something that is true along with the functional safety qualification flow. So now we move to the qualification of the single package. Okay. So here, each package is treated as a software unit. If it is a simple software unit, then we can use part 8.12 to qualify it. And as we have defined internally, I tell you the approach that is mixed in part six and part 8.14. And when I say part 8.14, you don't need to think about 8.14 to the letter. It is a sort of a sort of a revision version of 8.14 that is based on reliability models. Okay. So the first thing is to evaluate the package. If it is a simple one, we will go on the left side of this diagram. So we first evaluated the development process quality standard, we then look at the requirements and the specification of this package, and we look at the requirement based testing that are available. Now, if we submit part 12, good, you are done. Otherwise, we need to find the gaps, and we look over till we converge to to to meet a dot 12. So, if the package instead is a complex package we use a mixed approach that is waiting in both part six and eight dot 14. Okay. So with respect to part six, we go and look at the evidence that are available to meet the software unit design and implementation to meet the software unit verification. Okay, and the software unit testing. And we retrieve an assessment score. Okay. On the other side, we retrieve the Fedora and the red heart enterprise Linux reliability data. The stimulus condition and the environmental conditions. And we do an assessment of the fellow rate against the expected reliability model. On the other side we assess the stimulus and environmental conditions versus the located the safety and functional requirements. Okay. And we do an assessment of the combined scoring between the part six evidences and the reliability data evidence. If the combined scoring results in an acceptable threshold. Then we can see good. The qualification is done. Otherwise, we need to fill the gaps. When, sorry, when filling the gaps, we can decide to either fill these gaps from a development process point of view, effectively working to meet closely part six, or we can define additional stress testing. Okay, so that we can build synthetic field data to meet, you know, the reliability model expectations. Okay. So in this case, we are filling the gap working on the on the right side of this configuration flow. At the end, when we have a combined scoring that is acceptable, we can see good. The package is qualified, we can move on. Okay. Now, once we have all the packages that are qualified, we need to move to the integration phase of these packages and for doing so, again, we refer to part six dot 10. So we identify, you know, for each dynamic use case. That is for issues case that is described in the architectural description that is done on the top left part of the diagram. Okay. So we identify the involved packages. We assess the available integration test. And we, we run the integration test. So if we meet part six dot 10, then good. Otherwise, we need to fill a gap. And so we continue to work till we have met part six dot 10. And once this is done, we can see that our vehicle was is it be qualified. So this is, you know, in a bit with a bit more detail, this is the description of our functions if the verification flow. Then continuous certification, as I said previously for us, it is really really important to develop a continuous certification model that would allow the evaluation of the change request to determine if the current certification that if the car and say to determine if the current certification is still by it, or if there is any gap that must be filled. Okay. So the idea in this graph is that whatever whenever there is a change request, the change request is assessed against each phase of our qualification flow. So first there is assessment of the requirements and architecture. There is an impact or requirement architecture. Yes, we need to fill the gap. Otherwise, we move on. There is an impact on the package qualification. Yes, we need to fill the gap. Otherwise, we move on. Otherwise, we assess the software integration is the software integration still valid. So if it is not valid, we need to fill the gap. Otherwise, we can move on. So the idea is that at the end of the day, if for every change, if you're, if you can say that there is no impact on the current certification, the current certification is still valid. Otherwise, we need to fill the gap, and we need to produce another certification. Okay. So, all of this process is indeed supported by automated tools and regression frameworks. Otherwise, it will be pretty much impossible, you know, to sustain this model. So this is the way, you know, read that we can effectively deliver this continuous certification. Okay. So, on my side, this is pretty much everything. So, before moving to question and answers, I want to thank you all for taking the time to listen to me. Yeah, now the pre-recorded session is completed. Please, fire off questions and I will try to answer. Thank you very much guys. And anyway, enjoy the rest of the open source summit.