 Good afternoon. Welcome to this talk, SC Linux Mandate Reactions Control and Cockpit. I am T.V.A. Prasad. I work with KPID Technologies as a solution architect. I have 18 years of experience with 10 plus years of experience in the automotive industry. I have been part of production programs based on Linux and Android. I follow the AGL and the Alisa mailing mix very, very closely to understand the latest developments in the automotive industry. I play around with my Odroid C2 and my Dragonboard 4DNC in my free time. That's about myself and a few words about KPID. We are experts in automotive. We have 7000 plus employees, 51 patents, 25 innovation awards, millions of cars running our software. That speaks volumes about ourselves. Now let's look at the motivation for this talk. When we actually started working on our cockpit, we wanted our cockpit to actually support a Mandate Reactions Control or MAT. And we have selected SC Linux as the Mandate Reactions Control for our cockpit. So we have implemented SC Linux for our cockpit. We want to share the knowledge that we have gained as part of that implementation with the community. We also want to understand from the community if there are better ways to implement whatever we have implemented. We also want to understand from the community and Google if the solution, whatever we have implemented can be upstreamed. Or parts of it can be segregated and used in other automotive production programs. So this is definitely not a product pitch. Let's look at the agenda for today. As I mentioned earlier, we have selected SC Linux as the Mag for our cockpit. So the first item that we are going to look at is the SC Linux labeling part in the cockpit. Then we will look at the policy that's implemented after the labeling is done. Then we will look at the checklist against which we review the policy that's been implemented. Then we will look at the questions. Let's try to understand the reference architecture for our container based cockpit before we proceed further on the labeling part of it. As you see in this diagram, we have having a multi-core SOC on which we have a container based architecture for our cockpit. As you see, the IVI is running Android and then we have another container for the cluster and the HUD. We have another container called SysMan which is majorly responsible for handling the audio display and the early functions corresponding to RBC and etc. So these are the major containers. We also have an address container which is optional and the root namespace or the host namespace that we have in this approach is Android. Android is actually the root namespace on which the containers are actually running. We also have an ACLB controller which is providing us the CAN interface to the multi-core SOC to receive the vehicle data. So this is on a high level the architecture for our cockpit and I want to read it again. Android is the root namespace for our cockpit and we have the SysMan cluster and address as the containers. Let's look at the primary goal that we started off with. Protect any root processor from getting access to the vehicle and controlling it. As you see, as you see the architecture, we have container-based architecture and irrespective of whatever isolation techniques we have, we want to definitely protect any process from breaking out from the isolation techniques that we have like breaking out of the containers or breaking out of the process boundaries and getting into the root namespace or trying to play around with the vehicle. So even if it actually breaks out of the process or the container boundaries, the damage that it is going to do, we will want to limit it. Any process which is breaking out of it, its barriers or boundaries, the damage that it's going to do is going to be limited. That's the primary goal that we started off with. Now let's look at some high-level requirements that we formulated from the primary goal. So the first requirement is files and process in the container shall have an AC Linux label. So every process and every file in the container or the major root file system will have an AC Linux label and a policy around that label will be done, will be created. That's the requirement. Next requirement is processes accessing device nodes or interacting with the root namespace shall have a unique label. So this requirement is like this. All the processes in the container do not really get a unique label. The majority of the processes share a single label, processes only which require access to a device node or which are interacting or communicating with the root namespace will only have a unique label. Again, this is done to have a cap on the size of the policy. If we start giving a unique label to each and every process running in the container and the interactions that they are going to have, the policy, the size of the policy is going to get loaded and definitely there is going to be some impact on the performance. That's the reason processes accessing device nodes or processes interacting with the root namespace shall only have a unique label. To explain this further, let's say as I showed you in the previous slide where in the reference architecture, the sysman container has something called as an audio master or a display container, display master. So display master is actually responsible for handling the DRM part of it. So the display master is the only process which is likely to access the display part of it. No other process is allowed to access the display device nodes. So that's the reason we will want to label the process display master and we will also want to give the policy access to the display device nodes only to the display master. The next requirement is load the policy from the root namespace only. So we will want to definitely load the policy from the root namespace only. We do not want to load the policy from any of the containers or anywhere else. The policy has to set in the root namespace and it's just loaded from there. Then the next requirement is device node creation shall be done from the root namespace only. We do not want the device node creation to be scattered across multiple places and then have some run into some issues. We want the device nodes to be created from the root namespace. The device nodes will be labeled from the root namespace and then by mounted into the respective containers. That way the access to the device nodes is controlled. The next requirement is domain transition to the container domain shall be restricted. So processes that are spawned in the container run with a unique domain label. And that transition into that container domain is restricted. No process can just directly go into the container domain. There is a restriction in the policy that brings about transition, restriction in the transition transitions into the container domain. We also want to restrict execution from time buffers in containers. The next requirement is we want to use the character device interface for, we want to use the character device interface for GPIOs. We do not want to allow surface for GPIO access in the containers. And then last but not the least, the major requirement that we have is the size and performance of cost of the policy should be within the permissible limits. The size of the policy cannot go on increasing, it cannot go on increasing beyond a certain limit and that cannot go on impacting the performance of the system. Now these are the high level requirements that we started off when we started implementing. Now let's look at the approaches that we have evaluated. The first approach as you see here is name space support for SELinux FS. Let's look at the approaching detail in the next slide. So as you see here in this slide, we have each of the containers having SELinux, having a separate namespace for SELinux FS. So we have some out of three patches which allow SELinux FS to be, SELinux FS to be in a separate namespace. Now you have a separate namespace support for SELinux FS. That allows us to mount SELinux FS independently in each of the containers. And then we can have a separate policy for each of the containers including the root namespace. So SISMAN would have a separate policy that gets loaded from its SELinux FS. Cluster would have a separate SELinux policy that gets loaded into its SELinux namespace. So this is one of the approaches that we have. So let's look at the second approach. Use Android build generated SELinux and file context to develop policy for container in Yachto. So this is like this. Android as you see in this diagram, Android generates couple of integrated files, common intermediate language files on SELinux and these files can actually be moved to Yachto and you can actually generate and we wanted to write the policy for the containers in SELinux. And that will be placed in the container root FS. The SELinux is placed in the container root FS. The file contexts that are generated are going to be used to label the container files in the container. And once you have the SELinux placed in the root FS of the container, as part of the Android build, as part of the Android boot process, we are going to modify, do modifications to the Android source code to actually pick up the SILs. The cluster policy, the container address policy and all these SILs from the container root FS and then Android in its system image has a SIL compiler. All these SILs are passed to the SIL compiler and the policy is generated at good time of the system. This is the second approach that we have evaluated. The third approach as you see is where we extend the root namespace or the IVI as it runs Android. We want to extend the IVI SELinux policy to the container. That's the third approach that we have evaluated. Now let's look at the advantages and disadvantages of each of them. As you see, the namespace thing looks very good and promising, but where it allows you to have a separate policy for each container, that's good. But the disadvantage, the major disadvantage of it as we see it is it's not upstream. That is one of the major disadvantages. Again, at the same time, though you have a unique policy for each of the containers, it becomes difficult to actually define a unified or an integrated behavior for the whole cockpit. That's the biggest disadvantage that we see for this approach. The second approach, though it allows us to use the container as independent. The container policy is independent and reusable. It's like tomorrow when I want to use the container, I can actually take this container and use a different project where the container is already having the label supplied and everything looks good. But to the downside of it, it requires modifications to the AOSP source code to pick up the CIL from the container mount points. That's number one. The second one is again, the policies compile every time the system boots up. That's the second major disadvantage because that's going to have some impact on the boot up time that we have. The third is that the policy is not at a unified place. It's scattered across various places. You have some part of the policy in the Android world. You have some part of the policy in the Upto side. You have some policy. It's scattered. That's the major disadvantage. The next disadvantage is that the paths in the file context are going to be related to the container. It's not going to be related to the host. For me, it becomes difficult to actually label certain things with respect to the host. That is one of the disadvantages that we look at. Now let's look at the third approach. The advantage of the third approach that we have zeroed in on. It's easy to develop and the policy and file context are not scattered. These are the two major advantages that you have here. Again, on the downside of it, you will need some offline effort that's needed to label the files in the containers root interface. And again, the containers policy and the containers file context are tied to the host. You cannot have the container image generated with the SLM level and you can just straight away use it in some other project. That is one of the major disadvantages. But considering the ease and considering the way the advantage that the third approach has, we have selected the third approach for our implementation. Looking at the third approach in detail, as you see this figure, we have the T files for each of the containers. Each of the containers will have its own T files for the policy that we want to develop for the containers. And all these T files along with the system, SC policy, T files and the SC policy, the T files in the vendor specific T files, etc. They are fed to the SLM compiler along with the security classes, initial sets. The GenFS context and the port context, all this is actually fed to the SLM compiler. And finally, we have the SC policy that gets generated. Similarly, we have the file context that are scattered across the system SC policy and the vendor part of it and the board part of it. All this is fed to the SLM compiler and it is going to generate a unified file context.bin. This is in detail explanation of the approach number three. Now, let's look at how we are actually going to label the files in the container. So, as I mentioned earlier, we are using YAPTO to churn out the container root FS images for us. So, YAPTO is going to churn out the container root FS. And Android build as you see here is going to churn out the system.image and the vendor images. So, the container root FS that comes out of YAPTO and the file context.bin as we saw earlier that comes out of the Android build along with the container moment point is actually fed to a utility called as mkuser.image.mk2fs. This is a utility that is a byproduct of the Android build. It's a host utility that the Android builds as part of its build system. So, all these inputs are presented to this utility to generate a container root FS with SLM. So, now we have a container root FS with SLM and let me make a point here. Though this is actually going to label the files in the container, for having the correct set of permissions for the files in the container, a config.fs for each of the mountain points of the container has to be defined as part of the Android build. Please do note that you will have to define a config.fs for the container mountain points and the permissions that you would want for the container mountain points. Now, we have a container root FS which has got SLM label supplied to it. Now, let's look at the policy around how we went ahead and implemented the policy. So, basically what we have started with is once the init starts or spawns any of the containers in it, we transit from the process, the init of the container gets assigned the label as container domain. And subsequently, all the processes that are spawned by the containers in it have the container domain as their label. And here we want to make a recommendation that use setxcon if the containers in it is a symbolic link that makes it easier to implement. Instead of labeling the original and the simulink file, it's easier to use the setxcon if the init of the container is a symbolic link. As you see here, we have defined an attribute called container file type. At the same time, we have defined another attribute called container domain. So, all the files in the container are labeled as container file type and all the processes running out of the container will be having the attribute container domain set to them. So, now we have tried to actually develop a policy around both the types, container file type and container domain for our implementation. Then what we have done is we have, as mentioned in the requirements, we wanted to restrict the domain transition to the container domain. So, as you see here, we have allowed container domain to execute types of container file type, open and read files of the container file type. And as you see here, except for init and container domain, no other domain is allowed to transit into the container domain. So, that way, the only way that any process can actually transit into container domain is through init or container domain. Because container domain has to spawn execute processes in its context. So, that's the reason we allow both init and container domain to transit into container domain. No other domain is allowed to transit into container domain. This is the primary policy that we have come up for container file type and the container domain. Now, let's look at one of the major problems that we had to solve. As I mentioned in the requirements, we have a requirement where we wanted the device nodes to be created only in the root name space and labeled in the root name space. And all the device nodes that are needed by the container are actually bind mounted into the containers when the containers are being launched. But as I said, we are using Android as our root name space. And UAMD is the domain, is the demon that's responsible for creating and labeling device nodes in Android. And the second stage of Android, second stage of Android, spawns UAMD and waits for UAMD to actually create and finish creating and labeling the device nodes before it proceeds further. But as you see here in this figure, the containers are launched around 1.4 seconds right from the boot up time. And after which, UAMD starts at around 1.97 seconds from the start. And then you can see here, the dev node creation is done at 2.23 seconds. And you can see that the cold boot, which is like the type taken to create and label the device nodes took around 250 milliseconds. So as you see this diagram, it takes around 800 milliseconds after the containers are launched for UAMD to actually create and label the device nodes. But as we have some stringent KPIs to meet for the early audio, the early RBC and then a couple of other cluster key performance indicators to meet. This is something that we cannot do if the device nodes are created so late at 2.2 seconds right from the start. We have to have some solution to this problem. So the way we have implemented a solution to this problem is we have actually created or the device nodes needed by the containers are created in the first stage unit. And we have labeled the device nodes in the container launcher as actually much is already set up by the time. And once the device nodes are labeled, the device nodes are mounted into the respective containers and then the containers are actually spawned. We cannot really wait for the containers, the device nodes to be created by UAMD. We have also evaluated the solution where we wait for UAMD to create the device nodes, notify us that the device nodes are completed, creation and the labeling of device nodes is created. And then we actually bind on the device nodes into the containers and the containers are spawned. But again, as I said, that's not really meeting the start of KPIs for us. So we want to understand from Google and the community if this can be actually upstream and if we can actually take some part of this implementation for upstreaming. Now let's look at how we have come up with the policy for binder, what's in store for us with binder and SC policy. So as we have multiple containers and we need to have some intercontainer communication, we definitely evaluated different types of IPC mechanisms or RPC mechanisms for the container communication including Debus. But as binder is upstream, we have chosen binder for intercontainer communication. But as the containers are actually running in the vendor domain, the root namespace, which is like the IVI or the root namespace is Android. It cannot, the Android policy, especially after that came into existence after treble, does not allow IVI, which is running in the core domain to interact with a vendor domain using binder. We have the policy only allows vendor binder or hardware binder to be used between a core domain and a vendor domain. So as you see here, between sysman and IVI, we use vendor binder and hardware binder. And for the other containers, which are not really interacting with the root namespace, we use vendor binder or sockets for IPC. We also have our own custom binder implementation to meet certain requirements from the customers and vendors. So we are now actually working on a policy for the custom binder and the intercontainer communication that uses this container, the custom binder implementation. That's about binder and SC policy. Let's look at the vehicle data access or the vehicle data network access. So for the security of any of the systems, especially with respect to infodainment or vehicle, the major part is that you need to secure access to your vehicle network. So the network, the vehicle, the access to the vehicle network has to be really, really protected. So as you see in this diagram, the protection that we bring in is where we have sysman running in a separate network namespace. And the CAN0 interface, which is a network interface, is actually attached to the sysman container. So now the CAN interface is only visible to the sysman container. No other container will have access to the CAN0 interface or the vehicle's CAN bus. So as you see here, cluster runs in its own network namespace. IVI has its own network namespace with device nodes like WLAN0 or Ethernet or something like that. But for the CAN interface is only attached to sysman and no other container has access to the CAN interface. Now once we have attached CAN interface to sysman, we use CAN sockets to actually read the data from the vehicle's CAN bus. And we actually have a label for the processes accessing the CAN sockets. And then there is a policy around these processes which can access CAN sockets. No other processes allowed to transition into the demons accessing the CAN sockets. That's about how we are actually protecting the vehicle network. We have two layers of isolation. One is the network namespace which is providing isolation. And then next you have the higher-level mandatory access control as seen in this, providing us another layer of security to the vehicle network. Next major challenge that we had to face is the file execution policy. As you see here on the left side, the upstream Google's travel policy does not allow, except for a few handful of domains to execute files. As you see here, AppDomain, IORA, PreFHD, and then Xivote, WebView Xivote. These are the only few domains which get permission to execute certain files. But as we explained earlier, the container is actually running in the container domain, and the container domain would want to definitely execute files in its namespace. So definitely we will need container domain to have the permission to execute files. So we had to modify the upstream policy to allow container domain, as shown here, to allow container domain to execute files. Again, we will want to understand if this can be upstream, or if there is a better way to actually come up with some policy which can be upstream. The next challenge that we have is with respect to SysFy IPC. As you see here, the upstream policy does not allow any domain to have access to SysFy IPC. But again, as the containers that we have are based on just plain Linux containers, we will need SysFy IPC in the process communication. So for that reason, we have to allow container domain to be able to use the Salesforce message and the message queues. So that's the reason we have modified the upstream policy from Android to allow container domain to access Salesforce messages and message queues. SysFy IPC, we have allowed container domain to access SysFy IPC. Again, we want to understand from Google if this can be upstreamed. Let's look at some of the challenges that we have. The first and major challenge that we have is container manager. So as I mentioned, we have multiple containers and there has to be a container manager which is going to look after the health of each of the containers and respond the containers in case of any eventuality. The container manager has to be on top of all the containers and then take care of responding them in case of any eventuality. But how do we label the container manager as? Is it going to be in it or is it going to be a separate domain? If it is a separate domain, how does it transit to in it once the container manager actually gives control to in it of Android? So these are some of the challenges that we have. And then once we label the container manager as in it, the problem is there are certain problems with Android certification as Android mandates that there has to be only one process with the label in it. You cannot have multiple processes with the label in it. So this is one of the major challenges that we are stuck now with. We are trying to see how we can address this problem if the community has some solutions, please write to us. The next major challenge that we have is with respect to logger. As you have seen earlier, the container is a pretty complex system. The e-coppit is a pretty complex system with multiple containers. You have logs generated from the IVI, from the cluster, from SysMan, from RAS, from across all the copies logs are generated. So it's a challenge to actually define a policy for the process creating the log bundle and for the policy to actually give access to the storage there to write the log bundle. Because let's say the P store, the Android side logs like the P store, the ANR traces, the tombstones, the dumpstates, and then you have different sets of logs in your cluster and this one. So it becomes a difficult problem to actually define a policy which can be unified for an integrated logger for the whole system. So this is one more challenge that we are trying to address. Now let's look at the work in progress that we are trying to do now. We are trying to block execution from 10% containers. That's something that's a work in progress right now. Again, as I said, the only way that any process can transit into a container domain is through init and container domain. But again, any process running in the container domain, if it writes to a file in 10% execute set, it gets the container domain and then it actually tries to use a container domain. Just avoid that problem. We are not allowing execution from container from 10% containers. But 10% will have to have the read write permissions but execution from 10% is going to be blocked from containers. Again, access to proc in the containers is something that we want to block using the policy. After that, we also want to use assay lint. Again, this is something that we have not started but we want to start using assay lint to evaluate the policy that we have developed. Last but not least, we also want to actually create tools to measure the impact on performance. Because a lot of our customers are actually, our vendors or peer ones are actually asking us the details about the impact on performance. So we definitely want to create some tools to measure the impact on performance with the policy that we have created. We definitely are looking at using ebf to actually create some tools like that. That's about the working progress. With that, we are now moved to the policy checklist we have done. We have completed the policy implementation part of it. Let's try to look at some of the checklist items that we use to review the policy that's implemented. The first point in the checklist is they were on any process with the label of init. As I said, there has to be just one process with the label of init and then that's it. Any process that's found out of init has to run in its own domain. It cannot run in the domain of init. Though the newer versions of Android actually enforce strictly the versions at least still are not enforced strictly. So the policy has to take care of it. The next checklist item that we have is never grant DAC override permission. No domain whatsoever will have DAC override. What needs to be done is the file permissions or the process permissions have to be adjusted or we need to check the file permissions and the process permissions have to be checked but DAC override cannot be given to any process. That's checklist thing that we definitely religiously follow. The next item that we have is we check if the regular expression rules are carefully written because if the regular expressions are not carefully written the size of the policy gets bloated or there is definitely going to be an increase in the verification time. So regular expressions whenever they are used that part of the policy is reviewed properly to check if excessive permissive access is granted or another policy is getting bloated. The next checklist item that we have is if the proper binder macros are used the binder used, vendor binder used or the hardware binder whether the proper binder item macros are used. This also checks the custom binder macros that we have for our custom binder dev node. So all that is clearly reviewed with respect to binder access. So that we are not giving binder access to some process which does not really need it. Then again the next item that we have is the transition to SEO or shell domains is restricted in production software. We want to make sure that shell and transition to shell or SEO domain is restricted. Next ADB to the super user domain transition in production software is restricted. This is one more thing that we ensure. One of the other major important review item that we have is you might want some of these transitions to shell or SEO domains as part of debugging. So all those transitions that are needed for debugging are encapsulated in user debug or engineering. They have to be encapsulated in this because if they are not encapsulated and if the developer forgets to remove them in the production build it leaves a security hole. So as part of a review process we definitely ensure that any of such transitions which are put in place for accessing the shell or SEO domains is actually encapsulated in the user debug or engineering background. This has to be religiously followed otherwise that leaves a security hole in the system. The next checklist item that we have is any container or any process that needs access to GPIO has to use the character device interface. It cannot use the CSFS. That's a requirement that I mentioned earlier also. That's a requirement that we religiously follow. If we have multiple GPIOs scattered across multiple ports the way this can be done is there is an upstream implementation called as GPIO aggregator where you can use the GPIO aggregator to actually take the pins from port A and port B whatever they are the ports that you want to access and you have multiple pins scattered from port A and port B. You can take the pins from the port A and port B and provide it to the GPIO aggregator which creates a virtual port and that character device node is something that you can use. The advantage that you get with using character device node is that you can let's say your container needs access to certain GPIOs. The character device node especially that's created out of the GPIO aggregator can be attached to the container and then you're done. That part of the GPIO pins is not visible to anybody else and then once you have labeled the virtual character device that's generated out of the GPIO aggregator you can have a label assigned to it and then a policy can be actually developed against that or the label that you have assigned to the virtual character device that's created out of the GPIO aggregator. That's the advantage that you get with using character device for GPIOs. Last checklist item that we use is create specific type for the object or objects. Again, to explain this in detail, let's say I have multiple camera devices. I have multiple camera devices and now I want to actually write a policy for the rear view camera process and time the rear view camera device node. It becomes easier for me to develop the policy if I label the camera device using the... If I label the rear view camera device as REC camera device and I label the REC process appropriately then it becomes easier for me to write a policy against that whereas if I label all the four camera devices same with the same label it becomes difficult for me to actually write a separate policy for each of them. Similarly, for an iPod which is plugged into the system I will want the iPod to be labeled as iPod device and it becomes easier for me to actually write a policy for the iPod device in specific instead of writing in generic policy for all the USB devices that are connected. That's the advantage that you get with defining a specific type for the device nodes or the files that are being accessed. With that, we end the guidelines part of it. Let's look at the conclusion. So as we have seen, we have added SC Linux mandatory access control support for our puppet and definitely we need the community to suggest if there are better solutions. We also need Google to actually help us understand how we can actually upstream these changes and what's the best way to upstream these changes. Again, we also want to understand from the open source community if certain parts of the policy can be extracted or segregated for use in any other Linux based automotive production program. We will definitely want to open source and upstream this and maintain this for any other production program that uses Linux and its Linux. With that, I would want to close my talk. I want to take questions. Thank you.