 Hello, my name is Zahra. I work at Microsoft. My background is mostly on various aspects of systems, security, compartmentalization, and this talk is about one of the projects we are doing in Azure Sphere team on secure and dynamic hardware partitioning. Other genius architectures are becoming more and more popular as our applications are getting more complex, their designs, their dependencies, so our workloads are getting more complex. That's why we see, even on IoT and edge devices, we see this demand and support for adding multiple heterogeneous cores. For example, usually a combination of Cortex-A's, ARM Cortex-M's, and RISC-5 cores that are added for domain-specific scenarios. For example, for machine learning workloads of computer vision, use cases for balancing power consumptions and the application scenarios in a dynamic way, and specifically for also security use cases. We see support of separate processors, secure enclave-style processors for doing attestations, for doing systems measurement, monitoring verification, or supporting security debugging. It's an exciting area. We have multiple components, multiple event security-sensitive components, several trusted execution environments, several secure enclave processors, and we also have different systems stack, operating systems like Linux and hypervisors, and a form of like Arthas or bare metal containers on different cores. In our team, we are working on different aspects of systems designed for such heterogeneous architecture, it's like with our hyper partners. For example, here you can see one of our carburetors that we are working on, iMix 8 ULP that's from iMix 8 family. This specific one is like these details and internals are not like announced by NHP, so here I'm just going to talk about high-level architectures of it and things that are publicly available. As you can see here, we support for example two Cortex-A 835s, separate Cortex-M, each of them have like separate implementation of trust on trust on A, trust on M. We also have a separate core like secure enclave HLAC that's also doing other hardware-assisted security tasks such as attestation, systems measurements, monitoring, security bugging, things like that. So this is basically like a typical heterogeneous architecture that I want to discuss in this talk, that you can see there are lots of peripherals here, like from DMAs and GPUs and like crypto accelerators, CAM, and so these hardware should be like shared or depends on the use case, like from our customers depends on that, it should be like dedicated to each of these cores and be protected. To fully utilize the underlying hardware features here, we need a proper hardware partitioning mechanism. We need a mechanism that enables application developers to like have some control over like assigning which peripherals to which core and these are should be like utilize depends on applications like use cases. And so at the end, we want like developers, we want application designers to maximize their use of hardware so to benefit from all the hardware features that are available for them. And this cannot be possible with a fixed like partitioning, like a separation logic that is like cannot be changed by them. And so some of the like partitioning parts can be done aesthetically. For example, like secure world systems like a stack such as like DRAM usage, DRAM like memory ranges can be partitioned aesthetically because we don't expect a lot of change like an overall layout or some of like trust zone like GPIO or CAM interface for like crypto usage. So things like that, yeah, it's okay to be partitioned aesthetically and we don't expect like applications to change them a lot too. But there are other peripherals like DMAs, like GPUs these are should be like more on control of application designer. And that's what I'm gonna mostly talk about like what changes we need to do on like operating systems on like systems a stack between like this cores and also like the challenges we have on SIG for example, securely debugging this like enabling both like flexible and secure way of like controlling the hardware partitioning part here. For the hardware support like features for partitioning memory and peripherals also having a significant progress. This is a relatively active area we see like features like for example, RDC is resource domain and controllers and for partitioning memory and peripherals on iMix families. We see like SMM use on different hardware we see MPUs, we see XPPUs on like SPM like secure partitioning management on ARM V9 boards. We also had like trust and address space controller trust some peripheral controllers. All of these features are designed for like partitioning a part of the system. Some of them can be like cover more of the systems for example, like XRDCs and TRDCs for like covering the whole like Cortex-M and Cortex-A like interactions but some can be like only accessible for example, inside the secure board like trust some like controllers. And so they have different designs they have different protection capabilities and they have different like security models. And now we see that several of these features can be supported on the same device on the same SOC and the secure integrations of like these features based on like different turret models and like use cases, it's very challenging. And here I briefly introduce some of these hardware features that we are using for partitioning. For example, RDCs, resource domain controllers are features that like enable access control assignments on a wide range of memories and peripherals. And like we have different versions of RDCs like XRDCs, TRDCs depends on like the core we support but in a high level this feature enable us to assign a set of like resources to like different processing domains. And we have a limited number of domains like eight or 16 depends on the like platform. But when you assign like access control policies on like this set of like for example, a few set of resources on a specific domain and those resources cannot be accessed by another domain unless there is like explicit access control policy defined. And the policies, the policy can be like different for each privilege level. So RDCs like are aware of the secure world like privilege layers like secure world, kernel mode, user space mode and also the normal world like layer. So we can assign security attributes for like also logical separation of like secure world from the normal world, most memory and peripherals. RDCs are basically a combination of like multiple hardware sub-modules like multiple hardware components such as like MDAC for domain access control assignment and like every bus transaction for like such as pack for peripheral access control assignment like MRCs for memory region control and like other components that are different the implementation and their details are different depends on like the underlying hardware like which core Cortex-M or like Cortex-A. But in a high level like description, RDCs enable us to like assign access control policies for every bus transaction and every memory region and address space regions that we want and assign them to like a specific domains with like a specific domain IDs and assign specific security attributes to each of them to separate for example, secure world user space like resources from normal world or like from secure world, kernel like mode. So it has like a four level hierarchical access control policy. In a simplified example, for example, we're using RDCs we can have like, we can assign different domains for different scenarios or application sites like application stack that is better using like Cortex-A. We can partition their resources to for example trust on a specific resources would be like separated from user space like resources for application sites such as like DMA like DRAM like different DRAM ranges. And at the same time, we also could partition or Cortex-M for real time like workloads. So for example, in Cortex-M, like edge like secure enclave resources would be isolated from like TZM like trust on Cortex-M and like we're from DMA or GPUs and like micro power like partitions. So this would be like for each of like these workloads we can have like proper partitioning of resources. And also for the shared resources we can assign a specific like another domain. So the shared resources could have like a specific security policies that are more suitable for a shared environment than for like separate environments for like applications or real time workloads. We also have like a hardware, we also have the support for like multiple T's and enclaves and so as you probably know like T's are used for like shielding sensitive code base like sensitive code or data from host operating system or hypervisors that are having a like really large like code base and we want to protect this like sensitive parts of our system inside the like trust or inside like secure enclave. So these hardware components are designed for reducing the attack vectors and reducing the TCP of the entire system. And T's are usually like supporting different features and they're not all like supporting the same set of capabilities. For example, like for even like trust zone there are different implementations of like trust zone some only like protects like memory some like for example, TZPC like if enabled it's like allow us to isolate trust on a specific peripherals for like secure boots because like trust on by default it doesn't like enable it. It needs like some sort of like trust on basically virtualized TPM kind of capability inside trust zone or we are using like edge like secure boots for example, for as an alternative for like pure hardware based secure boots feature. So the features of like enclaves for like for attestation for measurement for like ceiling and these are different but that's why like if like an enclave cannot provide you like with all of these features it's usually like better to have like several like different components and combine them together to get like a strong security features that we need for different use cases. And when we have like trust on a specific like partitioning capabilities for example, trust on like PC for peripheral controller like address space controller when these features are like together we also have like additional capabilities for trust on a specific like hardware resources and the integrations with like RDCs like for trust on like Cortex-M, Cortex-A and like the RDC like on each of the cores the proper integrations like in a secure way in a like effective way it's really important. Also especially like that we're going to have even more fine-grained partitioning features like in its RV9 architecture with like realm support and like confidential computer architecture that we're going to have like even more partitioning capabilities partitioning manager for like fine-grained like privilege separation inside like or secure both. When we have such a genius environment our partitioning framework is really important to be aware of all of the trust boundaries that we have on like underlying hardware. And that's why or like partitioning framework here as you can see it has like several components running on different privilege layers from like or trust on kernel, TFA on like secure monitor mode and the Linux kernel on normal board and all of these components are collaborating together for getting updates on any like hardware partitioning changes assigning security policies and like making sure that the system is like all of the policies are in a stable way. And they also, we also need like cross boundaries like collaboration for enabling security debugging because like some of these partitioning policies are related to like trust on a stack some of like related to the kernel, the new boots the debugging cross this privilege boundaries is a challenging task. It needs like all of these layers to collaborate with each other. Also another important design principle for us is to be able to scale or hardware support. As I mentioned before, we're going to like see more and more like hardware security features for compartmentalization for like isolation. And if you really want to like properly utilize this hardware, we need to design the systems to be able to scale to maintain the stacks like in a secure way to like have an abstractions that could be like could have different back ends, different T kernel back ends different hardware drivers. And this is important for like properly using the hardware security features on or underlying hardware. On Linux side, we have our spear partitioning module that can be enabled with kernel config spear partition. And this module is the first layer of our abstractions for parsing and getting all user space like partitioning configs, partitioning policies from a specific like device tree overly and the interfaces for like all range of like peripherals. So this module, the main job would be like to have this unified abstraction, parcel of the required policies and configurations and it constantly collaborating with our T partitioning module. So it's collaborating with our T driver for like assigning the shared memory buffers and RPCs for checking the updates and security partitioning policies and also asking the TE for changing a specific policy. So you can see here, for example, if an application developer wants to like enable an RDC protected, like XRDC protected, can like crypto accelerate around their like platform under core, they can just use this device to a simple device tree config. And then the partitioning framework does all the hard work for seeing if there is a conflict or like it can be enabled and enable the whole partitioning policy. And we have like the similar interface, similar abstractions for all the peripherals, available peripherals on our devices. And these are the basically the devices that can be shared and these are different from like security sensitive like resources that we already like assigned for our secure world. These are important for like security of the whole systems, for example, for trust on a stack and like interactions with or like, like Enclave. And these are basically the hardware resources that can be shared and it's okay to like, to be under control of like application developers but definitely are separated from the security sensitive like resources that we already assigned to our like TE stacks and all of our security sensitive components. There are also some like hardware-based limitations and that is causing some non-trivial challenges for like on our Linux side. For example, when we have like a TZA or TZM enabled, we already have like assigned some security attributes on different domains. And that's why the Linux domain, like on the normal work when it's running on a normal or it cannot read or writes like privileged registers from that are going to affect the other domains, layout and like policy changes. And that's why we are using our trusted work, the component partitioning components to finalize the actual configurations for assigning the policy. It has like the right access to the registers, to RTC registers. And also this is important, for example, for domain-specific fault handling, for like debugging between different domains. And also like inside our Linux environment, since we also having a virtualized environment for like application, like virtualization and none of like these hardware features can be actually like for example, RTCs can be virtualized. So in a power virtualized environment, we have some challenges for exposing these features to our like guest virtual machines. We are resolving most of these challenges through our trusted like T partitioning module. And this module is the one that is like responsible for like finalizing the actual partitioning. So it has the hardware, different hardware backends like all of these hardware features for partitioning like XRGC, TRDC, all of these drivers are back-end of this unified abstraction. And it's also responsible for enabling secure error handling between and secure file handling between different privilege layers. So the interactions between like these components can cause some performance overhead. But we also need to do more evaluation like on that based on different application scenarios. But currently it doesn't have like a large overhead, it's reasonable and mostly that shared memory and like RPC communications, the comments for collaborating between these components are very small. So it's not like both it's not even like a large buffer share large data transfer or also like the attack vectors between them. It's a bit kept and careful, it's very narrow and small. So since we are enabling a harder partitioning on like the whole system, this is a larger scope with like lots of fine-grained memory ranges and peripherals. So proper file handling and secure debugging between all of these like domains is challenging. Especially like, for example, when access control violation happens in one of the domains, it can happen, it can cause like RDC faults in that domains or it can like be cascaded and cause multiple violations on different domains. So fortunately we have like fine-grained debug registers and debugging support for all of like the all of our hardware like resources that we are partitioning. But the problem is that like not all domains can access read or write and they don't have that read or write access to other domains like debug registers. So for properly like debugging between like all these cross like boundaries, we assign a specific like debug domain and or TE basically partitioning module. That one is like, it's responsible for giving this debug domain the privileges for the privileges for like monitoring, for reading and writing and for handling all of like partitioning a specific faults that any access control violations that are happening between like different domains. And that's why we had also some synchronization issues with our Linux kernel module that's like because the Linux domain was different from like this domain and when you want to like get some of the like debugging registers, the context switching and the synchronization for the Linux fault handler to like switch to that domain was causing some issues. And that's why we like adding like a specific SMC cost to our like TE kernel to our TE partitioning modules for giving the right like error messages for like the full report of what's going on and which domain to run pass it that to or Linux kernel for like proper debugging. So here, for example, you can see like a simple like version of our error reporting mechanism for like access violation on like one of the partitioning configurations. It says like, for example, this DDR like this DRAM, this region has the access control violation on like domain seven and in that address, in that address and there are like also multiple access violations that are coming from the right access. Like they're coming from like non-secure or privileged mode that's our tried like to have the right access and the access causes like access policy violations on that partition. So we also had like a specific like limitations, we see like limitations from like open OCD, GDP and for like domains because they're not designed for like supporting this modern hardware features for debugging and we need to like also extend this features for considering like for like extending them to this hardware debugging features and these concepts of like partitioning, the concepts of domains. So these are should be integrated to our debugging frameworks. So in this talk, I briefly described the problem of the problems in hardware partitioning and heterogeneous architecture as how it's challenging to do it securely in a more flexible and more dynamic way. And I describe our current solution, current design and some of the challenges that we are still working on that because it's an ongoing work. As a final point for this community, it's important for us to know that like since we are adding more and more hardware security features for peripheral isolation, for partitioning, for memory isolation and these features are going to be integrated in one single and heterogeneous architecture in one device and it's important for our systems as that especially our operating system to have to know these concepts of domains like to have the right abstractions for the partition resources. And without this abstraction, it's hard to enable secure and like effective ways for cross-domain collaborations for cross-domain like debugging and attacking investigation systems. So this feature, this concept is still kind of missing from our systems as that and especially from our operating system. I'd be happy to answer any questions you have and thank you.