 Good morning, everyone, and welcome to the first session of the beginner's track. So I am Pradipta. I will be one of the speakers today. And my colleague Prashant. Yeah. Hi, I am Prashant, background in Applied Math and Machine Learning, co-founder of Encrypt.ai. Thank you, Prashant. So I am Pradipta. I work in Red Hat in the area of container security and isolation. So how we are doing this? We are kind of, this session is divided into, let's say, three broad areas. So the first area is where we will set the problem statement, the context, and the problem scenario. And then we touch upon the solution that is there for the problem. And then followed by the third one, which is about a practical usage of how the solution is used in real field. With that said, let's get started. OK, so let's start with a story. And hopefully, you will be able to kind of connect with this story in today's age, with the age of generative AI. So let's say we created a fine-tuned model, which is for helping with mental queries. And this could be an open-source model which you fine-tuned with your data, or you created your proprietary model, whatever it is. And then we created chatbot with this. So what do we do? We created a chatbot, published it in Publicla. And deployed it in public cloud for others to make use of it, whether it is users, business partners. And our chatbot becomes hit. We start earning revenue. Now, this may give you that this is a fictional story that we start earning revenue. But yes, I think many of you would be able to kind of relate to this kind of a scenario where you have a model and it's being used. Now, when you start, when your model or when your app becomes hit and you start earning revenue, can you avoid adversaries? No, right? So when the adversaries, what happens when the adversaries enters? So let's say a scenario where you receive a ransomware call, or you get a notification about that part of your model with a snippet, with a proof that your model has been taken out, so your proprietary or your IP is in the hands of someone else. And they are asking for money. So what it means, your model breach or model theft. So this is a very common scenario. And this is the target of the adversaries itself. This is also about we are talking about a mental headbot, which means that as a user, if say you are a user interacting with that, you will be sharing confidential data, your own personal data. Because if you don't share your questions, which is personal in nature, then the chatbot is of no use. Then what if that data gets exposed? Yes, you have all the checks and balances where you have written, OK, you won't. As a provider of this software, you have this agreement that you will be using the data in specific way and form. But still, outages or still these kinds of incidences happens, right? Your private data can get exposed. So we call it as privacy. So while all of us in this field, all of you here, our intention is to not let this happen. But we have to continuously work towards preventing this happen. And this may happen. And if this happens, what are the consequences? So you cannot avoid the consequences if that happens, right? Bad press, yes. Everywhere in social media, you may have your company's details available, right, that what has happened. Most importantly, loss of users' trust, right? And so while we set this up, so loss of user trust is great. We are talking about a case where mental health chatbot, people come to your app with the trust in mind that the data is secured and it is not used in untainted, dead fashion and all. And if this happens, loss of user trust, right? Loss of competitive advantage, if your model is in the hand of someone else, your competitive advantage goes away, right? And last but not the least, regulators at the door. So what are you doing? How it can happen that the personal data or PII, as we call it, is exposed outside. So regulators at the door. Now what it means, right? It's a revenue loss. While everything cannot be kind of back to revenue, but in the world that we work in, revenue loss is a big thing. And most importantly, which I consider is loss of user trust. Next, when you come, can your users really trust you, right? So when this happens, what's the next? So we kind of do some introspection. So when we do introspection, what do we try to do? We try to kind of figure out or try to see that what could we have done differently? Talk about API security. Yes, I think all of you know about API security. What more can we do there? Then data security. So you have the chatbot where the data is sent to the chatbot processed and you get an output. It's an input, process, transformation and output, right? So how do you process? Is your end-to-end data flow encrypted, right? So that's one. Then better container security. You are deployed all these apps. Your apps is all running in Kubernetes cluster. So are there, are your images secure, right? Are they free of vulnerabilities, right? What about the container runtime? Now think of another scenario as well where your container image is not having any vulnerability, but in a typical Kubernetes cluster, your container is running alongside other containers. What if some other container was able to have a privilege escalation and via that the adversary is able to get all your data? You have done your best way foot forward. You have put your best foot forward, but you cannot prevent what happens for the other workloads that runs on the same system. What about the runtime environment security? Consider this case of adversary. So let's not talk, let me just for a moment not use the term adversary. What about the cluster admin or infrastructure admin? In a cluster, it's in public cloud, infrastructure admin has access to everything. They can take a memory dump, isn't it? You can simply do a memory dump, get everything in hex, convert it, and you get everything. You can do that. Now when we interest, now can, is there something which can addresses these gaps? So think of it this way. We implicitly trust certain users because we don't have a choice. We have to trust the infrared wind, we have to trust the SRA because they are someone who will access the system, fix your bugs, maybe figure out what is happening. But do you really need to trust these set of people or these entities with your data, with your model, with your IP? So that's the kind of thinking which we want, right? And looks like today with current technology, you can do that. So let's recollect a bit. So what I was talking about, your adversaries, which one is could be definitely hackers and all, which of course you don't trust the hackers, but they are there. But think of the people whom you trust, these infra admins, cloud admins, cluster admins. Now you always have these checks and balances when let's say you enter into a contract, you will have license agreement which tells the infra admin cannot do so and so things, but that's on paper. What's the technical enforcement of that? So you have contracts. So your user signs, let's say when your user comes, you ask the user, you give a license agreement, right? Okay, you say that I will use your data in so and so way, but that's between agreement between you, provider and the user. But is it enforced, right? Let's think about that. Then think of it this way. Can we ensure that even if the adversary accesses your compute, still your sensitive data cannot be exfiltrated from that compute? Here we are working with the assumption that you have defense in depth, you have taken care of everything, but you cannot prevent adversaries. They will come. They will try to figure out different ways to do stuffs, but what more protections can you add, right? So this is something. And then last but not the least, we are talking about AI model which if compromised, one is someone taking your AI model and your IP goes away. That's one part of the story. What if your model is compromised itself and it is modified in a way that it is giving wrong data? You as a user, can you be sure that the model that is being used has not been tampered with, right? So can you do that? So we will talk, of course, you can do that, that's why this presentation is, right? But I want you to think this aspect. So this is very important from where we are heading. The problem is there, we cannot, we can always try to prevent the adversaries, but we also have to assume that there are adversaries, there are implicit trust relationship that we base on a platform, on set of users. And we do have contracts, consent, governance, we do have. Every one of you has that. But can it be technically enforced? So today, there are technologies which helps you to have technical enforcement of this contract. So you have concepts like differential privacy, homomorphic and confidential computing. Now all these are there, in today's session, we will focus on your confidential computing, right? How confidential computing helps to have this technical enforcement. So that's the focus of our call, right? That's where it is. Now what is confidential computing? So think of you have a black box. And that's, we call it as an execution environment. Simply put whatever runs inside that execution environment that's protected from any entity that is outside of that environment, right? So if you look at the picture here, so TE as I call it, it is the trusted execution environment. So your app, your model is deployed inside that TE and your cluster admin, infrastructure admin, any other process, any other containers that is outside of that, right? That's outside of that. So there, what happens is that unless you explicitly allow as a workload owner, no one will have access. In fact, even if they do a memory dump, let's say your cluster admin, your infrastructure admin, right? As we call it. So infrastructure admin has access to everything. They can actually do a memory dump. They can do that here also. And maybe someone who is compromised or whatever, but even if they do memory dump, what they get is garbage. They don't get plain text, right? So that's, now if I go a step further, so this is like a very layman view of what it does, but little bit further down the road, what all it does? As I said, if you dump a memory, you get garbage. So it provides memory encryption, right? This is one of the foundational technology that confidential computing provides. Then it also provides isolation, attestation. So isolation, I think all of you understand. Attestation, I want to just spend few minutes here in attestation, which is one, again, one of the fundamental properties of confidential computing. Attestations, let's say I as a user, how can I trust that when I deploy my model, or let's say I as a model owner, or I as a owner of the IP, how can I trust the environment on which my IP is deployed? How can I trust when you are, when let's say a provider of my infrastructure claim that my infrastructure is secure, how can I trust what the person is telling is 200? Can I technically trust it? That's one. Another is, recall I talked about the model is compromised. You as a user, how can you trust that model is not compromised? Can you verify that the model has not been tampered with? And that's what attestation is all about. Attestation is kind of an exchange which gives you the proof that what the remote entity is telling is indeed true and it's not fooling, right? That's what attestation is all about. And then integrity. So again, this is also tied to what I was telling, what your code has not been tampered with, right? What is running inside the TE? How can you be sure that someone else has not went ahead and tampered with it, right? So that's, so this is the fundamental property of confidential computing. And that's what actually helps to create that technical enforcement that I was talking about. So instead of implicitly trusting the adversary or the admins, we remove the trust altogether. We don't even implicitly, we say that we don't trust you, right? I don't, if I am running my application in a public cloud, in a Kubernetes cluster in public cloud, I don't have to trust the provider of the Kubernetes environment, right? That's what it gives. Now, little bit further into this question, right? So fine, you saw that if you run inside the TE, then it's protected, but the problem is how do you run inside the TE? Because that's the next logical question, right? So there are two options today. So today in the CNCF world, there are two options. One is your Kubernetes worker node. The entire Kubernetes worker node can run inside a TE. It's possible today. So we call those as confidential clusters. You will hear the term confidential clusters. If you Google for it, you will get that. So you run a Kubernetes worker node in a TE. So any container that runs in that specific worker node is protected from these adversaries, right? So it adds the difference in there. There is another. You don't want to do that. You don't want to run every workload there for various reasons, cost reason, and there is also this privilege escalation, but you only have a requirement to run a specific workload, which is your AI workload, which is your sensitive workload into TE. So you can do that also. So you have both granular and other. And how do you do that? No special UX, nothing of that sort. You can continue to use the same Kubernetes spec, Qubectl command, Qubectl apply, Qubectl grids, all these things work. Only difference is how do we expose this confidential environment to the end user? So there has to be something which tells to the user that I have any confidential environment available. So that we do via what we call as Kubernetes runtime class. So the entire confidential support in a Kubernetes cluster is exposed via runtime class. So the expectation is a user who wants to use confidential computing to deploy or wants to use this technology and deploy their workload needs to use the specific runtime class, right? And we'll see those things. And this is the project. So you can go to github.com, Confidential Container. So this is the CNCF project. I'm one of the technical steering committee member and the maintainer of this. So this, and as why this project, we are trying to help all Kubernetes workload to adopt confidential computing, right? And again, further down to before we segue into the other stuff is the trust. We talked about the trust part of it, right? Because I am continuously harping on trust. You don't have to implicitly trust. And what do we mean? And this might help you a little bit. So if you look at the legend, the yellow background, what we see is who has trust on which components? If you look at the top one, the top one is today your workload which is running without confidential computing. So you see who all has access. Infrastructure admins, as I said, have access to it. So your workload is the dotted one, which is your VM and app. So I have used, and the reason I have used VM and app is two-fold. One is if you look at regular containers today, it runs as namespace or C-group isolation, right? That's what. But you also have an additional layer of isolation which you can get, which prevents like some other container impacting your container, which we call as the VM port sandboxing, we call it. So we run the container in a thin or a lightweight virtual machine. So we had another layer of isolation. So we have these two models available today. But even with these two models, yes, there's defense in depth, isolation is available, but the trust thing doesn't change. So infrastructure admin, OS hypervisor, processes running in the OS or hypervisor, then your cloud admins all have access to your workload data. Now they don't access it is by contract, but if someone misuses that, you have to technically enforce it. And that's what confidential computing does. If you look at the second picture, if you use a VM-based trusted execution environment, right? So now it gets into little deeper. So as I mentioned about trusted execution environment, so today you have two types of them. One is your entire VM can be a trusted execution environment. So this is what the second one shows. So your app runs in that. So who has access to the data? Only CPU. So you trust only the CPU. Infrastructure admins, BIOS, then host OS, cloud admins, no one, right? CPU, you have to trust. One entity you need to trust. So trust is with the CPU itself. And then there is another type of TE which we call as process TEs, which means instead of the VM as a trusted execution environment, your process, right? So individual process can be put in a trusted execution environment. You can think of it as a more lightweight aspect of confidential computing instead of VM. But there is another aspect, our focus, of course you can use both, but the process one requires one to change the application, to rearchitect the application to a large extent, or add some more components which you may or you may not want. When we go to the VMTEs, it's very easy. You take your existing container, just deploy it inside a VMTE, that's about it. You don't need to do any changes. But you need to have a VMTE, other aspect, but from your workload and application standpoint, no changes, right? So with that, I hand over to Prashant who will take you through the next pieces of the presentation. Thank you, Pradipta. So let's take a recap of the use cases, right? So because this is AI and machine learning based workload that we are looking at, so primarily we are looking to secure model inferences. So when you do like, when you interact with chat GPT, how can I ensure like end-to-end encryption of the data that is being shared with chat GPT as well as the model itself? So that's one use case. The second use case is in like healthcare or in banking where they might want to train ML models on highly sensitive data. So how can I ensure that the data does not get tampered with along with the model as well? So those are the two use cases that we are currently addressing at Encrypt. From that perspective, right? Like as Pradipta mentioned, just like what do we need as a bare requirement for these workloads? That the code, the model and the data should not be accessible to untrusted parties. You should be able to be in control as to who has access to it. And even if the admin gets access to it, they should not be able to make sense out of it. Then when it's being used, we should be preventing tampering of the code. So those are two cases, right? So one when it is in REST or in transit, but also when it is being used, we have to ensure that it's like at most privacy and security is maintained. So, you know, a typical workflow would be like this, where you have a model user here and you would have like a chatbot, right? So this is the machine learning workload and they would be sending and that would be stored inside of a Docker hub or a Quay image registry and that can be encrypted. So that's the data at rest that is secured and outcomes like an output as well. So what we provide with leveraging CNCF is that even in use the workload that is running inside of the container is also secured. So the privacy of the whole data and the model is maintained. So just a quick intro about us. So we are Encrypt AI and we are building the control layer for AI models. So for any commercial strategy, privacy and securities of utmost importance, right? So we are building the control layer. So the aspects that we cover are privacy, security and visibility. Privacy, no sensitive data should get leaked to the ML workload. It could be either PII information or any other confidential information. It should not be leaked to the model that is being used. And security can come in like you must be hearing about all these injection attacks that are taking place like making the model do nefarious purposes as well as the security of the model also gets compromised, right? So that's what we are pioneering. So we have built IP around all the three pillars here. And what happens is like as Pradipta was sharing about the mental chat board so you'd have inputs and outputs. Either they could be APIs or a model file, right? Where we come in is primarily like we do the access controls and authorizations and ensuring that only, you know, non-sensitive data gets to the LLM or the model. That means like all the GDPR and HIPAA compliance are met when you do that. End of the day, you get all the usage logs so it's very easy for your compliance on or your audit teams as well. So in a nutshell, let's, you know, get to the how Encryptia is leveraging CNCF projects to achieve the objective. So you have your user and then you have a pod here. We have our ML model inside of this Kubernetes pod and we have an attestation agent. So Pradipta was mentioning about the implicit trust, right? So that's what this attestation agent tells that the workload is indeed the workload that you're supposed to be running. And we also have a verifier that makes sure that the end-to-end data is encrypted, right? Whatever data is interacting with this pod is going to be encrypted. So it protects the entire AI workflow and also is responsible for key exchange. So nobody else can access these keys except for the verifier and the end user. So let's, in interest of time, let's take like a quick, shall we do the demo itself then? Yeah. So here at Encrypt, what we have done is we have built a MVP around like taking your regular containers and deploying them on like a confidential VM, right? So that process has been streamlined on our end. So in this demo, what we are showing is we are showing a chatbot. So we have taken LLM and containerized that LLM and deployed it on a confidential VM. So initially what will happen is I'll need to do a connect. So when I say like secure connect, that's when the keys for encryption are exchanged with the verifier and the end user. So once the keys are verified and also you can get some attestation reports that it's the authorized user as well as the authorized workload that is running on the container, right? So that's, we have to maintain that at most like verification and trust reports. So coming to the chatbot here, so I'll just say like, hi, tell me about CNCF projects, right? This is not a sensitive conversation, but you can imagine this could be like a mental health chatbot. So one of our customers is into customer analytics and they record like a lot of confidential conversations with their sales teams and HR calls. So just, I'll just scroll up. So you can see that when I said hi, tell me about CNCF project, what's get actually sent to the Kubernetes pod is an encrypted message, which then gets decrypted by the verifier there. And what gets sent back from the Kubernetes pod is a encrypted message as well. And the user has the decryption keys for it. So you can imagine that the interaction with the model is always like encrypted as well as the workload is also protected. So that way you can ensure that it's end to end data protected as well as model privacy is also maintained. So that's in a nutshell, one of the pillars that we are pioneering at Encrypt, but please do reach out to know about how we actually do it. We have like, Pratya, do you think it's a good idea too? No, so you saw the secure connect here, right? So this is something which Encrypt AI has developed on top of what CNCF confidential containers provides. Now what this does is basically proving that, how I was telling right, how as a user I know that what is running is really what it claims to be that is station part of it, right? And our CNCF confidential containers project do provide some of these primitives which is being used to prove. So it's in simple words, the attestation agent that you saw in the previous slide, it will send a request to an external entity which has good measurements or values, good values, acceptable values, and it does a match, okay. This environment tells that it has these things. Now are these valid? Yes, those are valid. And those data that is coming from that environment cannot be tampered with because of the T environment that you have, right? So that's how it kind of builds on top of it and you get this kind of an end-to-end workflow. So yeah. Thanks Pradipta. Thank you everyone, please reach out if you have any questions or just to chat.