 All right. All right. Hi, everyone. Welcome. Today, we're going to be talking about your very, very many storage and Kubernetes with Manila CSI. My name is Victoria Martínez de la Cruz. I'm a senior software engineer at Red Hat. And I'm Christian Schwede, also software engineer at Red Hat, working together with Victoria and the storage team, OpenStack storage team. All right. First, we are going to quickly cover the agenda that we prepared for you today. First, we are going to start with an introduction of Kubernetes storage and CSI. Then we are going to move to senior CSI and Manila CSI, which are the storage options that we have in the Cloud Provider OpenStack. Then we are going to do a brief introduction about OpenStack Manila. We are going to talk about how to deploy Manila CSI. Then we are going to show you a really base use case on how to use Manila CSI. We have a link with a demo that you can check out later if you want as well. And we are going to wrap up with some future work that we have in our agenda. All right. Thanks, Victoria. So let's start talking about storage and Kubernetes first. So storage and Kubernetes, even if you're running the most fancy status application today, you still need some persistent storage, right? And for example, your database wants to store some data, and you run it on the containerized orchestrator, so you need some storage at the back end somewhere. So Kubernetes introduced this back in the release 114 with the idea of physical volumes. And these were when these became generally available. All of the storage drivers were entry drivers back then. So that actually means that any storage appliance that you want to use, the drive existed in the upstream core Kubernetes distribution. Now, that actually required vendors to participate in the upstream development model and to contribute to the upstream source code. It also required that they align with the release cadence. And for the upstream community, it actually means that they had to review and sometimes even maintain storage code. So the requirement to actually have a more flexible way came up and to add storage drivers for container orchestrators or Kubernetes. And this is how actually the idea of the container storage interface was born. And as it already implies from the name, it's not only for Kubernetes, it's for container orchestrators in general. So it was developed as a standard for exposing different block and file storage systems to containerized or container orchestrators. It also enabled the storage vendors to develop a plug-in only once, even if you wanted to use it. For example, on Kubernetes and some other container orchestrator, let's say Rancher, Docker, whatnot. Some part of the code is actually shared. But let's see, one second, sorry. But if you really want to go into detail, we have a couple of links available in the slides later for you about the containerized storage interface back. Because some drivers existed before, there need to be a migration path actually from using the entry drivers to the CSI implementations. And there are some migration tools available. And this should be done probably by now. One thing I didn't mention yet is there is some commonly shared boilerplate code. And this one is maintained by the upstream community. For example, the Kubernetes community, or the CSI community to be more specifically. So when you have a storage driver, you need some kind of plug-ins actually that maintain and interact with the APIs that are commonly used. For example, to attach and detach a volume. And these are sidecar containers and maintained by the upstream community. All right. How do you use Kubernetes and the CSI drivers running on top of OpenStack? There is some integration available, a set of plug-ins to integrate Kubernetes and OpenStack. And it's part of the cloud provider OpenStack. Actually, it's pretty nice to run Kubernetes on top of OpenStack. For example, what you can do is to give every department that wants to run their own deployment of Kubernetes their own instance with full access rights or full administrator rights. But at the same time, isolating these from the underlying infrastructure. And these deployments are then completely isolated from each other. So one department could run a completely set of different deployment or workloads to the next department. But it's not only about isolating the workloads or the compute workloads, but also about isolating storage workloads. And part of that is maintained or provided by the Cinti, CSI, and Manila CSI drivers that are part of the cloud provider OpenStack. And they actually share some set of features. For example, both of them provide specific access modes. I will get to that in a minute. In this case, it's read write once and read write many modes. Both of them provide access to multiple back ends and hard multi-tenancy using Keystone. The major difference from a user point of view is actually the kind of storage that you expose, with Cinti, CSI providing access to block storage and Manila CSI to shared file storage. OK, let's get to access modes. Quickly, there are basically four different modes that are most interesting. So read write once is basically you have one node that has access, read write access, to your volume. And I'm highlighting the concept of the node here. So any pod running on the same node could at least theoretically have access to the same set of or the same volume. There's read only many access mode, which is pretty nice. For example, if you want to, let's say, restore a backup on multiple nodes or many nodes, and probably most interesting, and this is what the talk is about, is read write many mode, where a volume can be actually mounted on multiple nodes, and all of these modes have write access, for example, to a shared file system. And pretty new, in Kubernetes 122 is the read write once pod mode, where a volume is actually mounted or available only to a single pod, which is pretty nice for sensitive data handling, as well as ensuring that only a single pod is writing to your file system. All right, so Victoria is our Manila expert, so I'm handing it back over to her. All right, so now we are going to quickly cover about OpenStack Manila, if you're not familiar with it, and compare that with Manila CSI. So let's take a quick moment to introduce Manila. Manila is the shared file system as a service for OpenStack. It has support for more than 30 backends, being those proprietary and open source. Some examples that we can mention are NetApp, CepFS, Dell EMC cluster, among others, and it also has support for multiple protocols, being those NFS, CIFS, GlasterFS, HDFS, CepFS, and more. Regarding features, OpenStack Manila offers a really extensive API that you can use to manage your storage backends and your shares. In Manila CSI, we have a smaller set of those operations that you can do, but those are under development in our case. But right now, what you can do is the basic operation for shared creation, expanding existing shares, creating snapshots, restoring firm snapshots, and mounting, obviously, your Manila shares. Now, let's take a look to the original design that we have when we started working on OpenStack Manila. Basically, in this scenario, we were envisioning an OpenStack Cloud, in which you will run OpenStack Manila service, and you will create shares with any storage backend that you might have in your deployment, and you will provide those shares to the instances running in your computer service, like in the case of OpenStack, that is NOVA, and you will be able to share that storage among those different instances. That is the most basic scenario on the initial use case. And now, we can extend it to the diagram that follows. Now, we can imagine an environment in which you have OpenStack Manila running as we show initially, and we can leverage the OpenStack Manila service, providing shares not only to the instance you already have running in your cloud, but also extending that to containers and your bare metal nodes. With the Manila CSI driver implementation, basically, you can leverage this deployment that you already have to provide storage for your container workloads. Now, let's make a quick stop, and let's analyze how Manila CSI plugin works. Any CSI driver is composed by two components mainly. You have the controller plugin and the node plugin. The CSI Manila controller plugin is the one dealing with all the Manila operations, basically communicating with the control plane. So all the operations that has to do with creating a share, allowing access, denying access, basically is the Manila CSI control plugin, the one in charge of doing that. Now, we have all the node-related operations, basically, all the communication that go to the data plane, which are carried out by another CSI driver dedicated for that particular file system. So in our case, Manila CSI node plugin acts as a proxy driver that communicates to CSI NFS, for instance, or CSISF, in order to perform the actual creating share operation or mounting share operation. This has been done this way in order to minimize the cost of developing a maintainer of drivers. It didn't make sense at the point to actually rewrite the whole thing if we could use what was already available in the community. Obviously, because of this, right now, we have support for these protocols. But we expect to see in the future support for more protocols added. Having this in mind, let's take a look now at this sample deployment. In this deployment, we are imagining an open-stack cloud in which you have your Manila service running. And in this cloud, you have two different Kubernetes cluster running. And you have two different storage vacancies, providing self shares and NFS shares or other storage shares. In this case, I want to make a quick stop because I want to call out a limitation that the Manila CSI driver has. Actually, it's not a Manila CSI driver limitation, but it's actually a CSI limitation in which basically, well, initially, we wanted to provide the opportunity for you to use multiple protocols using Manila CSI. You can only provide a story for one specific protocol for one instance of your Manila CSI driver running. So if you want to have multiple protocols, such as in this case, you will need to have Manila CSI for NFS running in one of the clusters and Manila CSI running with NFS in the other cluster. Obviously, you can have one cluster running in different instances of the Manila CSI driver. So this is not a big problem. You have a really easy workaround. But we expect to see some enhancement in the future for this case. All right. Now let's talk about how to deploy Manila CSI. All right. Thanks, Victoria. So when we want to deploy Manila CSI, let's first have a look at what we actually need to run it. So obviously, the Manila CSI driver itself, right? And the Manila CSI driver itself requires, for example, a couple of sidecar containers to, let's say, attach and detach volumes. We need a couple of roads and road bindings and Kubernetes, custom resource definitions, stateful set, and a demon set. You could actually deploy these manually if you wanted to. There are some YAML files in the upstream repositories. But actually, there's a better way to do. I get to that in a second. In addition to the Manila CSI driver, you also need the NFS CSI driver, at least if you want to run it on top of NFS. And last but not least, you need to create storage classes in Kubernetes to actually access the Manila share types. Yeah, the Manila shares. So how do you deploy Manila CSI on Kubernetes itself? As mentioned, you could actually deploy all the YAML files, but there's a better way using Helm charts. So we're going to use the Helm charts. First, we're going to add the repository for the Cloud Provider OpenStack, because that's where Manila CSI does exist in, and then going to do a repo update and install the Manila CSI driver. Basically, the same is then be done or needs to be done for the NFS CSI driver. So again, we have some Helm charts. I shortened the URL a little bit, but the full link will be available in the slides. But as you can guess, it's on GitHub, so yeah, you will easily find it as well. And then install the CSI driver itself. And the next step is to create actually a storage class or multiple storage classes. I skip that for a second now. Victoria will mention that very shortly, but there are multiple ways to do this or multiple ways to create storage classes, and you find a couple of examples in the upstream source code repository. Now, if you run the downstream version of Kubernetes, Red Hat's downstream version of Kubernetes on OpenShift, there's also a Manila CSI driver operator available. And this operator basically checks if Manila is actually running the underlying OpenStack deployment. And if so, it installs the Manila CSI driver, CSI driver, the NFS CSI driver, and creates a storage class for each share type. And the interesting bit here is that it also resinks. So when you create a new share in Manila, it will become as a new storage class within, I think, 60 seconds. A small note here that it doesn't delete actually storage classes. So if you delete a Manila share in the underlying OpenStack deployment, it will still be visible in the Kubernetes deployment or OpenShift deployment. The operator itself is installed by default on OpenShift 4.6 and newer. Before that, you had to go or to install it manually, for example, using the OpenShift marketplace. All right. Now, assuming that you have running Manila CSI implementation now, the next step is to use it. And for that, I hand it over back to Victoria. All right. So actually, using Manila CSI is pretty easy. Once you have your deployment done, we are going to start by defining a storage class as Christian mentioned. If you are using the Manila CSI operator, that's something that is going to be automatically for you. The Manila CSI operator basically is going to query Manila to get the share type or share types that it has, and it's going to create storage classes for you. Otherwise, a really basic storage class definition is going to look as the YAML file you have on the slide. The most important part that you have to declare, obviously, is the provisioner and the name for it that has to match the share type that you have created in the OpenStack Manila side. When you have this, you need to create your PVC. This is actually very simple as well, since, well, this is the PVC that we use in our demo. It's very simple. It has a couple of lines. The most important part that you have to define here is basically the access mode that you want to use, depending on your use case, and the storage class name referring to the storage class that we created before. And with that, I think we can move to the demo, which I'm going to share quickly here. Let me see if it's visible. Is it visible by the size of the... Yeah, okay. So let's hit play. All right, so we're going to start by creating the storage class that we mentioned. It's exactly the same storage class that we show in the slide, so in our case, we have it already created, but we are showing the YAML, so you can take a look. We are going to take a look into the Manila side and check that the type is created. That is the type in the OpenStack side. Now we are going to create the PVC. First, we are going to show the content of it. I think, oh, no, we're not. We create the PVC. We make sure that it has found it to a PV. It's found. Now we are going to check on the OpenStack side if the share has been created. We can see with Manila List that the share is created there. And now what we are going to do basically is create the container workloads. We are going to create three writers. It's a really basic writer definition. We have two running on the same node, and the third one that is running in a different node, and we are going to start looking into, well, various things now that we are running. First, we are going to check that the PV that we created has been mounted in the same month location, which is a slash month. Now we are going to check that all of them are writing in the same share. And lastly, at any minute, we can check that from one writer. We can see that the other writer is writing in a specific file in real time. We can do that in any of the writers that are running right now and will basically mimic exactly what we are doing in each of them. And with that, well, basically, that's the short demo that we have for you. It's accessible, and all the ML files that we are showing are accessible as well if you want to check that out. And we are going to close out with future work. Let me see if I... Okay. Perfect. So for future work, we have a couple of things. In our team, we're working on some performance testing. Basically, we want to check how everything, like all the levels of abstraction that we have are working. We have set from one side, we have OpenStack, and we have Kubernetes or OpenShift running on top. So it's a lot of things happening at the same time. We are doing some IO tests with FIO. We are also running some API-scaled tests with OpenStack Browbit, if you're familiar with it. Also, some latency tests. We don't have results yet to show it to you, but we expect that in our future presentation, maybe we can have some information to share with you. And secondly, I want to mention some experiment that some colleagues have been doing on backups. Actually, there was a talk at KubeCon in Valencia by Robert Basek. Robert Basek is one of the main or original authors of the Manila CSI driver. He shared some data that he has been getting by using Valero and RESTIC and also Canister IO when performing backups. And well, I don't know if that talk is already published, but if so, I know that there are slides, but I know if the recording is available. But if so, take a look because the information that he got was pretty nice. All right. So with that, we are open for questions. Merci. Could you try? Yeah, there is a microphone over there. Thanks. My question is, you mentioned there were a number of front-end protocols that Manila supports. Why would I choose one protocol over another one? I guess depending on the use case, you have on your business needs. OK. So within Kubernetes specifically for exposing, are there any performance benefits of, say, CepFS over NFS? Not really, no. Not that we are aware at least. So for Kubernetes specifically, it probably doesn't matter too much which one you pick. And the front-end might be for other workloads. Right, exactly. Most people should use NFS because that's what's familiar for them, but it really doesn't matter. Thank you. No problem. I don't know if there are more questions. If not, thank you so much. Thanks for coming.