 Hello, and welcome to the Rook Intro and Ceph Deep Dive. Today I'll be joined by my colleague Satoru, also Travis and Sebastian who will help out with the Q&A section. Before we get started, I do also want to mention Travis gave a lightning talk a little earlier in this cube con. Please take a look and watch that if you want just a quick broad introduction to Rook. But for now, let's start with some of Kubernetes' storage challenges. This is where I like to start when talking about Rook. Kubernetes is a platform used to manage distributed applications, and in the ideal, these applications are stateless, but this isn't always the case. This isn't something that can be always achieved. There is very often some reliance on external storage. This external storage is non-portable. It can be a deployment burden, and it also needs someone to manage the storage, especially for day two operations, which composes 99.9% of the bulk of what storage is. There also may be cases where you have storage available, but you are sort of vendor locked into your cloud provider or to something else you have set up in your organization. Rook aims to overcome these challenges. So what is Rook in a little more of a deep level? Rook makes storage available inside your Kubernetes cluster, and Rook supports multiple different backends. Each is a different Kubernetes operator and has its own custom resource definitions for users to create storage from it. Rook provides automated management for any of these storage operators. This includes deployment configuration as well as updates, and once the storage has been set up, it can be consumed like any other Kubernetes storage with storage classes and persistent volume claims for applications. Rook is also fully open-sourced using the Apache 2.0 license. The storage providers I mentioned before, Ceph is the most stable, but we also have Cassandra, a network file system, and you can buy DB available also. Some quick stats that we like to highlight for Rook. We're a CNCF graduated project. We have just released our version 1.6 of Rook, which we're very excited about, and we've recently hit 215 million downloads of the Rook application. And as promised, I'm getting into a Ceph deep dive here with Rook, so let's hop into that. Starting with, what is Ceph? Ceph is also open source, and it is a tried and true distributed software defined storage solution. Ceph is pushing 10 years old at this point and provides block shared file system as well as S3 compliant object storage. All in one package, it's been enterprise tested, and it has a lot of performance features as well. If you would like more information on Ceph, you can look at the link below. Let's talk about the architectural layers of Rook with Ceph, starting with the Rook operator itself, which owns the management of Ceph. There's also Ceph CSI, which is the CSI driver that dynamically provisions and then mounts storage to user applications. Ceph itself is the data layer which Rook encapsulates and manages. This first layer, Rook management, is shown by the blue boxes. It is primarily the Rook operator in the middle top, and also consists of some additional demons depending on the configuration. Something I can also note here about Ceph is that these Ceph OSDs that you see several of, those are object storage demons, and they connect to underlying storage, and then with Ceph, they aggregate that, pool it together in a software defined cluster, and then expose that to the user. At layer two, we have the CSI provisioning of Rook. This is where a user application requests block or file, and then those are provisioned based on a storage class and presented to the user. Generally, block is a read write once volume for only a single application, and file system can be read write once or read write many. There's also a similar system in Rook setup for object storage where a storage class exists, and the user can make a bucket claim for that object storage, and Rook then implements a bucket provisioner to create the bucket for the user. While I'm already talking about buckets in purple, if the user requests object storage, they can then get information about the storage work has created via a Kubernetes secret, and then connect via S3 protocol using those connection details they're provided in the secret, and you can see that in purple. For block and file storage, the user application is directly given access to via a kernel driver to the Ceph cluster underneath, so Rook does not sit anywhere in the data path, it is straight from the application to Ceph. I'd like to highlight some of the key features of Rook now that I've talked about the architecture. First of all, simplicity. Rook is really focused on simplicity, starting with installation, which really is installing some common components, including our custom resource definitions followed by the permissions that Rook needs to operate. Following that, the user can generally just directly install the Ceph operator, though there is configuration if they want, and following that, just configure the Ceph cluster resource as desired and then deploy that. There is an example of a very minimal Ceph cluster resource off to the right, so it really can be quite simply expressed. We also want to focus on as many environments as possible in Rook. It's pretty natural to think of bare metal, where you can bring your own hardware or use shared hardware and provide storage to your Ceph cluster. Rook also operates and is focused on cloud providers, and this is largely to expand upon cloud provider storage with Rook's capabilities. Digging a little deeper into that last bit, in a cloud environment, Ceph will use persistent volume claims as the underlying storage. There's no need for direct access to local devices, and this can give a consistent storage platform wherever Kubernetes is deployed, regardless of where Kubernetes is deployed. This is great for multi-cloud installations. This also can help overcome some of the shortcomings of cloud provider's storage, whether this is allowing storage across availability zones, or a lot of providers actually take many minutes to fail over storage that Ceph, and by extension Rook, can fail over in seconds. Rook also allows for a greater number of PBs per node, effectively limited only by whatever performance is there than strictly saying only 30, like some cloud providers. Some users also really want to get better performance to cost ratio by picking the storage from their cloud provider that gives them what they want for the best cost, and then using that storage within Rook. Rook is customizable for or basically any cluster topology. It is customizable across and within topologies, and in order to provide high availability and durability, we can spread Ceph daemons across failure domains. We can also deploy on specific nodes if you desire to totally separate storage nodes from application nodes and have some more support that way, or even just have a single storage node if you have a very small environment. Similar to ease of installation, Rook updates are mostly fully automated. Ceph updates and even Ceph major upgrades are handled totally by Rook. Similarly, Rook patch updates, for example, going from version 1.6.0 to version 1.6.2, that update is fully automated. You just need to tell Kubernetes that you want to update Rook. Rook minor upgrades do sometimes require some manual work. This allows for taking advantage of the latest features, and occasionally a Kubernetes Ceph CSI or Rook feature is deprecated, and users may need to take some manual steps to migrate, but this is pretty rare in more recent releases. Everything will be documented in Rook's upgrade guide for upgrades. We do still try to make everything as automated as possible and as simple and streamlined as possible. The CSI driver, which I've talked about a lot, provides a lot of the functionality that Rook provides. It allows dynamic provisioning of read-write once or read-write many volumes for both block and file system. It also allows volume expansion, and it allows snapshots and clones, though that's still in beta. There is an older flex volume driver that's still available, but support is limited, and we really recommend using the CSI driver whenever possible. Another feature that Rook provides is the ability to connect to a Ceph cluster that's outside of the current Kubernetes cluster. For example, if your organization already runs a Ceph cluster, you don't need to create a new one inside of Kubernetes with Rook. You can connect your Kubernetes cluster with Rook to the existing external Ceph cluster, as we call it, and then you can still dynamically create your block file or object storage for your Kubernetes applications. The Ceph cluster is just managed outside. I also talked a little bit about the object storage provisioning. This is something that is pretty unique to Rook. This allows administrators to define a custom storage class for object storage, and then a user can create what is called an object bucket claim or OBC, and following a pattern that's similar to persistent volume claims, the Rook operator will create an object bucket when requested, and then access to that bucket is given via a Kubernetes secret. There's currently a Kubernetes enhancement proposal called COSI, which is short for container object storage interface, which aims to supersede this functionality. Rook has a, what I've often called a proof of concept of the themes of COSI, and you can look forward to hearing more about COSI and Rook in the next, I would say year or so. I'd also like to specifically talk about the new features we've added in Rook version 1.6 really quickly. Firstly, starting with support for a new Ceph version, that's Ceph Pacific. In Rook version 1.6, support three Ceph versions. We generally aim to support two Ceph versions, so support will be dropped for version 14 in version 1.7, but we do want to give this time for users to migrate as they desire. Broadly, Ceph file system has a couple new features, which are exciting. We have support for multiple file systems per Ceph cluster, which is in general availability. It's no longer in a preview state. We also support mirroring a file system from one Ceph cluster to another. We also support high availability for Ceph manager demons, and pod disruption budgets are enabled by default, which helps us maintain a stable Ceph cluster during maintenance and updates. OSDs have a couple enhancements that we can call out specifically. OSDs can be bulk updated in parallel. Rook will do this automatically, and it respects failure domains to retain that high availability. Also, we're continuing to try to use LVM, the logical volume management less and less, for new OSDs. This has allowed us to restore support for creating OSDs on partitions, and just puts less stuff in the way of what we're trying to create for the user. I'd like to pass it off now to my colleague Satoru, who's going to give a demo. Next, I'll demonstrate how to create a Rook Ceph cluster. There are two types of clusters, host-based cluster and pbc-based cluster. In host-based cluster, you specify hardware configurations directly in Ceph cluster cluster resource. Then persistent data is on host pass. In pbc-based cluster, you specify volume claim templates for OSDs. Then persistent data is on pbc. Host-based cluster is suitable for simple cluster, especially if you use all nodes and all devices. But cluster resource gets complicated in host-based cluster if not all nodes are used or there are various hardware configurations for each node. In such cases, you might have to list all nodes and all devices like this. In pbc-based cluster, you are free from describing hardware configurations. Instead, you should specify volume claim templates inside a cluster resource. And you should specify the number of OSDs and the storage cluster name and the OSD size. Since pbc-based cluster is not so intuitive, I'll explain its detail by creating a simple pbc-based cluster step-by-step. I'll use this environment in this demo. There is a Kubernetes cluster consisting of one node. This node has two local persistent volumes, local 0 and local 1. And this Kubernetes cluster has a root operator. So this demo contains two steps. First one is create a simple cluster. The second one is expand this cluster. So let's start the first step, create a simple cluster. To create a simple cluster, let's apply samplecluster.yaml and toolbox.yaml. Samplecluster.yaml contains a simple safe cluster cluster resource. And toolbox.yaml contains toolbox.sport, the definition of toolbox.sport. Toolbox.sport is to execute safe commands from Kubernetes. So let's see the samplecluster.yaml. So it's safe cluster cluster resource. And the most important part is the number of counts and volume claim templates inside storage cluster by sets. Number of OSTs is one. And this OST is consumed from local volumes. And its size is at least 5 gigabytes. Okay. So let's apply this yaml. Sorry. Okay. Let's also apply toolbox.sport. And let's watch the list of all look safe ports. And these are CSI drivers. And next, this is monitor ports. Okay. So now monitor ports initializing and now running. Next, manager ports and OST prepare ports are created. So please wait for a while. Okay. Manager port is creating and OST prepare port is now creating. OST prepare port is to initialize the new OST. So now it's running. So it means configuring the new OST. So it takes a bit long time. So please wait for a while. Probably. Okay. Yes. It's completed. So our lock safe cluster is launching. And now initializing. Okay. It's running. So the get port. Okay. All ports are created. These are CSI drivers. And the manager ports, monitor ports, operator ports, and OST ports. And we also created, I also created toolbox port. Okay. So connect kubekontrol.get pbv. Okay. Our local zero is bound to pbc that is created by look. And its size is the pbc is bound used by this look safe OST zero port. Okay. So we also, we can confirm the status of the safe cluster by executing safe commands from toolbox port. Okay. So safe cluster is successfully created. And its low capacity is six gigabytes. And it's the same as the capacity of local zero, local zero persistent volume. Okay. So all ports are created toolbox, monitor, manager, and OSTs. And the local zero persistent volume consumed by OST, OST zero. So the next step is expand this cluster. It's very easy. You just need to increase count field in cluster resource. Kubekontrol, look safe, edit, save cluster. Okay. Look safe, storage, class, device, set. Okay. So currently, of course, the number of OST is one. So let's increase it to so and save. So let's see the, let's watch the port and look safe namespace. Now, look safe operator port is under reconciliation. Okay. The new OST prepare port to initializing the new OST is running. It's initializing the new OST. So it takes some times. Okay. It's completed on the new OST one port created. So it's running. Okay. So the OST one, new OST one is added to my safe, look safe cluster. Kubekontrol get BV. Okay. Both local volumes bound to OST ports. And let's see the cake, safe status. So now the low capacity is increased from six gigabytes to 12 gigabytes. So it succeeded to expand my Kubekontrol my safe cluster. So if you need to increase the number of OSTs more and more, you just need to prepare PVE persistent volumes and increase the count field. It's very easy. And there are some advanced configurations or PVC based cluster. So first one is the, if you use CSI drivers with dynamic volume provisioning, you can create persistent volumes for OSTs on demand. In other words, if you don't need to prepare persistent volumes before increase the count, you increase the count, then persistent volumes are created on demand. And the second one is if you need spread OSTs evenly among all storage nodes, I guess it can be applied to in most cases. You can use topology spread constant future in Kubernetes. So for more information about these topics, please read this blog post. Thanks, Satoru. And thank you all for tuning in. I'd like to leave you all with some information about how you can get involved or just get more information about Rook. I'll leave this up while we now start our Q&A session. Thank you again.