 Alright, hello everyone. Welcome to the Rook intro and step deep dive talk for KubeCon North America. Here we are virtual again. Happy to be with you. I am Travis Nielsen, one of the maintainers on the Rook project since it's beginning. I work for Red Hat and happy to be here with Sebastian. Hello, virtual KubeCon. I'm Sebastian Han. I work with Travis as one of the Rook maintainers and also work for Red Hat. Thanks and also to answer questions, we'll have Blaine Gardner and Alexander Trost, other Rook maintainers. Alright, first big announcement. Rook is graduated. We are very excited about this. We've been working hard on this for the last few years. Just a little history here. So Rook did get initially inducted as a sandbox project back almost three years ago in January 2018. He progressed to incubation in September of that same year and then graduated now in October just this last month. So thank you, a big thank you to the Rook community, CNCF committee, Rook maintainers and everyone involved. It has been a wonderful journey. We're grateful for your support and look forward to continued growth in the project with that support. So what does this mean? To get to graduated. It really means that we have a lot of large scale production deployments out there. We know people are using it to store their critical data in production environments. And this is, it's a great evidence of that. We went through a security review by the third party that CNCF helped us coordinate. And maybe the third point is that really Rook is an open project. It's not only open source, but the governance around the project says that, Hey, this is this is vendor neutral. We have multiple companies contributing and expected to contribute so that it stays or remains a project that is useful to the whole community and not vendor driven. So again, thanks to for all your support. We've had a great journey to get. All right, let's get started on, you know, what, what are we talking about? What is Rook? What is that address? What, what are issues with storage and Kubernetes? Well, Kubernetes is traditionally a platform, of course to manage your distributed apps. And those apps have been stateless for the most part. If you want storage, you rely on external to the cluster storage that's not portable, you have to figure out how to deploy it. You have maybe you have a reliance on a cloud provider to give you that managed storage. And quickly you get into a vendor lock in if you're using cloud storage. If you're managing that storage, okay, you have to, you know, day two operations at once you've set it up, you've got to upgrade it, you've got to manage it, make sure it stays reliable. So all of those are challenges that that everyone has with any stateful applications in Kubernetes. So here's where it comes in. We saw this as a real challenge with Kubernetes and we wanted to address it and kind of solve this problem for the community. So Rook really does have the goal of making storage available inside your Kubernetes cluster. It's not something external. It's something that's deployed as any other application in your Kubernetes cluster. It's managed with Kubernetes operators and CRDs, just like any other application. And because it's driven with operators, the deployment of the storage platform is automated. So deploying it, configuring it, upgrading it, you get to decide how it's deployed through the CRDs and other settings, but ultimately Rook manages all of the headaches of that day two management while day one as well to get installed. And once you have deployed Rook, now you can consume the storage just like any other Kubernetes storage with storage classes with persistent volume claims. So bring the storage platform inside Kubernetes is the point. Rook is open source with an Apache 20 license. So it is. So what storage are we talking about here? We have several different storage providers in Rook at different levels of progression. So Ceph is our stable storage provider. It's been in since the start of the project. Since then, we've added other storage providers as well to expand the storage platform. NFS, Cassandra, UiDB, CockroachDB. These are still in alpha and we're always looking for community involvement to help grow those. We do have one storage provider that is deprecated. So EdgeFS, the owners of EdgeFS are working on a replacement for it. The timeline of that I'm not aware of, but that's why it's deprecated since something new will be coming. All right. So kind of overall project. How big is Rook? Where is it? So version 1.5 is our latest release. At least it will be as of TubeCon. I have November 2020. We will have that release out. As far as how popular project is, all the GitHub stars, downloads, going on 200 million downloads now for our containers. About 300 contributors now on the GitHub project. And as mentioned, we are a graduated project. So thanks again to the community. These are just a few of the stats that show how much it is growing. Okay. So Rook and Ceph. If we're getting into the Ceph portion of this, what does it mean to bring Ceph into Kubernetes? That's what Rook will do. Rook will bring the Ceph storage layer into Kubernetes that make it work well. So what is Ceph? If you're not familiar with it, Ceph is another open source project, but it is a software-defined storage solution that provides block, shared file system, and object storage that is S3 compliant. So these are really the three building blocks of any storage system that you'll find out there. And Ceph has them all in one system. No need for deploying one solution for block and another one for object or file system. Ceph provides all of those types of storage together. So early in the project, we really saw Ceph as being a reliable platform. It's been around for a long time. We just wanted to bring it to Kubernetes. So how did we bring it to Kubernetes? Think of it as having three different layers in the system where Rook is really the operator in Kubernetes that owns the management of Ceph. So the deployment of it, the upgrading of Ceph, like Rook manages the configuration of Ceph. If you're familiar with Ceph, you know that it is kind of a complicated system to deploy as distributed storage. So Rook takes a lot of that complexity out of it and manages it for you. Second layer now, we've got the Ceph CSI driver. So the CSI driver, just like any other storage platform in Kubernetes, will provision and then mount the storage into your app pod. So this is how you consume the Ceph storage once you have Rook installed. And then finally, the third layer is the data layer. So when you are reading and writing data to the cluster, Ceph is purely the one that is responding to that layer. And there is no Rook management code in the data path. It's just Ceph at the data layer. So you really get performance, high performance storage at that layer. All right, so let's look at some pictures of what these layers look like. So for the first layer with Rook, so this is a picture trying to show that, okay, we've got a single cluster with three nodes. So each of these black boxes is a node in your Kubernetes cluster. And each of these nodes is running pods that run different types of demons. So in the center, center node here, we've got the Rook operator pod. And of course, that that's the management layer that's going to manage everything else. So this operator pod is going to deploy Ceph and the CSI driver depending on what settings and how you tell it to be configured. Okay, so these blue pods are really core Rook pods that are providing the management layer. The discovery pod is discovering what storage is available on each node. So Rook knows how to deploy the Ceph demons. The green pods are CSI drivers. So on each node, you need a CSI driver that will mount the storage. And then the red pods are all the Ceph demons. So there are quite a number of Ceph demons that provide that data layer. The Mons and the OSDs are backed by local storage on the node. So that's very important for Ceph to have that local storage. Anyway, Rook deploys all these pods for you and does that management at the Kubernetes resource layer, creating deployments pod services. Okay, so now at the next layer, we got CSI provisioning. So this is the picture of how your application can request storage from the CSI driver. So if we start on the left here, we've got an application that needs block storage. Okay, so you create persistent volume claim, usually with read, write once access. Okay, so that when you create that claim, the request goes to the storage class, which is which uses fRBD that Rook sets up for you. And then the CSI driver will mount that that's fRBD storage into your application pod. Something very similar happens for shared file system, where you've got two applications who need to share the file system. So they both have their persistent volume claims. And the request goes to the storage class. And then this CSI driver for the shared file system mounts that storage. Okay, now the third type of storage we've got with Ceph is object with the S3 endpoint. Now, this, this is different, but we have exposed it in a way that will be consistent with as consistent as it can be with block and file storage. Okay, so the, if you want object storage, really what you want is a bucket. So you can read, write objects to the bucket. And so we create a bucket claim, which requests that storage from the storage class crafted for object storage. And we have a bucket provisioner then that actually creates a bucket in Ceph, the Ceph object storage and provides that back to the application. All right, so that gives us all three types of storage at the provisioning. Now what does it look like at the data path? You've installed Rook at layer one, you've got the CSI driver with your storage mounted at layer two. Now your application just needs to write and read data to the, to the cluster. So again, your application, it's already got the volume mounted. And there is the Ceph RBD kernel driver then that will take care of writing to the Ceph cluster for you. And that driver will, it knows how to connect to the different Ceph daemons, the Mons and OSDs and manager to make that, that write that data to the cluster. Again, at the file system, shared file system layer connect to the MDS daemon in Ceph, which manages the, you know, the locks and the file system semantics and the S3 client then connects to the RDW endpoint, which then writes the objects into the Ceph cluster. Hopefully we're good there after layer three, then you've set up Rook and now you can write to the cluster. So what does it take to get started? This is, you know, I'm sure it sounds like a complex system, but really to get started with Rook, we try to make it as simple as possible. And installing Ceph has never been this simple, really. There are three steps for installing Ceph through Rook. The first thing is, well, you need to set up authorization or RBAC in Kubernetes. So you say, Kukut will create common.yaml. And next you need to run the operator. And so you create the operator. That gets the Rook operator running, but now the operator, you have to tell it how to deploy Ceph. So now we've got what we call the cluster, Ceph cluster CR. And you can see a picture of that on the right here, where you tell Rook how to deploy Ceph. You tell it, well, what version of Ceph do you want to deploy? And do you want Ceph Octopus or do you want, which is V15, or do you want an earlier version, Ceph Nautilus, which would be V14? And you can choose what do you want, the latest version or a previous version of Ceph. So you can choose when to update the Ceph version. And then some others. There are many, many settings here. We're just showing a few of them. How many mons, how to deploy the storage. That is the basic cluster. Once you've created these three, these three manifests, you will have a basic Ceph cluster up and running in Rook. As a side note, there's also a Helm chart we have that will simplify these first two steps and we're working on Helm charts for our other CRs going forward. So let's assume Rook is set up now with Ceph. What does it take to consume that storage? Just like any other storage platform, first of all, the admin has to create the storage class. After the admin creates the storage class, the application requests that storage with the PVC and then you create your application pod that will mount that volume. So here on the right, we've just got an example pod that shows the PVC that's going to be mounted into this demo web server. And really, if you've done any storage in Kubernetes before, this should be very familiar. It's the same pattern that you used to plug in any other storage provider outside of Rook either. Okay, and now I'll hand off to Sebastian to talk more about key features and everything in Ceph. Okay, thanks Travis. Now let's dive into some of the key features of Rook. So first environments, Rook Ceph is capable of deploying Ceph in terminal environments. So bring your own hardware. If you have your own infrastructure and you want to work on premise, then this scenario is for you. We also support deploying Rook Ceph into the cloud with various cloud providers. So if your entire infrastructure relies on the cloud and everything is running there already, then you can consume storage through Rook inside the cloud environment. But first, you might ask yourself, why would you run Ceph in the cloud? So basically it's all about consistency, because Kubernetes has been adopted by all the major cloud providers today, then it is fairly easy to run any cloud native application as part of any cloud providers available out there. And also there are a bunch of limitations, shortcomings, coming from the cloud providers. For instance, the limitation of PVs you can attach to a given node. Let's assume that we're in a cloud environment and obviously we have supervisors and those supervisors typically have a limitation in a number of disks that can be attached to a given virtual machine. And typically that limit is around 30, which means that if you're looking at providing dynamic provisioning to applications, then you might be limited by the number of VMs times the number of PVs you can attach to it. If you do run Rook in the cloud though, Rook will be running inside virtual machines. And because we will be attaching devices, then we use our own technology to provide dynamic provisioning, then we can scale up to thousands and thousands of PVs instead of that 30 limitation by virtual machine. And not only we bypass that limitation, but also because we are aggregating storage, then we're providing much, much better performance overall. If you were to use a single disk, which is typically attached to a storage class, which essentially represents the flavor of storage, then that flavor would have IOPS as well as throughput limitations. So for a given application, you will be limited to what that flavor can provide. Or if you use RookSeth, because as I said, again, we are aggregating many PVs, then we're providing much, much better performance out of the box. We have configurable cluster topologies. Seth is really amazing as being topology or Seth knows and because it's being told to where it is running and in what kind of topology it is running on. Rook can support that. So it really allows you to deploy a cluster and define what your topology is. So if you run Seth, RookSeth inside a data center and then you have multiple rooms, multiple racks and multiple nodes, then we can easily build such topology by assigning label to nodes. And all of that will be reflected as part of Seth. By doing so, you're increasing data availability and resiliency because you're spreading it across different field domains. And again, those domains can be nodes, racks, rooms, and even across zones between different data centers if you want to stretch your cluster. Seth CSI driver. So from the very beginning, Rook once CSI was released and the spec was released, we started to collaborate with the Seth CSI team to integrate as much as possible. So just like Travis mentioned, we support a large variety of dynamic provisioning modes, RWRWX and ROX, and this for both block and fast system interfaces. Rook, the latest release of Rook 15 ships with the latest version of the Seth CSI driver, which now supports volume expansion and has better feature supports, snapshots and clones. We still do support the Flakes driver, but it has really limited support and we really encourage everybody to switch to using the Seth CSI spec. Upgrading is automated. Typically, upgrades are some kind of a tedious process. And it is really making administrators nervous, especially when it comes to storage. You'll always be wondering, okay, if something goes wrong, will I be losing data? First, this cannot really happen with Seth. And also, one of the good things about Seth is, from the very beginning, it has proven to be super robust at performing upgrades. It's one of the few storage solutions, not to say software most of the time, that can upgrade in a rolling fashion with no downtime and while providing a service. So Seth is already really good at it. And with Rook, we really took that further to the next level by aggregating collecting all the operational knowledge that is needed to do upgrades, and we embedded all of that logic into Rook. So to that Rook, it is really easy as just changing the image in the spec of the deployment file of the operator to a date through the Seth cluster, then it's as easy as changing the Seth image version into the cluster CR. And Rook will handle all the details when it comes to the upgrade internally, going nodes by nodes and making sure they're all recovering and being upgraded and backfitting properly, and then go to the other one. So upgrades made easy, basically. Ex-nucleicester connection. So this is a really popular scenario and we have introduced it in the 1.3 release cycle. And the use case is quite simple because not everything is about green field environments. You already have clusters out there, up and running, which are maybe standalone clusters providing object storage, or you might have Seth clusters providing block for, let's say, OpenStack, for example. You're ready to run applications on Kubernetes, but you're not really ready to move all of that storage because it is difficult and it's challenging to do to Kubernetes. So you want to keep your cluster, which was deployed by whatever tool can be, can be defensible, can be Rook as well, if you want. The goal is to, from your Kubernetes cluster, you deploy Rook and then with a few details, you connect to the external cluster. And then once the connection is established, we're really into this consumer-production relationship where Rook is at this point only consuming the external storage, getting all the connection details and passing them to CSI so that we can provide persistent storage to containers. But that is the only thing we do. We don't get into the business of managing nodes and things like this when we are in external mode. So we just consume the storage that is available to us. ObjectPocket provisioning. So just like Travis mentioned earlier, when we were showing the topology of the storage interfaces are supported by RookSeth, we mentioned the objectPocket claim. And again, this is quite simple. As a user, the only thing I care about is because I'm developing NS3 compliant application. I just want to have a bucket with a few policies on that bucket. And I want to be able to consume that bucket, storing data and retrieving data. And that's all I care about. So in a similar fashion of claiming for block or file, then I want to claim for a bucket. And for that, we have embedded for quite a while now the lib bucket provisioning. But because we really wanted to have it in a more native slash native Kubernetes way, there was an effort upstream to implement a cozy interface, which is similar to CSI but for container object. And it is really the CSI equivalent of object. So the CAP, the Kubernetes announcement proposal, was merged upstream. And now we will be able to, vendors will be able to integrate their object solution through a native Kubernetes interface, which is really great. So we just released Rook15 and we're super excited because we have a bunch of features and it's probably one of the most feature rich release we have ever delivered. So first is encryption with KMS support. During the one full cycle, we introduced encryption for OSDs on PVCs and we were storing encryption keys into Kubernetes secrets. So we all know that Kubernetes secrets are not so secret and secure because they're just merely a base 64 hash value of your value. To move that to the next level, we have introduced support for KMS. And the first one we have introduced is hash equal volt, which is really popular. And now we not only deploy encrypted OSDs, but we store those secret keys inside volt. And volt as a key management service will manage all the keys for us. And in this really secure fashion, in a really secure environment, all the connections to volt are TLS encrypted. And today we only support the token base authentication. In the future, we are planning on supporting the Kubernetes native authentication and obviously much more KMS as well. Mirroring of block data. This one is really interesting because we have been having support for bootstrapping RBD mirror demons for a while, but we never got into the business of configuring mirroring between two sites automatically. And now it's possible with Rook 1.5. Just to get a step back a little bit, step by design is strongly consistent. Meaning that whenever a client writes a data, then it has to wait for all the replicas to be written so that the writing is acknowledged. And because to that, it makes difficult to stretch stuff between regions, for example, because of the latencies. And that won't be really practical. So as a solution for that, the Rook team has the RBD mirror demon so that we can replicate block devices asynchronously between clusters. Now you can have two Kubernetes environments and connect each other. And all the block devices, so the PVCs will be replicated. Obviously this works along with steps TSI and work is still moving ahead to improve that support. So now we do support automatic configuration of peers. And where peer is basically a site. Last but not least, the stretch Kubernetes cluster, which is essentially one cluster that is stretched, one Kubernetes cluster that is stretched across different zones. So the use case typically is that I have two data centers, but I only have two. I don't have three data centers, which will be really ideal. But I only have two. And because SAP is a core base, election base, we need to have some core resolution when we do certain operations in order to maintain the stability of the cluster. So if we had three sites, then it would be really easy. We would have each month on one side and done because I only have two. And my first site is really limited. It can be a VM running on the cloud or, or it can be in the admin desk, something like this. We have to have a solution for SAP to work a bit smoothly with latencies, with high latencies basically. And we implemented this new mode in MOOC. So now you can have a stretched Kubernetes cluster and an arbiter zone, which will only contain one month to resolve quorum and act as a member of the quorum and perform elections and pursue decision making. All of the other zones will have storage available. And I guess that's it for today. So it is really easy to get involved as soon as you reach group.io, then you will have links to everywhere, basically, the doc, the Slack channel and how to contribute, obviously, on GitHub. Thank you for your attention. Thanks for being here today. It was a pleasure talking with everybody. And hopefully we will see everybody in person next time. Have a good day and stay safe. Bye-bye. Yes, thank you. It's been great to be with you. Hopefully you've learned a little bit about Rook and we hope to see you in the Rook Slack or other community. Thanks.