 All right. Welcome everybody to the Rook presentation on storage for Kubernetes. We're happy to be with you here today. I am Travis Nielsen, Rook maintainer, one of the original creators of Rook. I work for Red Hat. I'm happy to be here with you today with Alexander. Hi everyone. I'm Alexander working for Co-TechnologySync. I'm also maintainer of the Rook project and yeah, we're going to start diving into what Rook is, storage for Kubernetes. And yeah, I think we don't necessarily want our customers data stored on like floppy disk anymore. So why don't we go a step further to the future? Yeah. We have hard drives. We have SSDs either by fancy PCI connections or even then NVMe SSDs, flash-based memory, like it's all much, much faster than it was some time ago. And it's not just faster, but it's also so much more capacity that we have with these new storage types and quotes that we have nowadays. And because of that, on how we already started rethinking how we deploy applications with containers, with Kubernetes and all the cool new projects around, especially at the CNCF, we have to kind of also move this storage aspect into the new cloud-native dimension. And as what you already kind of like the typical Kubernetes setup have with like at your shipping boat, you know, that's kind of the analogy here. And you have your well, small kind of management boat, let's call it a management boat, which just takes care of all everything related to Kubernetes from setting up like the basic monitoring stack or to even just deploy your application in the end, like the automation, CI CD, and that's all great and all. But for example, for storage, if you're, especially if you're not in the cloud or in like a, where you can just say, hey, give me storage, give me a storage while you inform my application. So basically a persistence layer of some sorts for your application. You're kind of having a bad time with having storage for your applications. That's where Rook is coming in. So if we kind of see as like we have our applications, our containers running in well, Kubernetes, keeping it open there, like there's other like just also a bunch of distributions of the Kubernetes out there and you have your storage. So far, even if we did not look at SAF, I know I'm already mentioning some dropping some names here, but even if you just take SAF here as an example, there's always kind of like this gap between how do you plug your storage into Kubernetes. That's one of the great things that we have to mention before we further dive into what Rook is and all, because it's one of the key aspects in the end for you to be able to use storage in form of Kubernetes native objects, like persistent volume claims and like storage classes and so on. So what we have, thanks to the community around Kubernetes, the great ecosystem with Kubernetes, the container storage interface also called CSI and a nutshell CSI is basically one interface for the storage backend. So for example, SAF and like the container orchestrator, especially again, container orchestrators to implement that a user can easily get a volume of storage for the application. Like if I'm deploying, I know we can argue if that's the best case, but if you want to deploy a MySQL database or like a WordPress as well, for example, you need some storage, you need some storage depending on how you deploy it for like, for example, well, the database files or like the PHP files for the WordPress block. For all that, this CSI interface makes it quite easy for applications containers to at least from Kubernetes. I just say, Hey, I need 50 gigs of persistent of storage of a volume. It's basically you're claiming from Kubernetes perspective, you're creating a persistent volume claim. And that part then talks with the storage in this case, in this example, we still go on SAF to say, Hey, please create a volume and with this and this amount of storage. And that is also one of the points where Rook is coming into. Rook is basically a SAF operator for Kubernetes. It takes care of automating the deployment, bootstrapping, configuration and upgrading of a SAF cluster. And the other components of such a SAF cluster will be going into what a SAF cluster is in a second. So right now, just bear with me. And your operator will also take care of creating everything needed for storage for PVC is for persistent volume claims to be consumed or like to be able to be even provisioned. If we take a look at SAF itself there as a storage platform, we don't just have one type of storage, we have several types that kind of like the three most common types of storage, which is, which are block storage, shared file system, and object storage as free and like the most common commonly used API to talk with object storage. So SAF can do all three. And thanks to what SAF can do, taking that to the Rook project with one of the latest releases of 1.10, latest minor releases, 1.10, with over 10,000 GitHub stars, so many container image downloads, so many more contributors on the GitHub project and the project itself being CNCF graduated. We are happy to help there with the operator for SAF, for anyone that doesn't know about what an operator does. So like an operator basically starts at the top there at observing the operator watches, either like a custom resource, a custom object or something. Like if there are changes, it's analyzes those changes and acts upon that. We can kind of draw like a line with like a conductor for like an operator or an orchestra, where a conductor will on the one hand say you need to play quiet or something, but the conductor will also, if it's still too loud, again, observe or well hear it in that case, analyze or slash understand it and react by hey, quieter, quieter please or even louder, louder basically. And it observes a state and makes sure that it's always applied. It is always this desired state. Let's talk about a bit like how Rook accomplishes this in its architecture, how it looks for example when it's run on a Kubernetes cluster. This diagram is probably going to look quite overwhelming for anyone that hasn't worked with SAF, but the main takeaways here are that with these three different types of storage and that's again block storage, file system, files and people just call it file, I prefer calling file system storage and object storage is all in one SAF cluster. It's all possible with just one SAF cluster. The main point is that this complex system from, again, deployment, configuration, upgrading, management, and even the components for example like an object storage, if you even want to use objects that you don't have to, if you want to have file system or block storage, that that is what Rook, the Rook SAF operator is taking care of, observing the state that you want, and taking care to ensure that it's achieved. Again, with CSI being mentioned so much already, we have ways to easily consume the storage from for block storage using the SAF CSI RBD driver, Rados block device. Let's just keep it at that. That's for block storage, for file system storage, which is the SAF CSI self-FS driver basically, which is, well, taking care of managing the volumes for file system storage and provisioning them so you don't have to do really too much. It's not with CSI, but it's worth that it's mentioned here for object storage, for applications to easily go, oh, hey, Kubernetes API, or just in general for, you know, thinking about like everything kind of a service, like, hey, I need one object storage bucket, and through also the bucket provision now, there's some logic which is implemented also from Rook side, that it's easy for an application to get an object storage bucket with a correct, with a pair of credentials for the application to already immediately start talking with an S3 storage with the bucket and storage data there. Going into a bit more detail than the first diagram we've just seen of like the architecture structure of how it would look if you have a SAF slash Rook SAF cluster running, all these components in the end are taken care of by the Rook operator through the means of the Kubernetes API so that if anything happens, all the operator does in the end is use the Kubernetes API to make sure that the end state is achieved. And the Rook operator then, for example, also talking obviously with the SAF API and create certain resources, pools, block storage pools, file systems, and configure them there accordingly. As an example for like this desired state that we are some examples for that, we have a custom object. I've hopefully previously mentioned it already a bit. Custom objects is also one of the things an operator in Kubernetes can watch on. And for example, the Rook SAF operator brings a custom resource called SAF cluster. It's not all of them, it's just one that we take as an example here. Even with this small example, we have just a block of YAML basically, which says, hey, I want a SAF cluster with this and this version, this and this certain configuration for like their data is stored and so on. And there's even more options that can be set to, for example, say, which disks should be used in which server, or even if all servers should be used, all devices, all disks in the servers. And this is exactly the desired state, which we are talking about in the end. We have the operator watching these resources. And when you create such a resource, or even if it's updated in any way, the operator sees them for example, oh, there's no Rook SAF cluster yet. So it would go ahead and start creating everything needed to run a SAF cluster in Kubernetes and takes care of starting the monitor pods, the monpods, SAF mons, the SAF manager, SAF OSD. And if you wanted to have like option services as the RJW component for file system, there's the MDS component and all those things, this is what the operator takes care of, takes care of talking with SAF, configuring everything, setting the correct setting parameters and making sure that that is all in sync based on the YAML block we applied to the Kubernetes API as an object. With that said, Kubernetes and SAF in the end is great. It's even without Kubernetes, it's great, wrong set folks. But even combined with Kubernetes, it's making everything even greater because with Kubernetes kind of being this abstraction to some degree of a hardware or even your cloud, this whole API ecosystem that Kubernetes has enables the Rook operator to easily run a complex storage system like SAF. And now Travis will walk you through some more SAF storage related features and what features we have worked on in the latest releases. Thanks Alexander for taking us through the introduction to Rook, what it is, why we're doing it, and how it's useful to you. We love Rook, we love providing this platform so you can have storage in your Kubernetes clusters. So let's talk in more details now, dive into where is Rook more useful, what features do we have, why would you use it? First of all, how do you deploy it? So you've seen with Alexander's examples already that there are some YAML files, you can go create those, specify your desired state. I'll also point out that there are two Helen charts that we have, one chart will install the operator and all the related resources. And the second chart will install all the Rook's CRs. So you can create a Rook cluster, you can create the block pools, the SAF file systems, the SAF object storage. All of those types of resources for SAF have these different custom resources. So that's how you basically deploy the cluster. Now there's lots of ways to configure Rook, Alexander showed just a few of those different options, but really I want to point out you can deploy across multiple cluster topologies. You can deploy it where there's only hosts or multiple hosts, you can deploy in a zone topology or there's a number of other topologies where you can define racks or data centers or however your topology is laid out. And then SAF will make sure that the data is replicated across the cluster in a way that keeps the data safe, reduces the risk of data loss. As one zone goes down, the other zones, for example, would still hold the data even if one zone goes down. And this helps SAF keep the data highly available. And again, you can customize it whatever your topology may be. Now most people do run SAF, I would say, in a bare metal environment, or that's where the main scenario was when we started on the Rook project because there was no other solution for on-prem storage to really interact or be installed with Kubernetes. There are also scenarios that are common for running SAF in a cloud environment where, first of all, you want a consistent storage platform wherever Kubernetes runs. Why should you have to use one platform in one cloud environment or in bare metal? Like let's just have one storage platform that works consistently. Second, there are shortcomings that cloud providers have had from the beginning and they still have, like being able to access storage across AZs or they have slow fail-over times because you want seconds of fail-over time instead of minutes. You want to be able to have a large number of PVs, perhaps, if you have many small volumes or need small volumes for applications. Some storage providers may have a limit of 30 per node where with Rook, basically there is no limit. You can have, say, 1,000 PVs per node, but there is no real limit. It just comes down to resources. There are also any characteristics of large volumes that you want, where you provision large volumes in the cloud provider and then SAF allows you to have smaller volumes on top of those larger volumes and your applications don't know the difference. Finally, SAF monitors and OSDs, they will run in a cloud environment backed by PVCs, but there's no need for direct access to local devices. It just works seamlessly in those cloud environments. Next scenario, commonly used. Some people have SAF clusters that are already running outside of Kubernetes and they want to connect their Kubernetes applications to that SAF cluster. That's a perfectly common and valid scenario where you take the SAF information from that existing SAF cluster, the SAF monns, the SAF key ring, the cluster FSID, a few other items and then you import those into the Rook cluster and Rook will then configure, basically Rook configures a CSI driver to connect to that SAF storage to just make it seamlessly work with your Kubernetes applications. That CSI driver, I already mentioned, again, it has many features of standard CSI drivers that you're familiar with. It is very flexible and has many features, so you can support Read-Write once volumes with SFRDB, RBD, SFFS supports Read-Write many volumes. It does snapshots and clones, volume encryption, volume expansion, ephemeral volumes, and all of the features that you would really expect from a storage platform. This is just a few of the features that it has. Next feature, Alexander has already touched on this. If you want object storage with an S3 endpoint, SAF provides that with SFRDW. The way you do this is you define a storage class for your object storage and you say, wait, storage class is usually just for volumes that you touched your pot. Well, the way Rook has implemented this, the storage class allows you to provision a bucket with an object bucket claim. The operator creates this bucket when you create this claim, which is a very similar pattern to PBCs, but it's for object storage that lets you have access to a bucket. A KMS encryption, so if you want your data encrypted at rest, KMS encryption allows you to do this. We support multiple back ends for storing the encrypted keys. KMS hash decor, IBM key protect, KMIP. We're working on adding support for others so that you can ensure that your data is secure and encrypted at rest. Stretch cluster. In some scenarios, you really only have two different data centers or two zones where you have storage available. In this configuration, SAF still needs a third node or a SAF monitor that acts as a tiebreaker between the two. So this third node is needed somewhere else to work with the other two zones, but ultimately, the storage is only found in two of the zones. You've got replicas in each of the two zones to provide greater resiliency. Data mirroring. For any real application that needs to replicate its data, it needs to ensure that that data is also available for applications that need to fail over when some cluster goes down. Say a whole Kubernetes cluster goes down. If you've mirrored your data to another Kubernetes cluster with SAF, as long as you've implemented DR for your application, at that level, you can switch it over. You can make your application switch over its active configuration and run completely from the other cluster because the data is mirrored over between the clusters. All three types of data can be mirrored with SAF. You've got the block and the file system and object with RGW multi-site. All of those can be mirrored using asynchronous application, then it's for enabling DR scenarios. You really want resiliency in your applications for disaster recovery. So let's talk about some of the recent features that we've added, really in the last few months, 1.9 and 1.10. So we've put some effort into our crew plugin. So this is a tool that runs outside the operator. You may be familiar with other crew plugins that are available in the community. So this is one that will help you administer these one-off scenarios in Rook that the operator just doesn't handle directly. So you can do things like print out the status of your cluster. You can show the health of the cluster. If you need to perform really advanced operations on the SAF monz and OSDs, you can start what's called a debug pod. If you need to get rid of an OSD, you can purge it and then other simple things like just restart the operator. So we're continuing to improve upon this plugin and add new features. It's really going to help with disaster recovery scenarios again and things where the operator really just doesn't cover. The operator covers scenarios where it's constantly running and monitoring desired state, but sometimes disaster is happening. You need to be able to just run these one-off actions to get the cluster back into a good state when the operator isn't even capable of running, perhaps. We're excited about that. I hope you can try it out. Give us feedback on what features you'd like to see in the crew plugin. A few months ago, we did a documentation facelift. So we're just happy to have an updated framework with MakeDocs. You can feel it's searchable. We'll continue to add content and if you have any feedback, we'd love to hear your feedback on the documentation as well. As always, Cep is making updates to the data layer and our goal is to continually add support for what Cep is doing. So we've added support this year for Cep Quincy. This came out earlier this year. And then support for Cep Octopus was also dropped by Cep. It reached the end of life and so now we dropped support for that. So the latest releases of Cep Pacific and Quincy are supported in the latest work releases. Some of the CSI improvements that we've made in the 1.10 release with the latest CSI driver. There's cave-in integration for encrypting your volumes, RBD volumes for block storage. There's NFS support for snapshots, restore and clone and volume expansion. There's PV and snapshot metadata. So if you want to know more about your PVs, that the metadata is attached to the PVs, there is shallow read-only support without cloning the underlying snapshot. NFS has been an area of focus for the team as well, adding more support for enterprise NFS features so you can connect generally from the outside of the Kubernetes cluster to the Cep storage inside Kubernetes. For example, adding client ID management via SSSD with LDAP, Kerberos client authentication support. Again, really supported for external clients for now. This is an area of active development. We're looking forward to feedback on how you'd like to use NFS. So a brief thought on road map and where we're going with Rook. Again, our goals are always to really support new Kubernetes features when they come out to make sure Cep is always managed in a very robust and thorough way. So we need to support Cep features when new Cep features come out. We make sure they're exposed and configurable through Rook in a simple way with CRDs. Finally, it's always a goal of ours to respond to community inputs. We want to know what storage needs you have and how we can improve and just make sure your needs are heard. Or as a CNCF graduated project, we really want to make sure we're doing what the community needs to be done. We're here to respond and hope you feel that response as you come to our Slack and GitHub issues and other places for feedback. Some of the features areas we're working on, you may have heard of COSI, the object storage interface for containers. It's going to kind of take the place of OBCs eventually. It'll be a while out because COSI is still so new. It's just an alpha in Kubernetes 1.25. So we'll have an implementation of that soon for Rook. Other areas, encryption. So we do have encryption across the wire, for example, but also other types of encryption and other places to store the encryption keys, depending on the KMS. We've got disaster recovery scenarios and lots more. There's always more to do in the storage realm. So I have to get involved. We hope you can visit the Rook booth at KubeCon while you're here and visit our Rook GitHub, visit our Rook Slack, Twitter, follow us on Twitter. And if you'd like to talk to us and don't find us at the Rook booth, you can join us in our community meetings on a bi-weekly basis. We've got a link to that on the main Rook GitHub. So we hope to meet you, hope to hear from you. And thank you. It's been good to be with you today and thanks, Alexander, for all the info on Rook. Thanks as well. And yeah, as Travis already said, we're going to be at the Rook booth. It should be located at the CNCF open-source pavilion. So if you're at KubeCon and seeing this, be sure to check out the booth. And thanks for listening.