 Okay, hello everyone So today we're gonna talk about storage for containers in containers or introduction to Rook. My name is Zuhara Söder. I am I work at core technologies, I'm cloud storage engineer there and Like I have been a contributor to rock and set projects like I graduated with like the green distributed systems and Like then started working on this F project and now I'm working on Rook. Yeah, we can connect on the then the slides will be online You can like check them out. So We're here to talk about storage. Everyone needs storage all the applications online So you need fast reliable storage not floppy disks, of course, it needs to be reliable performance for your cloud native applications Sometimes they need huge amount of storage. So Comes the need for distributed storage now your Cloud native applications likely live on Kubernetes Kubernetes easily manages distributed apps. You can run Like practically anything on there and storage should be just as easy so your Which means that if your application like fails over like the storage should follow the application If you're if your pod just moves from a node to another one It should also follow follow that and like across nodes and partitions and still be able to work Now Kubernetes unfortunately doesn't have a built-in distributed storage solution So your options would be like having an external storage like standard storage device And then you have to manage it and find a way to connect it to Kubernetes You can use a cloud provider like AWS But then you might be logged in to a certain vendor or like what if you don't want to Connect to external cloud you want to manage everything on your own Another option would be to run it on Kubernetes, but can you actually handle that and we'll see how this works here, so For example, if you wanted to use stuff as your data layer or any other like storage provider You want to connect it to Your your application Through Kubernetes. So how does Kubernetes connect to storage? It does that by the container Storage interface or the CSI. It's basically one interface that storage back ends have to implement and like container orchestrators implement to and yeah, like users can consume the storage via a Storage class and persistent volume claims and like basically everything Connects together and it works. So the picture would look Something like this So yeah like for SAF there is an already existing project called SAF CSI and This connects your Kubernetes applications to SAF, but why stop there why not also manage SAF in Kubernetes. So this is where Rook comes in So what's Rook? Rook is an option for easier storage in Kubernetes. It's basically a SAF operator would come back to what operator means a little bit later and it also contains custom resource definitions and like when you create your custom resources you can like basically manage and automate the deployment configuration and upgrades of your SAF cluster. Now for from a user standpoint, you would be consuming storage via the CSI as I said before It will be like available via storage classes or and persistent volume claims So Rook allows you to manage local or external SAF There is like we'll come back to this later but basically like you can have SAF on your own cluster and be able to manage it and It's like in the same Kubernetes cluster. It can be an external Kubernetes cluster Rook lives on your own cluster and still be able to manage everything and Of course, it's open source. It's based on the Apache 2 license So I said operator. Let's come back to this Who here knows what operators mean? Okay, we have a few So operators are basically like a maestro or orchestrator like you have a script and you manage everyone Based on what that script is. So the operator pattern basically Starts with the observed step you discover what the current actual state of the cluster is and then Analyze it to determine what differences there are from the desired state so you have configuration current state and at the end you you act trying to reconcile the current state with the desired state and Like this keeps running in a loop. So the operator is always there anytime you change any configuration or anything changes in the cluster state like your basically operator will know and act accordingly so a little bit about the Rook project Version 11.11.7 was released recently. It has over 10,000 GitHub stars according to operator hub Sorry Docker hub Statistics it has 280 million downloads. There are over 430 contributors and it's graduated from CNCF cloud native Foundation Like three years ago. So it's it's pretty stable So how you want to get started with Rook first like installing It's available via helm. It's just like one liner and you have you have Rook ready it had like the helm chart you can change anything in the values, but like the values are like Decent enough that you don't need to change much. Well, of course, there are also example manifests in the In the Rook repository for all sorts of different Configurations from like running on a small mini cube That's one node for like testing and development to any number of nodes. You can you can run There is a quick start guide like the links are clicky. I like trust me. They're in the slides later and Yeah, a bunch of colleagues of mine Did a multi node cluster installed that installs in about 12 minutes. There is a link on YouTube. You can check that out we want to do like a Tiny dip in the yaml for you. So this is the the CRD or actually the custom resource for the self cluster one of the Properties here is the set version So this allows you to like if you have if you want to upgrade to set to a new version of self All you need to do is just change the image and everything is done automatically not quite but like the Like the link here explains all the all the pitfalls and all the troubleshooting steps You need to do but it's almost as simple as this and like at core I'm working currently on like doing automatic upgrades that discovers when a new version is there and Those automatic upgrades for you. So that's something we're working on a bunch more into the self cluster spec You can specify the number of nodes. Sorry the number of monitoring nodes manager So these are default numbers you can increase or decrease as you want, of course, we don't recommend decreasing and What else is there like you can like change allow multiple bin per node So that's like you don't have them on the same node if they're like, you know distributed I can also define affinities and tolerations. These are concepts in Kubernetes that allow you to decide where like each Pod lives in your cluster. So, yeah, like you have all the tools available when covering in Kubernetes And of course the dashboard you can enable it. You can I'll show you how it looks in a little bit Of course, like you can also have fine-grained configuration So for this node of this particular IP, I want only this certain device and Like this other device but in the default case like when it comes out of the box It tries to use all the nodes and all the devices which is like good for development But if you have a production cluster, you might want to be a bit more verbose in your definitions So, yeah, like you can even specify configuration per per device So, yeah, very fine-grained Now in the previous talk we talked about how Seth has three modes of operation or ways to access it You can create a block storage device here Like one thing to note is that you can change the size the number of like replicas this has and yeah A bunch of other options. You can also create a CFFS so You can access it by the name and then later on in your application you create a a persistent Volume claim for that certain name and then you just specify the size and you have like a file system ready And of course, you can also change the replication factor. Same thing goes for object storage. So rgw you can specify all the necessary Configurations so in the end You will get this if you run kubectl get Pots you get all these parts each for like MDS and manager and OSD's and all of this To show them a little bit better. This is how the look looks like on a class on your cluster So the big boxes are kind of nodes. So there is one pod for the operator a bunch of others for like a discovery service if you have it enabled you can have Monitoring like yeah, so there are all the months and OSD's and all of that stuff is there The green ones are for CSI. So this this part of stuff connects you connects the the Ceph cluster to Kubernetes and allows you to claim volumes and all of that. So How about monitoring? as I said before Ceph manager has a dashboard. It's pretty decent but better yet you can enable Prometheus and Use Grafana to get beautiful dashboards. So yeah, that's an optional and it's like just as easy as putting one liner and It's it's ready. How about like troubleshooting If you have any issues with Ceph There is an extensive troubleshooting guide that lists like many problems in Rook. Some of them might be just Ceph stuff But you also have a crew plugin crew is like a way to add external features to CubeCuttle essentially you install it with this command it gives you Like visibility into the cluster you can have like for example CubeCuttle RookSeph health like gives you all the health and the status of your cluster and In addition to this it gives you access to many Common maintenance operations that you will have to do in and Ceph. Of course, there is the slack if you want Like you can ask any questions on slack and people will will help you with that quick water break So what's the future of Ceph? What's the future of? Rook One feature where I'm personally also excited for it is the cozy or It's the container object storage interface It's like similar to CSI as we explained before but it's for object storage. So it gives you Like both can request and access buckets as they need or blob storage if you're like you want to use the Azure terminology this has been available in Kubernetes and Alpha since 1.25 it makes Cloud it makes object storage cloud agnostic so that like you can easily access it similar to how currently block and file storage is agnostic be the design is in progress and Version 1.12, which is coming sometime next month. Hopefully Will be will add support for for cozy a Bunch of other things the crew plugin will be Rewritten in go lang it was before in bash and this will extend Like features and all of that stuff We want to add a bunch of observability Commands and some commands for for disaster recovery Basically distilling Ceph expertise inside a plugin there is also some stuff for improving security and improving encryption for Ceph s and OS these this is like they're they're currently using the encrypts We want to basically be able to manage this in inside stuff. So, yeah, there's like the roadmap Link here if you want to check out a little bit more So, yeah, so this concludes my introduction on look pretty quick, but if you want Like any support regarding Rook at core. We're building a core storage distribution that's based on Rook and Ceph and we are making which working to make like container storage easier. There's a free trial. It's basically similar to Very similar to Rook, but it gives you extra support. So if you need any support, you can get in touch with us at score and After the presentation, I have a bunch of stickers. I noticed the lack of stickers outside So if you want any stickers, you can get in touch with me. And yeah, thank you so much Here are some clicky links for the Rook project All right, I'm open to any questions. Oh, sorry. Yes I can repeat the question if you say to me for now Okay, so the question is if you if I am deploying Ceph on better metal Is there value of using Rook or if using containers to connect Like basically using that it depends it depends on if you're using Kubernetes setting up Kubernetes It is like a little bit tricky. So if you're using Kubernetes in your applications, then yeah, it might be worth Like using Rook or at least using the CSI, Ceph CSI to connect to that now you can use Rook to Like manage external clusters, so Like even if it's running on bare metal, you can use Rook and it all it does is Set up the CSI for you and if there are any configuration changes you want to do You can connect to that. So Yeah, this is an option for you But of course as I said, like if you have already have Kubernetes Then Rook would be great if you don't have Kubernetes then maybe not the best option there Okay, any more questions from the troubleshooting point of view. Okay So I want to go back a little bit. So here Could have included a bit more examples, but basically The question was what what differences are in like troubleshooting Ceph in on its own versus Ceph with with Rook So I don't have lots of experience on it But what I can say is that like your experience with Ceph still applies like you can there are Still some issues you might have in Ceph, but like one thing There are certain parts of Rook that managed certain issues that Are in our in Ceph that like, you know relieves you from having to do this troubleshooting There is also the aspect like there is like some extra Layer that you're adding so there is like some troubleshooting stuff related to Kubernetes and whether you're like using things right, but yeah, essentially it still applies but Rook makes troubleshooting easier in many situations like there are like the crew plugin has Has many maintenance operations like ready for you If you are for example one one example comes to mind if you're if you're one of your STs fails And you want to manually change the like the device or change the OSDs. There is like like a crew Like command that you can use and like it basically restores everything for you So, yeah, like the your existing knowledge is still valuable But there are some stuff that are managed for you some stuff that are extra So and of course take a look at the troubleshooting guide and see if I think all right, I Am surprisingly faster than I expected so if Any questions you have feel free to reach out to me now or at a later time? Otherwise, I'll give you those minutes back. Okay. Thank you so much and yeah, have a good rest of your day