 Thank you for joining this 15 minute lightning talk where we're going to be talking about Cubester. I'm Mark O'Cade. I'm a senior technologist at Custom by Veeam and Cubester gives us the ability to identify our Kubernetes storage, validate our storage options, are configured correctly, as well as evaluate those storage options using a benchmarking tool such as FIO. Let's get straight into the demo. So to start off with the demo, I want to just briefly give you two or one area really where we should really focus on as to where to get more information and that would be cubester.io. So as you can see here is it gives a bit of a brief description of what Cubester is and how it can help. It gives you those three, identify, validate and evaluate and if there's three things I want you to walk away with today is those three things. It gives you a very short demo of how it works but obviously this demo video will give you a lot more insight into some of the areas that also works. Also gives you a breakdown of how to use Cubester as a tool and I'll get to the platforms that Cubester is available for shortly but then it gives you a bit of a rundown on how to run certain tasks on there but again we're going to touch on that during this demo. So in terms of where can we run Cubester, so if we head on over here you can see that we have the ability to deploy Cubester on Windows, Linux as well as macOS. So pretty much multi-platform enabling us to wherever our IDE is allow us to run this to ensure that we can test and validate our Kubernetes storage. Okay so in terms of what it looks like, so to begin with if we take a look at the help command of the cubester.exe you can see here that we have a very simple command line driven tool in that we have the ability to run a CSI check which we're going to get through as well as an FIO. Now on its own and that's what we're going to do here this command is going to identify all of the available storage that storage and storage classes that we have available within our Kubernetes environment. So and what this will do is this will leverage the the cube CTL config or the context that you're using to be able to do that and I'm going to jump between two as part of this demo as well. So here we can see that I'm obviously using an AWS EKS cluster but you can see that it does a validation on the version of Kubernetes, it does a role-based access control check and it does an aggregated layer check. It gives you a warning about the CSI driver beer is depreciated and unavailable after after a specific date but that's really just that warning that we see within the public cloud when we know that there's a transition coming around CSI which is one of the reasons why this tool is such a handy little tool is that the the enhance and advancement of CSI really does shine a light on what we need to make sure that it's configured correctly. So it gives you some information about the the CSI driver that you have so we're using the AWS EBS CSI driver and you can see here that I have a storage class known as EBS-SC and a volume snapshot class CSI AWS BSC you can also see that I have a clone here as well and it then gives us some instructions as to well what do you want to run so we've got the the ability to validate which is testing that CSI snapshot restore functionality which we're going to do shortly and we have the ability to perform that FIO that flexible IO test against the storage that we wish to do. Now what we've also got is that entry provider which also may be something that you also want to get the benchmarking results from and you can see here that we have a GB2 storage class available for that but on this instance we can't run the CSI snapshot validation because well that it's not there so the FIO test is available still in there so if we drill down a little bit further and we check out the help for the CSI check you can see that there's a few more options that come up or a few more flags that we're allowed to to start playing with in that we can say to clean up clean up the objects and the default is true anyway so what this is going to do is this is going to go and create a pod an application it's going to create a PVC it's going to create a PV it's going to create a snapshot of that PV and then it's going to restore that back or it's going to clone that back so validating that the snapshot functionality of CSI is actually working as it should be now if we didn't want to clean that up maybe we were running some tests or wanting to contribute to Cubester maybe we want to keep those those pods up and running so that we could dive into them and and do some more work with them we can also dictate where we want the pod to be running by default it will run in your default namespace that might not be applicable we might want to run it in a specific namespace so we've got that option then a required flag would be the storage class as well as the volume snapshot class so let's go ahead and pick those two that we just saw from running Cubester and what this is going to do is run through that process so it's going to create the application which is going to create the pod that we first mentioned and if I open up another window here just over to one side and I run a cube CTL get pods you're going to see that this Cubester pod is container creating and it really doesn't take very long at all and you can see already that that's running if I also run a cube CTL get pvc you can see that we've got our Cubester pvc in there as well we're then going to take that snapshot so again we could do a volume snapshot and you can see there that we're just keeping up with what we're what we're doing here and what this is then going to do is create that clone or restore that data back into the namespace simulating that the CSI is configured correctly now what it will do and eventually this will this will complete it doesn't take very long at all but what this will do is validate and give you an okay or a thumbs up now if your CSI is not configured correctly maybe you don't have the snapshot controller configured quite right or within the correct namespace is one of the areas that I've seen especially within AWS and EBS and the CSI driver that you manually have to implement is it's going to come back and give you some information about what is happening why it's not working at that point the Cubester pod and the pvc will still be in place as well for further troubleshooting alongside that so you can see here that we've created the snapshot and now we're restoring that application and you can see there that we cleaned up the resources so if I now just go up and check that volume snapshot we can see that that's been removed we should also see that the pods have also that they're being terminated as well as then finally on the on back on here you can see that the CSI checker test is that the application successfully snapshotted and restored and that's really the key key to success from a validation point of view to say everything is everything is good from that angle so then like I mentioned this is around being able to test against not just day zero validating your storage options understand identifying your storage options but also being able to be a useful tool on your existing clusters and you may have clusters in various different locations so in this instance I'm going to quickly just jump over to my Microsoft aks so Azure Kubernetes service cluster and I'm going to run the same test against my Azure disk CSI and hopefully that is still up and running but again what this is going to do is is going to go and create that it's going to go and create that pod it's going to go and create that that same that same process is going to happen here and if I go okay so we're definitely on our AKS cluster if I then run get pods we should see our cubes to container creating we should see the restoring of the application happening now you can see the clone there in this instance so this is the PVC that has been cloned from the from the snapshot up and the restores is ongoing and happening within there and you can see there that that was super quick because I'm using different nodes I'm using different storage and compute nodes within my AKS cluster with much more memory much more CPU which does actually affect the storage capabilities that you have within your within your cluster so we could also run that against the Azure file CSI as well but this is really to highlight that this is around I validating that your CSI driver your snapshot and restore functionality or the snapshot functionality from the CSI driver is configured correctly and fit for purpose from a data management point of view but you can hopefully see the advantages of being ever to see or validate that your configuration is your storage is configured correctly so then we get on to the evaluation of the storage so let's jump back into our AWS cluster and clear that down so we can see so now we want to focus on running flexible IO now flexible IO has been around for quite a number of years open source gives us the ability to benchmark our storage systems so here you can see that we've got our options our flags that we can use again namespace size string etc so if we just want to run a basic fio test using the default which is a 4k random read or a random write as well as a random read write 128k block size then we can just just get going there and that what this is going to do in the same vein is going to go and create another pod or another application but you can see here that it's now that fio pod and this is going to take about a minute to to run through and run that scan against the storage ebs-sc so it's running that fio test that i mentioned the default fio on storage class ebs-sc with a pvc size of a default of 100 gig now you'll see in later commands that i can run that against a 400 gig or bring your own or pick your own size that you need for your workload and that's really the key here is that i'm demonstrating something against some storage that i really don't know what the workload looks like but you will be looking for well what does my database need to look like in terms of performance with x amount of users so you can then choose what that looks like from a from an fio perspective and what you can also do is obviously run this from a day zero point of view so that you know that the storage that you're put putting this on is performant enough both the storage but also the compute and then also being able to run that against um like day two operations and make sure that the storage that you've already purchased or the ones that you're already consuming is also configured correctly so you can see here that just from a random write and this this is how it how it outputs that fio results so we can see that for a random read iops point of view with a 4k block size that we get an iops value of 461 for writes it's 507 then you get to see that block size change to one 128k and you start to see the the the iops decrease actually but obviously gives you an idea of what that looks like so we could go through that and it would run a similar and potentially give us more iops against that we can obviously jump back into our azure and run the same test against that the other interesting piece here is that we can also jump into the preconfigured or community led fio configuration files and use those as our as our our testbed so if there's one that you generally use for your application or you've created your own for your application where you can bring your own minus f bring your own bring your own file and here you can see that on that it just walks the walks the the file system backwards reading the file so and then gives you the the output to that cube step is a handy little tool that enables you to explore your kubernetes storage options by being able to identify the storage options that you have available within your cluster both from a day zero point of view and onwards it validates that the storage options are configured correctly especially around data management tasks such as snapshot and restore and then it also allows you to get some performance benchmarking in the way of evaluating that storage using flexible io hope that was useful thanks for watching