 Hope everyone's doing all right today. Hope we enjoyed our lunches. I personally had one of the boxed ones. It was a wrap of some sort. I believe it was chicken, but I had it in the past hour and before a presentation. It could have been shrimp. It could have been tuna. I couldn't tell you what it was, but my money's on chicken. So it was pretty good, at least from what I remember. So my name's Matt. I'm from Datara. This presentation is going to be on Datara, Cinder, and our Tenancy Integration with Cinder. So a little bit about me. As I said before, my name's Matt. I am a software engineer at Datara. I've worked in the industry for about six years. Previously, before I was working on OpenStack and Ecosystems, I worked on automation frameworks. I was working for Riverbed for a few years and was working on their SteelScript automation framework, which is the primary method by which they automate their Steelhead product. So currently, working for Datara, I am the author and maintainer of the Datara Elastic Data Fabric Cinder driver, which is a mouthful. Also the Docker driver, DCOS integration, Golang SDK, Python SDK, and I swear about half a dozen other jobs. That's sort of startup life. So I'm available on IRC or via email. On IRC, you can typically find me lurking on there most every day. So a little bit about Datara. We were founded in 2013 and were currently located in Sunnyvale, California. It was founded with the purpose of creating an elastic data fabric for private clouds and multi-clouds. We have some Tier 1 investors, some pretty big names out there, Andy and Pradeep. If you're familiar with those. So we have quite a lot of interest from folks who have created very large companies, specifically ones with performance interests. So in April 2016, we exited Stealth Mode and made our product available to the wider market. We have quite the broad data center DNA in our employee structure, people from Cisco UCS, Microsoft Azure, 3PAR, NetApp, EMC, Riverbed, quite a few different places, including the maintainer of LIO, who is our CTO. So about our product, Datara has the elastic data fabric. This is a high-performance, low-latency hybrid storage array. So many of the different features of this array include things like elastic scalability. It is a scale-out product. As you add more nodes to the product, your performance increases. It is policy-based data placement and management. So this is actually a very important part of the product. Almost everything is controlled by a policy of some sort. So say, for instance, you wanted to store your data in one part of our hybrid array versus another part of our hybrid array. That is a policy that you can set for that particular volume or group of volumes. It has a self-adaptive load balancer. So this is essentially a way that we self-optimize our product, not only from the front end and the access of it, but also where your data is placed and where it is accessible from. And actually, one of the things that I'll be covering today is the tendency and resource isolation. So from the very basic level of our product, everything is isolated by tenant. It is actually one of the most basic data structures if you access our product through its API. And so if you are ever accessing it, you are under a tenant, whether that be the root tenant, which has access to everything, or an individual tenant or sub-tenant, which has access only to a specific subset of the resources on the machine. The product is also very, very easily programmable. It has a highly scriptable API. And this is of particular importance, at least to me, because I am at least an API-first consumer. I'm one of the first customers of the API. And if it is not up to snuff, then I'm unhappy. So a little bit more in detail about the product. It is a cloud-independent storage as a service. So you can take our software, you can put it on pretty much any sort of hardware. We do have a sort of a regulated list of hardware that will get you the best performance with the software, but you can put it on pretty much anything. The important part about it is that it has a control plane that it can manage whatever data fabric it needs to underneath it. Currently, what we have available are things like NVMe Flash, SSDs, and HDDs. But it can do even things like managing replicas that are stored in cloud, or current public clouds. Just as easily as it can manage those local HDDs, SSDs, and NVRAM. So safe, for instance, in the future, when NVDem, or sorry, not NVDem, but 3D Crosspoint is widely available and is now this new tiering storage layer that you want. You just simply add a note of that to our product. The control plane will take that over and handle it and then tear to it as it needs to based on the policies that you've set up on the box. So to compliment this, so we have the EDF product itself, but as sort of a necessary compliment in this day and age, we have a global operations portal. Now this is a relatively recent addition to the product itself. We have a very good engineer working on it, and it allows you to access all sorts of metric information about the product that may or may not be currently stored on the product right now. So as the product is running, eventually data can go out of resolution. Things like that in this global operations portal is essentially your way to look at that data as it's been pulled off the product. So if you wanted to say plot or look at your usage of the product over time as your storage uses increases and you want to be able to predict when you need to add a new storage node in order to meet your customer's needs, the global operations portal is the way that you would find that out. So onto the main topic, multi-tenancy. So we have multi-tenancy and OpenStack. So if you are a bit familiar with how multi-tenancy is done in OpenStack, you know that the project concept in OpenStack is the concept of a tenant. And actually if you go back into the code, especially in earlier releases like Juno and Kilo, you will actually find that projects were called tenants and there's still a lot of vestigial things left over from that. So a project in OpenStack can be, it's essentially a way of sharing the same underlying infrastructure well between different use cases. So a project can be allocated more or less of any particular piece of infrastructure than another project. They provide flexibility in segregating resources among use cases. And for an example, business case, you have say engineering group A and that engineering group A is using project A in OpenStack and it has quota A associated with it. Now you have engineering group B, that engineering group B has project B and that's the project that it's using in OpenStack that project has a quota associated with it, quota B. Now let's say that group A has twice the head count of group B. Well, just assume that head count also means productivity and resource needs, though it doesn't always mean that. So in this case, since group A needs twice the resources of group B, you would utilize the project system and the quotas associated with those to allocate twice the amount of resources to group A than you would to group B. And so group A would be able to provision twice that amount than group B just with an OpenStack itself. Now moving on to multi-tenancy and Cinder specifically, Cinder uses this multi-tenancy to essentially set per project quotas, like we mentioned in the previous one in OpenStack, but those quota amounts relate to block storage. So you have your volume allocations that you can create. So the total number of volumes, you have your total number of snapshots, you have your total number of backups that you can create, and then the last one is the per volume size of each of those volumes that you create. And then of course the snapshots and backups are very much related to the size of the volumes that you are snapshotting and backing up. So most operations in Cinder that will eventually reach a backend, reach the backend driver. If you're not familiar with the architecture of Cinder, a call comes into Cinder. Cinder then calls the manager. The manager is managing the drivers on the backend and then the driver communicates with that backend product. And that's how you sort of abstract out the vendor backend. So once the call is passed all the way down to the driver, it actually passes in the project ID with that. And so because we get the project ID, that means that the backend knows which tenant is being used for these requests. And we can sort of make our life a bit easier in the process because we get that information. So moving on to multi-tenancy Cinder and Daterra, this information that you see on screen on the left is actually the Cinder.conf entry for the Daterra driver. If you were to set up Cinder to work with Daterra as a backend, this is about the information that you would place in there. It's essentially pointing the volume driver at it, getting the San IP of the actual backend it would be using, the San login, the San password, and a couple of Daterra specific pieces of information. Now the very important one is there at the bottom. So it's the Daterra tenant ID. So in this case, the important one we're gonna be working with is the one called map. So this is the feature that I put into the Daterra Cinder driver that I'd like to tell you about today. So valid values for this Cinder.conf option are map, some explicit tenant. So just insert an explicit tenant that you've created on the backend, or you can use none, in which case the root tenant would be used. So we have three distinct ways to use it. Now multi-tenancy, if we continue on with this, the strain here, the first one which I find to be the most interesting is automatic project ID to Daterra tenant mapping. Now essentially what it allows you to do is your project ID that you've set up in OpenStack and you've been working under, and so your whole team's working under the same project ID, that project ID will be passed to the backend. The backend then passes it to my driver. The driver goes onto the backend and Daterra sees if there's a tenant matching this, and if there's a tenant matching it, then it grabs that tenant. If there's no tenant matching it, then it automatically creates the tenant. The nice thing about this is that it means you don't have to go to the backend to manage anything. You just create a new project in OpenStack and Daterra automatically understands, oh, you need a new tenant on the backend. Let's go ahead and allocate the resources for that, and everything that you allocate under this new tenant will be under it on the Daterra tenant in the backend, or under the project will be on the Daterra tenant on the backend. Now for the other two, we have the explicit tenant usage in cinder.conf. So this is very useful if, say for instance, you have multiple ecosystems pointing at the same backend. So if you have multiple ecosystems pointing at that backend, you can use different tenants in order to segregate them because you don't want these different ecosystems essentially stepping all over each other. So explicit tenant usage allows you to do that. And then there's also no tenant. So if you have a very simple use case, you have one project in OpenStack or even multiple projects, and you don't really care if they might step on each other in the backend, or if you wanna let OpenStack itself handle it, then you don't have to specify anything. We'll just understand you want the root tenant or whatever the default is for your login. Now, a little bit more about Daterra multi-tenancy. So multi-tenancy, some of the features of Daterra multi-tenancy are that you can have micro-segmentation of users, tenants and applications. You can understand and solve that sort of noisy neighbor isolation through the QOS that you have for the tenant. And you have IOPS bandwidths and other cost controls, like Total Write, Total Read, even latency and controls on the backend under the tenant, as well as IP pool, VLAN tagging and such for network isolation. So here in this image, it gives you an example of having OpenStack, CloudStack and Kubernetes all running against the same Daterra backend cluster. So why do we care about backend tenancy? Because you have tenancy on the front end, why do you care about that backend? So the truth of the matter is, it's kind of weird saying it in an OpenStack presentation, is that not everyone uses OpenStack. There are other use cases for other ecosystems, such as Docker Swarm, Kubernetes, CloudStack, DCOS, all of these different ecosystems out there and even just bare metal, if you use the storage directly and provision via your own framework, you need to be able to have all of them use the same clustered backend because you have a massive amount of resources in this cluster and maybe not any one use case wants to use all of it or should use all of it. Most everybody still needs storage. So tenancy supporting back ends, such as ours, allow for a much smoother operation of multiple ecosystems and non-ecosystem combinations based on the applications to share the storage backend. So we have and hopefully have enough time for this. So here we have the Daterra EDF API. If I have enough time, I will go through a little bit of a demo. The demo is written pretty much directly against our REST API. Using the Python SDK, we do have a Go SDK available as well. The API is tenant aware. Everything that you do within the product is tenant aware. And you have streamable metrics and those metrics are available by tenant. So hopefully we can get through this demo. So let's switch over. I have the Daterra dashboard right here. Let's go ahead and start the Boston demo. All right, so what this is doing is it's setting up a whole bunch of projects. Well, really it's just setting up two projects and I need the watch over here. There we go. So in the other one, while that's setting up, it's going to be watching the the OpenStack client output for projects and for the volumes and servers. So up here in the top, you see projects. The projects that we've newly created are silver and gold. Just random names that I chose. There's nothing meant to be implied by the difference between silver and gold. We create the root project or sorry, a root volume and each of those projects, we set up a security group for it and then we clone that volume and essentially just boot it into an instance. Now once we boot that instance, we attach a data volume to it because usually your root volume is going to be much smaller than any data volume that you want to persistently store on, although technically you could persistently store on the root volume as well. And this is not nearly large enough to show all the information, unfortunately. So we're testing the connections to the servers and now it gives us the prompt. So if we look back at the Datara dashboard, the Datara dashboard is not showing metrics right now, which is great. Oh, there we go. So now the metrics have shown up. So these are actually not, this isn't any traffic that I'm running. This is just the root view of the Datara dashboard. It contains all the different tenants that are running on this cluster at once and it's an aggregate of all of them. And as you can see, the cluster is in use by somebody right now, but we don't want to see their view. We want to see our view. I've created two projects using the Cinder driver and I want to see ours. So we can look here and see which projects they are. So we have silver and gold. Those are the two projects and these are their IDs. These are based off of the UUID of the project. I just go ahead and append a prefix to it and then create the tenant on the backend. We can go and find these on the Datara dashboard. So the silver tenant starts with 6-1-C and the gold tenant starts with 6-3-6. So if we go to tenant management, we see 6-3-6 right there. All right, and let's go ahead and look in there and we see zero traffic in there. So we essentially see an isolated view of just this tenant's resources. Now real quick, before I run out of time, I'm gonna run some traffic and I believe that was the gold one. This is 6-3-6, so let's run traffic gold. It's running against the 6-3-6 project. It's just logging in there and running DD against the data volume. So give it just a moment. Let's check the API. So we're gonna get the gold IOPS and we see IOPS for gold. And you can see that we have IOPS going on, IOPS in bandwidth right there on the front for metrics and it includes only this one. So if we were to, oh, apparently it's gonna decide to clean up, which is perfect because I'm running out of time anyways. But essentially what I was going to show you is that if we look at the, sorry, the metrics for silver, we don't have any because there's no traffic running for silver. We only see it for gold because that's the one that has traffic running. Okay, so going back to this real quick because I have like 10 seconds. So for the future in the booth, we're always trying to improve our tendency support. So if you have any suggestions for features that we should add to it, just please feel free to drop by our booth and give us your thoughts and opinions on it and we'll give you an explanation of some of the more detailed oriented aspects of the data system. And the booth is available right over there next to Easy Stack and in front of Cloudify. So thank you for your time, folks.