 All right good afternoon everybody My name is Rob Daly. I'm a senior systems engineer with Swiss stack. This is Doug salt as you introduce yourself I am Doug salt as I'm a senior solutions manager at Swiss stack and our Swiss stack booth is right over there So check these guys out after the presentation How's everybody doing today? All right pretty good. How's that people doing? So we're here today, we're gonna talk to you a little bit about a racer codes It's just something new within Swift and what we want to do is give you a real quick rundown of Swift and erasure codes and then what Doug is going to do is give you a real live demo of Using erasure codes within a Swift cluster. So What are erasure codes if you don't already know so erasure codes really the concept is to be able to store the same amount of data that you have But use less capacity to protect it right originally the way Swift was written is that we use replicas to Protect our data right to maintain a level of data durability With erasure codes we want to meet that same level of data durability, but use less capacity to do so right so from a Kind of low-level perspective right the way that erasure codes are implemented are just there you know traditional read Solomon algorithm and you can tune how many Segments or fragments you want to use for parity versus how many you use for data? And those are all tunables that you can choose for different ways of storing the data that you have now It's not a silver bullet for every use case that you have and Doug and I are going to talk a little bit about What good use cases are for erasure codes, but again? It's not a silver bullet there are still use cases where you're going to want to use replicas to save and protect and store your data Where it makes more sense to do it that way versus using erasure codes for that data And there are policies and ways to to leverage both within a a swift environment and we're going to show that to you So from a swift perspective of erasure codes, this is something that's been coming for a while Right, this is something that has been worked on for close to about two years now And there's been contributions from a lot of people right Swiss stack has helped lead the way alongside with Intel has made major contributions Red Hat and other members of the community to be able to make this happen and erasure codes I've just been released with the kilo version of Swift version 2.3.0 and Now it's still considered beta and really it's just because it's a new feature within Swift, right? So it's still considered beta, but it's out there It's it's available to use now and we encourage you to start testing it out and and leveraging it for your applications How do I implement a erasure coded policy storage policies were introduced going about a year or so ago and Storage policies allow me to have different types of protection for my data So in other words, I could have one container of data that is replic replica based and I can have another container using a storage policy That's erasure code base and Doug is going to show you how that's done Today that it's somewhat of a pluggable Architecture I can choose different types or different libraries that actually do the erasure coding for me So today what we're going to show you is an implementation of Intel's library that is doing the erasure coding There are other ones out there like J erasure X or that you're able to use to be able to you to Erasure encode your data And then finally we're still using the same mechanisms to place these Erasure coded bits across your cluster right so swifts Guarantee for data is to be able to distribute it widely across the disk space that it has available and do it in an Intelligent manner that it can always be redundant in in the event of a failure of any component and we're still using that same Placement algorithm. We're just doing it a little bit differently using erasure codes So what we're going to show you today is how to create an erasure coded Storage policy right and what we're going to do is use the Swiss stack interface We have a web console within our product and Through our web interface what you can do is you can create a new storage policy and all the things that I was discussing right? How many fragments of actual data do I want to do I want to split my object into how many parity bits Parity fragments do I want to make part of that Part of that protection scheme that I have and Doug's going to go over all this in a little bit more detail So we'll get to what some of some of these variables actually encompass So just to give you a quick rundown of what we've got We've got a Swiss stack controller It's just our software running in a lab environment and that controller is managing well several clusters But the cluster we're going to be showing you has roughly 15 storage nodes with a bunch of disks attached to them Couple storage policies already deployed to them and some proxy nodes in front proxy If you're not familiar with Swift proxy nodes are responsible for routing the requests that come into your back-end storage And they're kind of the intelligence for figuring out where to place data where to retrieve data from and So that's the architecture that we have in the lab today that Doug is going to show you right now. Thank you, Rob All right, so let's hope that the demo gods favor us All right, so I'm going to start off in the Swift stack home site So Swiss stack if you don't already know is a very simple way to get a Swift node up and running It gives you pure open stack Swift, but then on top of that would give you a managed UI We make it extremely easy to deploy to monitor Can you up my volume? Make it easy to deploy into monitor. So I have a Environment and the environment has been built special for Vancouver It is a 16 node cluster 15 storage nodes and so I'm going to go in and manage that I can show you those nodes Again nice and easy through the interface and what I want to do right now is create a new storage policy So as Rob said in the kilo release storage policies came out and storage policies allow me to have multiple rings Guaranteeing different durability for my data So I'm going to look and right now I've got two storage policies implemented One of them is a replicated one that gives me three replicas Another one is currently a racer coded and it's going to give me seven data bits in five parody bits So anytime I send some data to that storage policy It's going to be broken into 12 segments and I only need any seven to reconstruct the data So to show you how easy it is to create a new storage policy We could create a new storage policy and so I'm going to call this one easy 10-4 because I want ten data bits in four parody bits Now I could pick any values I want if I want an 8 by 7 if I want a 19 by 3 and that all really depends on what makes sense for your environment I would highly recommend that you have at least as many nodes as the total amount of pieces that you break your data into Because what you don't want is to be storing multiple pieces on the same node because Swift wants to be as resilient as possible So the first thing I need to do is any drop down to policy type And I'm going to switch that over to a racer coding and as you can see the screen auto adjusts and gives me a few new options I can pick if this is going to be my default storage policy Which I don't want it to be I can pick my partition power partition power And there's a good example down at the bottom of the page tells me how even I'm going to distribute my data in the Swift cluster And so it tells you if I had a partition power of 16 This would be a great cluster until I exceeded 1,966 disk so I think we're going to be just fine with a partition power of 16 As Rob mentioned there are multiple erasure coding libraries, but we're going to pick the one from Intel That's the one that ship standard with Swift stack and it's actually hardware accelerated if you're running the Intel Xeon platform And so again Erasure coding works through CPU cycle there's your trade-off you gain some space But at the expense of using more CPU if you have a proper Intel chipset and you're able to leverage hardware accelerated Erasure coding that's going to be a big benefit and then down here I can pick the amount of data fragments So we're going to leave that as the default 10 because I named this one 10 for and we can pick the number of parity fragments But again, you're totally free to change that to 6 if you so desire and then lastly We're going to pick the amount of data that the proxy node will receive before it starts chunking things up And right now we have that set to a megabyte and that seems to be a very good tunable so nice and easy I'm going to click create storage node And so I've created this new storage node right down here. I'm sorry storage policy with erasure coding Now if I want just to show you a little bit more Swiss stack before I jump into the actually putting demo data into this Cluster you can see all my different nodes if I wanted to manage one of my nodes and assign that to certain disk Storage policy allows you to pick what nodes and what disk get this policy So you can see that right now I have a group that gives certain disk the standard replica policy and my current EC7 policy I can click right here add or remove policies I can take that new 10 by 4 and I can choose to add it immediately to these discs And it's really as simple as that to get those added. Doug. What's the the difference between my mic is on? What's the difference between gradually and Immediately adding that capacity into the storage policy sure so immediate says that I want to consume this immediately it says go ahead and set it up and The only reason you'd want to do immediate is when you first create a storage policy Otherwise if you have a bunch of disk and you've already got data on them And you have data in your storage Policy today your 10 by 4 and then you swap out your disk with From 4 terabyte drives to 8 terabyte drives or you add a bunch of new nodes You're going to want to do that gradually because what Swiss going to do is immediately try to distribute that data as unique as possible So it's going to be moving things around on the back end and gradual is going to make sure that there is no Performance impact to your environment while the data is getting replicated and moved around to ensure that it's even better Protected in case of a hardware failure. That's a much safer means for adding capacity into an already existing Existing cluster it is so normally you should definitely pick the gradual button For us to have it for me with demos picking the immediate button and Then if I wanted to deploy this I could just go right back to to my cluster I could hit manage and I can hit deploy but Instead of having you watch a screen where I deploy this what I'm going to do actually stay in there I'm going to go over to my swift web client So there's multiple ways to look at my object store one of which is to go to any proxy node And I can see the containers that I already have and the accounts that I have set up So if I wanted to create a new container for storing data in I can simply click the plus I can pick different Storage policies have defined our 10 4 isn't there because I didn't hit the deploy button on the last screen And I can pick a standard one and I can call this you know documents And hit okay, and I'm going to have a new container now. Obviously. This is a pretty rudimentary console There are other consoles out there that are available. So for free you can download cyberduck, which is a open-source System and so let's refresh this hope my VPN tunnel didn't and what I think is really cool about this right is what Doug Showing you there's different ways of accessing this But what's really interesting is where Doug has signed in and authenticated into the cluster? He's got right now two different Containers right in terms of the application or the users that are touching those they all look the same on the back end They're being used very differently, right? I have one that's erasure coded and is playing out the data over many disks into many different fragments And then I have another Container that's just using a replica policy But again all within my account space of the cluster so I can pick and choose again. I think that's a powerful feature So my cyberduck token obviously has expired but instead of re-authenticating there I've opened up cloudberry and you can see I have my new documents container And if I right-click on it and I look at the HTTP headers We can see that this one's protected by standard replication Whereas if I look at my EC data container that I created and I look at the headers I can see that this one's protected by my EC 75 storage policy and of course I can create as many containers as I want In fact, there's a great use case for creating very many containers and try instead of trying to put billions of objects into a single container So now I can upload an object So if I want to take my keynote from this morning where I spoke on Swift 102 and put that into the EC demo folder I can just click and drag it over and copy it up and while that is copying up I'd like to prove to you that we've actually separated the data into 12 chunks because I have a 7.5 So I'm in one of the storage nodes and this is getting a little bit deeper but OpenStack Swift has a command called Swift Git nodes and I'm able to run Swift Git nodes and I'm able to Tell it the account the data in the ring. I'm sorry the account the container and the object that I'd like to access and so when I do that I Can see that my data has been split if we count these across 12 different locations So based on the name of that object What Swift returns to you in this command are the locations that all of that data is being placed and we can see 12 different locations here. Can you explain a little maybe about what these handoff locations are there for sure? So if I tell Swift to break my data into 12 pieces Imagine that one of those nodes is offline if one of those nodes is offline I don't want to just discard that piece of data. So handoff locations are backups so if you look at my IP address is here. I've got 72 77 71 These are all different nodes it's going to try to as uniquely as possible distribute the data across all those nodes and of course my first handoff node is number 75 is IP address and Because I'm splitting my data 12 ways and I actually have 15 nodes. My first few handoffs are unique to the other actual sites if Swift can in all Instances it will keep the data as far apart as possible So if I had 30 nodes none of the handoff locations would overlap any of the actual data storage locations But because I'm splitting it to 12 pieces if I was to lose Three of my nodes then by the pigeonhole Mathematics there must be two pieces of data on the same server and it will be able to recalculate recover and move those around Dynamic but again those handoff locations I think of them as like hotels right when the primary locations that are up there are not available for the erasure coded bits They're just going to be temporarily stored in these handoff locations until the primaries become available again So I think of it as like a hotel for your data, right? Those are that's a good analogy now back up location We're running a little short on time Normally what I do is I would jump into one of these directories and show you the little piece of data But instead of doing that let's jump back to the keynote the slide deck Because there is a little bit more information I'd like to give you when we start talking about erasure coding and specifically around disk use space, so The question is how much data space is this going to save me because again we're saving space So we use a pretty simple example here and we're going to say that we're going to split our data 15 ways and 10 of those are going to be data bits and five of those are going to be Parity bits now again. They're not truly parody bits because we can pick any 10 we want So the mathematical formula is actually really really simple We take the total number of pieces we're splitting our data into in this case 10 plus 5 15 And we divide it by the amount of data bits. I need to reconstruct which means that 15 divided by 10 is 1.5 so for every megabyte of data that I send the Swift cluster It will result in a use case or use of one and a half megabytes of actual data being consumed out of my Swift cluster now if you compare this to a Replica system of making three replicas. There's clearly a savings there In fact, it's about a 50% savings again The trade-off is I'm going to be using more CPU resource to do the data calculation The other trade-off is if I have small objects, and I'm splitting them up 15 ways I'm using more IOPS on my disk so by default. I'm using a minimum of 15 IOPS Whereas if I use replicas and I take a small object and I split it three ways I'm only using three IOPS and of course as the object size grows There's going to be a point where those lines cross and the ratio coding can actually be more efficient Or a faster use case for big objects All right, so Let's go back to use case because we didn't have a slide on that and we actually do have a minute in 52 seconds for many So what are the use cases for a ratio code? I know everybody wants to throw a ratio code ratio code a ratio code at everything and as Rob explained It's new out. It's technically considered beta You know, you do want to make sure because it just came out with the Kilo release that You test it thoroughly first, but a ratio coding makes a lot of sense when we start talking about data That's going to live in a single Geographic location because it's going to save you space and it doesn't need to go other locations And why do I say that well? If I had some some important file and I want it split between San Francisco, I want to have it available San Francisco, Texas and New York What's going to happen is if I use replicas I would set my replica count to three and that object will live in all three of those data centers and when I access that object It's right in that data center and it's locally available And I can even lose to hold data centers before I lose an object Right, but if I use a 10 by 5 what's going to happen is I'm going to split that Object into 15 pieces and each of those data centers is going to get five pieces But in order to reconstruct my data From any of those data centers. I must pull five pieces From another data center So either I need to pull five from one or two and three or any combination four and one That's going to add latency, especially when we start talking about geographic distances. It's also going to add A lot of bandwidth and right now bandwidth across your pipe is actually more expensive than disk bandwidth on a spinning disk So a racer coding makes a lot of sense to me when we start talking about a video surveillance system For backups that live right at one site and then of course if you went to my talk earlier today Swift 102 you'd know that you can create multiple storage policies and through a thing called server-side copy You can actually have your backups going to an eraser coded Container on site and then through server-side copy moving that to a replicated one that Expands of beyond a multiple geographic cluster So I think I think we're at a time But hopefully we gave you guys a good overview of what Swift has added in terms of erasure codes How you can leverage Swift stack to be able to implement that test it? We encourage you to come over to our booth is right over there There's there's a bunch of eager people ready to talk to you about all this stuff Again, thank you very much for your time if you've got any questions Please Doug and I'll be over there at the at the booth and check our blog There will be up-and-coming performance and other benchmarks on erasure coding as this gets deployed more in the lab great Thanks, Rob. Thank you. Thanks everybody