 All right, warm enough in here for everybody. All right, so welcome to our session. My name is Paul Luce. I'm a senior staff engineer at Intel. I've been at Intel for a little over 20 years, primarily doing storage software development. I've worked on everything from raid controllers to Windows drivers to Windows file systems to most recently, this stuff. And this is by far the funnest. So that'll let John introduce himself. My name is John Dickinson. I'm the project technical lead for Swift. And I work at a company called Swiftstack. What we do is we come alongside of Swift clusters and provide monitoring and management tools for that. We've titled this just to talk generally about Swift storage policies. But I really do think that storage policies are the biggest thing to happen to Swift since it was open sourced four and a half years ago as part of OpenStack. And it's really exciting. So with that, yeah, sure. Let's jump into what we're going to do here for the agenda. So John's going to give you a little bit of background information, a little more than what he just gave you. And then a Swift primer, just to get everybody kind of up to speed on the super basics. There's a lot of other Swift sessions later on in the week. So we'll encourage you to attend those for more details. And then I'll jump in and take over and talk a little bit about why we thought storage policies was a good idea. What storage policies are. And then probably most importantly for everybody is how do you use storage policies? What new usage models do we enable? And then we'll wrap up with some status and we'll wrap up the project. So let's give a little bit of background to start with. Swift is really important, I think. And it's important because fundamentally I think that everybody has data. It's always growing. And you need to have ownership of everything of your data. And what I mean by ownership is you need to be able to know what touches your data from the hardware to the software that's managing the storage to actually the tool chain that's doing that. And the only way you can actually do that effectively is with open systems. It doesn't necessarily mean it's running inside of your own data center, but it does mean that you can get involved in the community, that you can influence the source code, you can see that, you can edit it, you can contribute back, you can affect the direction of the project, you can get involved with that. That is why I think that OpenStack is important. And that's why I think that Swift is important as a storage system here. But there's a lot of practical concerns you have as well with a storage system. It's important that we actually can store stuff and do things well. And so we need to actually offer something fundamentally new besides just the fact that you can see the source code and vote in an election sometime. So what is that? Why are we here as far as Swift? What does it fundamentally offer that's really valuable? I'm going to go into a lot more detail tomorrow. And I'll tell you when this is a little later at Swift 101 talk, we'll go into a lot of example use cases and architecture stuff. But to start with, the high level reality is that the old paradigm is if you need to store something, you plug in a hard drive. And then when you fill up that hard drive, you plug in another hard drive. And then you're stuck with this hard problem of siloing across these different hard drives and remembering how all that kind of stuff works and where you stored the things, it's really painful. And then you add in the complexity of the reality that hard drives fail. And so you buy a raid card. And basically, you've just made your problem bigger because instead of just having individual hard drives, you've got these individual silos of giant raid volumes, which have their own problems with rebuild times and performance and these sort of stuff. So the really important mindset to change, I think, is the fact that you have a storage system that abstracts this. It separates the data from the storage media itself. Then you can fundamentally do new things. And some of those new things actually involve supporting the existing modern applications that people are building today. So people are building things that demand scalability. People are building things that demand global access. Just to take a little brief note, I heard a new use case just yesterday that was really awesome. It shows a great use case of Weiswift is really important. And this kind of feeds into where we get into the flexibility of storage policies. You may have heard there was an airline that went missing in the Indian Ocean. And in order to find where should we expect to look for debris and stuff like this, they found a website that was hosted in Australia that was, you click on any point in the ocean, and it just kind of shows where the debris field is going to be in one, five, 10 years from now, which is really kind of a cool thing. It's fun to just click around to play with it. Well, what happens is apparently the Guardian got ahold of the story and just destroyed their website. And they were just a standard normal lamp stack sort of thing. And they figured, well, we could just add more Nginx servers and scale it that way. But it turns out they were running on the Nectar Research Cloud that the Australian government has set up. And their data was stored in Swift anyway. So instead of this figure how to scale out our web servers, why don't we just point the clients straight to Swift? And if you go today, you can even see the download URL for that is directly from Swift. And boom, their scaling problems were gone. The storage engine itself was able to offload those hard problems of storage from the application. That's why Swift is important. That's where Swift fits in. And I think that's one thing that was just really exciting to hear about yesterday, sitting in the hotel lobby. So to kind of back up and get a little, finish up the summary here, talk about how the important thing is to abstract away storage volumes from the actual data. And you can do that. You look at the evolution of storage media and technology there. Storage media fails. You want to deal with heterogeneous stuff. Old things go away. New formats come along. You may still have old iOmega zip drives somewhere in a drawer. But you can't get any data off of them because you don't have a zip drive anymore. And if you did, you don't even have the right plug on the back of your computer to plug it in anymore. So the point is new technologies are coming along. Optical stuff, memristors, fancy ways to write onto hard drives. And we need to be able to abstract that behind the storage system to take over that hard part of figuring out how to effectively store it from the application. So where does that give us? We can do that. But we still need to be able to support these modern applications, say, for example, hypothetically speaking, a website that gets mentioned by a newspaper about a major news story, or mobile applications, or user generated content of all this kind of thing. We've got to be able to support this massive concurrency and still remain highly available as the system operates. So that's what we do with Swift. That's where we are. So let's talk just briefly about how it works. And then we'll build in where storage policies come in and talk about the flexibility and the power that that gives you to directly match your infrastructure to your use case. What is the Swift API? We talk about it having a REST API. And you hear about accounts. And there's an object storage system. But how does that actually work? This is what a Swift API request URL looks like. You've basically got a few key parts. The most important pieces there are the account, the container, and the object. That's the things that Swift stores. And the account keeps a list of containers inside of it. Containers keep a list of objects inside of it. Both of those keep a little bit of aggregated metadata. And then the objects where your data is actually stored. So when you make a request, let's write a new object. Well, you send an HTTP put to your object name. And you get back an HTTP response code. And when you want to get the object, you send a get. And you get back 200. Here's your data. And that's the API. And there's a lot of extra features and functionality we've added onto this. But this is the important piece as far as how do you read and write your data? What are the parts of Swift? How does this work? So the client, the user who's needing to talk to the data, remember what I said, there's this system that's distracting away the storage volumes. The user talks to the proxy server. The Swift has basically two pieces, a proxy server and a storage server. The proxy server is what implements most of the API, but importantly takes all of those API requests and then coordinates the communication with all of the storage servers. The storage servers are responsible for talking to the actual physical media. What this allows you to do is a couple of really cool things. First, at the lower level, is that the storage servers then can deal with hard drive failures, deal with different sized medias, deal with different types of storage volumes and things like that, without ever having to worry about how that propagates up to the user. The proxy server, on the other hand, is able to know when servers go up and come down, when you need to deal with making sure your data is durably stored in the system and coordinating all of the responses there. And what this allows you to do is survive drive failures, server failures, network issues, potentially even rack failures, DC failures, and still have a highly available durable system. So, quick question here. Does the client ever talk to a hard drive when using Swift? No, it does not. Does the proxy server really have to worry about talking directly to hard drives? No, do we really care about what kind of hard drives plugged in and they're all the same size or they're all the same manufacturer model number? This gives you a lot of flexibility as far as how you are able to build out and scale your system and keeping very high uptime. So, here's the big picture. We need to know, we've got these nice logical block diagrams, very simple, but what does it actually do? What does it look like in production? Well, a lot of times, you're going to have building out with some sort of load balancing tier, talking to proxy servers, and then the proxy servers are going to have basically what you've got here is your capacity tier that has different types of storage nodes where the hard drives are. You've got auth servers that are associated with that, being able to plug in different auth systems to integrate with your existing environment. So, when an object is uploaded, the request comes through the load balancer to a particular proxy server, and then that data is written out three times, because Swift is a replicated system. Once it has been written out durably to those drives, then the storage nodes can send a successful response back to the proxy server. The proxy server correlates those responses, make sure that we got at least a quorum written out to all of those hard drives, and then can respond with a success back to the client. Now, the converse is that when a request comes in for a git, the request comes from the client, talks to one of the proxy servers. The proxy servers look up one of those locations and is able to get the data from one of those copies, and then it's sent back to the client. So the important piece that comes in there is what about the data placements? This is an overview of how we're actually assigning. How do you choose where a particular object lives on those hard drives? You've got a lot of hard drives. You've got, you know, you may have dozens. You may have dozens of thousands of hard drives plugged into one cluster. And these are hard problems to solve as far as effectively balancing this. The data placements is done based on a server, a drive, a zone, and a region. And I think I have it better. This is what I wanted to show. So it's kind of a tiered system here. So what happens is first we want to make sure that we put the data as uniquely as possible across all the drives. So if you have more than three drives, make sure that every copy is stored on an individual drive so that one hard drive failure does not cause any sort of durability or availability issue. And then you propagate that up with servers, zones, and regions. And so when a request is made to create a new object, we calculate out where that should go and choose three of those. So in this case, we're going to choose a hard drive on three unique servers. Notice we only have two zones. Zones oftentimes deployed as an individual rack. And therefore, you can lose even an entire rack and still have access to your data. So how do we do this? Let's get probably the most technical piece here. Paul's going to actually go into a little more detail in a bit about this. But the way we do this is with a consistent hashing algorithm. Paul's going to go into detail in just a bit about that. But to give you a brief, high-level introduction to hashing, we all understand it because we've all used it for all of our lives. So think of an encyclopedia. Remember when you were a kid, you had to go home and look something up in encyclopedia before Wikipedia. And you needed to go do your book reports on the jungle, so you, the region, not the book. And so you pull out J and you go to jungle. And then you want to go look at kangaroos. So you pull out K. And then you do ostriches. So you look at, oh, things like that. That's a hashing algorithm. You've taken a broad subset of all the topics, and you've figured out a way to fairly evenly divide it. And what's really great about that is you can kind of further subdivide and very quickly go look up where your data is. So keep that in mind when Paul goes into kind of a little more about how the storage policies are implemented. All right, thanks, John. OK, so why storage policies? Well, as John just took you through, and we'll go through further in detail later this week, Swift is pretty successful today. It's pretty flexible today. But as with any software project, any really project in general, there's always technical opportunities. So we've identified a few technical opportunities and have answered them with this thing we're calling storage policies, which by the way has been a great collaborative effort between multiple companies in the community. It's going really, really well. We started working on this, I think, last summer, right, John, a lot of great contributions. I want to say thanks to everybody who's pitched in on this. But some of those technical opportunities, some may be obvious, some may be not. As you may or may not know today, Swift is a replication system. And your choices are how many copies? So you get it one way or the other for the entire cluster, whether that meets your needs or not. And it does meet a lot of people's needs today. But we think we can do better. How about all our old nodes equal? Well, most clusters deployed unless they were just brand new purchased are not homogeneous. You've got some machines that are faster than others. Maybe you've got some hard drives in some systems that are SSDs. You've got some legacy stuff. You've got this mattering of stuff out there. And as John said, although you don't need to care which hard drive is in which system, sometimes you might want to care. And one example with today's Swift is the use of account container databases on SSDs. But today, you can't use those SSDs for objects, just for your databases. So what if you wanted to use them for objects? And what about adding something like erasure codes? So I think everybody's probably familiar with that. If not, we'll talk a little bit about it Wednesday afternoon in an Intel session. And you'll be hearing a lot more about it if you come to Paris, hopefully. Today, it would be very difficult to bolt something like erasure codes into a replicated framework. But with storage policies, we're really building in the extensibility to do things like erasure codes. We'll see more about that later as well. So with storage policies, as John alluded to, we're also, besides bringing some really interesting technical ideas, we're really increasing the modularity and extensibility of an already successful system. So this is a pretty cool diagram that John had put together that gives you a couple of different perspectives on where the knobs are that you can tune with storage policies and with some other features that were recently added to Swift, like disk file, if you're familiar with that. And I think Pete has a design session this week on disk file architecture. So that'd be a good one to check out, too, if you're on the design side. But what this picture shows us is, looking at the top, the client now has a selection of things as well as the operator. And from two different perspectives, there's different things that you can tune. You can choose your hardware, which is primarily a deployer thing, but your client has a say in that as well. You can choose your encoding, like I just talked about. Policies will allow something like a ratio code to come in the picture. And by the way, it is coming into the picture right after policies. And then your protocol as well. So this is your back-end disk protocol. So when you combine things like storage policies and disk file, you really get an incredibly modular and extensible system. Okay, so can't really do the topic of the Swift ring justice in one slide, but we're gonna go into the hood just a little bit because the concept of Swift's consistent hashing ring is really a fundamental part of storage policies. It is the foundation of where storage policies really gets its enablement from. So the Swift ring is actually implemented in the code as a static data structure. So I'm not sure how many people are super familiar with this, but again, there's a whole lot of really good resources. John and Sam at Swiftstack have a video out there that's really good. Greg Holt has a series of articles on consistent hashing ring that's really good for background technical information. But actually in the Swift implementation, what we'll be talking about here is this conceptual ring and these idea of partitions, which are sections of the namespace within the ring, not to be confused with disk partitions, which is probably a pretty common thing for people new to Swift. I know it was confusing for me. But the way this works is you take a URL like what John showed earlier with the sample of the put and the get, and we calculate an MD5 of that. You can see that up at the top. And that provides us an index into the ring that identifies a particular partition for us. And each partition then maps to a set of tables that are part of this ring structure. So this ring structure is not actually a circular structure of some kind. It's two tables and a few other elements with some descriptive information about the ring. But what happens then, I'm not gonna talk about the names of these tables because they're hard to read for a human. But let's say this thing works, okay. So let's say we did that lookup of a URL. We'll take that index and we'll go to the first table and you can see we'll go to the first table and that will pull us a row in the first table. From that row, depending on how many replicas we have, we have a certain number of columns. In this example we've got three columns because we have a three replica ring. So that row gives us three other indices that we use to go to the second table. And that second table is actually a list of devices. It's a list of nodes, right? Or more specifically, it's a list of hard drives, right? With their IP address of what storage node they live in. So this static structure is responsible for the placement of all of the data within the cluster, right? And if you can imagine when I mentioned earlier the account and container databases can live on SSDs, what gives them that capability is they have their own rings, right? They have their own set of tables. When you go to make an update to a container database, it doesn't go to the objects ring, right? Or the objects set of tables, because that would point it to a bunch of hard drives. It goes to a special ring that you create for the container databases that the cluster administrator has specifically put SSDs on to help eliminate that as a bottleneck in the system. Okay, so that's just kind of a quick intro into the ring. We'll go backwards and forwards a little bit. Okay, so what are storage policies? Well, as you might guess based on what I just explained, hopefully you might guess, it's the use of multiple object rings, right? That's really the simplest explanation. And honestly, if you're super familiar with Swift, that's probably all you need to say about it. And you go, oh, I get it now. Boy, I can do a lot of stuff with that. Simple little concept. So here's an example. Three different rings, three different policies. So here's the first one, trip replication. As we saw in the last couple of slides with triple replication, you get three locations from your ring when you go and say, where do I store this file? Or where do I get this file? With storage policies, you can do something like reduced replication. Okay, so only specify two locations. So you might say, well, Paul, earlier you said, I could pick any replication level I wanted, two or three. So what's the difference with storage policies? The difference is this can be in the same cluster. You can have two X and three X replication coexist in your same cluster. And now you give that choice to your application. So it can decide what level of replication it wants. Okay, and then last example here, coming soon to a theater near you, erasure codes. Okay, so with the erasure codes, we'll use the ring to store the location of all of our data and parity information. So you'll get a bunch of locations back from the ring that's assigned to the erasure code policy. Yeah, this thing's not working. Okay, so the second part of storage policies, the second sort of big feature, I should say this is a fairly significant undertaking. Not only as far as like what John mentioned, it's importance to the project, but the sheer size of the effort, we've started last summer. And I think by the time we merged to master here soon, I've been saying soon for weeks, but it really is soon. We'll be close to 5,000 lines of code out of a total of including test code about 60,000 lines of code. So it's a pretty significant change, touches a lot of places, which is kind of amazing because it's so simple. But the second big piece of it is the use of a new container tag called X storage policy. So this is the actual, the only change to the API, which is the other really cool thing. All this capability, all this feature, one little metadata tag does it all. So when the application creates a container, it just provides this little tag and it gives it a storage policy. Says, okay, everything, when you write an object into this container, make it erasure coded or make this container a triple replication, make that container a double replication. Can do it all within the same cluster. And then your application just picks its durability policy based on which container it writes to. Or geographic dispersion. It could be one global Swift cluster and you have an EU set of nodes in US. Yeah, we've got some usage models coming up that'll talk to what John was just mentioning. But yeah, it still floors me. It's such a simple concept with so many different usage models that we enable with this. Okay, so I'm gonna flip through the animation on this one. I won't go through this in detail, but I did wanna give everyone here a sense of sort of the size complexity. I don't really get complexity by looking at this picture, but this is your super high level software architecture diagram of Swift, showing the components that run on the proxy server at the top and down at the bottom the components that run on the various servers, container, object and storage. And the ones that are colored in orange are the ones where we've had to do a decent amount of work to get storage policies to become a reality. So just to give you a feel for it, and if anybody wants to talk any more of the details about this, of course, we'll be around all week. Okay, so let's talk about usage models. I sound like a lounge singer. Be here all week. So I think some of the fun things to really drive it home to say that, okay, we've got this level of work. This is kind of why we're doing it. Where Swift fits today, how it's gonna be extensible. Now let's actually bring it back home. So we know how it works. So I think the first obvious use case would be, I'm gonna store web content, but I need to store thumbnail images. So I don't wanna stand up a second Swift cluster just to store thumbnail images. I can recreate those any time they're missing, but it sure would be nice if I could store them and didn't have to recreate them on demand for every request. Great, let's create a storage policy that has thumbnails. And let's put our source images in the normal triple replicated for high durability storage. That way I know that I can always fall back to my durable storage, recreate the thumbnails, put those into the thumbnail container, and everything's gonna continue to scale very well. And I'm gonna save a lot of money on hard drives because I don't have to have that third copy for the data that I know I can recreate. It's not as important to me if I lose that data. I don't want to, but that data I can recreate. Another usage model, performance tier, right? So I mentioned this when I talked about today's usage of SSDs for account and container. Imagine now that you create a container and call it your SSD policy or your performance policy, whatever you want. Now when your application knows that it's really important data needs lower latency, you can now store it to an SSD policy and it doesn't have to be three copies. It could be two copies, could be one copy, it could be whatever you want. Again, because it's its own ring. And I think something important to note on all of these things is that notice that in the iconography here, these drives are actually on the same physical hardware. Some of the drives are on different physical servers, some of them are on the same. It still means that these things can overlap. They don't have to be completely discreet to rings. It means that you can, again, very much tailor it to your use case. So the last one I think is very interesting is looking at the basically the geopolitical realities of where we are, especially outside of the United States. It's very nice just for locality to have different regionalities. Wouldn't it be kind of cool if you could just say, I've got an East Coast location and a West Coast location. But especially outside of the US, for example, in a lot of the EU and in Australia, they have laws that say you cannot let this data leave our border. So it would be really nice if you could say, I've got a global Swift cluster because I need to have locality of access for some subset of data, but these backups are this personally identifiable information. I need to guarantee that it doesn't, for example, leave Germany. So I'm gonna put it in my German data center. I'm gonna put that storage policy there that says this data is only gonna be right there. And that's I think one of the very nice things, both for I need to constrain my dispersion, but also to say that I can expand my dispersion if necessary in order to have locality of access. So that you can, this is kind of a follow on of the global clusters concept that Swift supports today. But now you can say, I wanna have this data that's going to be globally dispersed. This one's gonna be East Coast. This one's gonna be Asia. And you can all manage that in the same Swift cluster. And yeah, and last, but definitely not last, last in our slides is I'm certain when we get this out there, people are gonna come up with usage models that we didn't think about as developers. These are things that we've thought about sort of, in a vacuum, but as soon as this gets out there, I think we'll see a lot of other stuff take off. But erasure codes, as I mentioned earlier, there's your container with triple replication. We had erasure codes and now it's again, policy-based via container. So an application decides that it doesn't want triple replication and needs to save not only disk space, but power is important as well, right? If we're talking significant number of hard drives, power in the data center, cooling, all that stuff comes into play, all that capex, op-ex kind of stuff. But you can see here now we store EC fragments instead of objects. And again, we've got another talk on Wednesday. We'll go into a little more detail on erasure code and stay tuned, because we're working on it hard over the summer. So where are we going on this? If you keep up with the OpenStack development mailing list, you know that I've been talking about this for a little while within the community. In fact, we were talking about it to some extent in Hong Kong. And as Paul said, it's been a massive effort from a lot of people. And we're not gonna release it before it's ready, which means that basically another way to say that it's your data's important, we're not gonna throw some feature over the wall just to have a feature by a time-dased release without having proper documentation, proper QA, proper vetting, proper design, all of that sort of thing. So for those reasons, it was not a mistake, but it was something that was, we did not feel comfortable by releasing by the Ice House release. Since that, which was really just a matter of a few weeks ago, we've been working quite a bit on getting those last little feature gaps covered, making sure we can do the appropriate documentation and QA and things like that. Storage policies, I'm really hoping should be starting to hit master or the patch proposals to master very, very soon no matter if I would hope this week or next week, we'll see. And then it should be continued to go through the integration process with full developer review and QA vetting. And so at that point, we should have that in the next Swift release. Swift release is a little more often than the six month open-stock cycle. And we will definitely have this in the Juno release as far as the combined open-stock release. So we're looking forward to that continuing and being finalized real soon now. After we're done with the storage policies, we have that framework built out. We'll be building out the erasure code storage policy so you can manage that effectively. And that's what we have on this. I think we have, let's see, a about 10 minutes, nine minutes. And for any questions you may have, we've got a microphone in the middle. But a few things I wanted to call to your minds later this week, that we have quite a few sessions that are around Swift and Swift sort of things. Everything from I'm a developer, I want to extend Swift, how do I do that? I've got some use case, some case study sort of things of real world users who are deploying it and a lot of great stuff. Couple things I wanted to point out. We've got a workshop on Thursday where you can walk away with a Swift running on your own, your own personal Swift cluster. It's a great gentle introduction to I want to deploy Swift and play with it. And a couple more that are not on this slide are at 340 today in this room. Joe Arnold's going to be talking about global clusters. And then at in B101 tomorrow at 11.15 in the morning, I'll be talking with Joe on a Swift 101, just kind of here's the technology and architecture for beginners. It's very gentle introduction to what Swift is, how it fits, where it works. And with that, I think we have a couple of minutes for questions. And we actually also have a Wednesday session at around three, I don't have the exact details, it's called Intel Contributions in Object Storage. So come here a little bit more about policies of Racial Code and Cosbench also. So question here? The question is, what's the granularity of the storage policies? Where are they specified? Is it the object, the container, or the account? And the storage policy is determined, is set at account creation. And so you can create, for example, a Racial Code container. You can create your global cluster container, your EU container. Container creation, you meant to say. Yeah, create container creation, sorry. Yes? Great question. So the policy sounds similar to tiered storage. So the question is, can you change the policy after the fact? And very explicitly at this point, we are keeping that outside of the scope of what we're doing. So furthering some of the design principles that we've embraced within Swift, we are ensuring that we have all of the tools and hooks and power available to be that good storage engine. And if you need to have a simple script that can migrate from one storage policy to another based on a certain set of rules that is entirely possible and supported and pretty simple to do, it is not something that's going to be included inside of Swift Core's code today. Yeah, so the question is, is there gonna be APIs for this? To start with, what we're doing right now, what is in scope today, is the ability to, the API is simply move it from one container to another. And you can use that using Swift server site copy feature to say copy this data from that container to that container. And there you go. Over here? Yes, exactly. So talking about geotagging and saying that what does that mean and how is that done? And it's defined in the ring and it was just confirmation that yes, what you would do is if you had say a New York location that you would, the drives that are physically located in the New York region, you would add to the New York ring. And that's how that would be done with geotagging. Let me add a little bit to that, John, just to clarify. That's an example of a usage model that isn't completely fully baked yet, right? That's something that as a complete solution, we'd probably wanna file a blueprint this summer and do a little bit more looking into the account and container side because those pieces aren't covered by storage policies. So if you had a compliance regulation that required the metadata to also be in that geo, we need to think about those kinds of things. Also at Intel, the software and solutions group is working on a similar feature for compute networking side and we'll wanna look at ways to tie those together so that you could actually spin a VM and have all of your storage be specified and isolated to the specific geotagging policy that's associated with the VM. So there's a lot of work to do still there. That was just sort of an example of what we can do now that we couldn't do before. In the back. I'm talking about the life cycle management of the data. Can you nest policies? So yes, to a certain extent. So let me, there's two questions. The first one is the automatic tiering and basically some sort of HSM sort of thing. No, that is not going to be just like the question up here. And that's not something that's in scope for SWIFT right now. The second question about nesting policies. I wouldn't use the word nesting, but yes, what you're looking for is entirely possible to say that this is gonna be erasure coded in this particular geography as opposed to this is gonna be replicated in this particular global state, something like that. Yes. Can a single disk drive host multiple containers with multiple policies? Yes. And in fact, that's what happens with SWIFT today. There are the data on a particular hard drive is data from all over the cluster. There's not, in SWIFT, the API is not such that you are going to be provisioned a pool of storage and then it's assigned to a particular thing. It is really you are given a namespace which is placed across this given set of hardware. And one other relevant point to that, we've designed storage policies such that the same hard drives can be used for multiple policies. So although our pictures for simplicity kind of showed these policies mapping to these drives, you'd have one set of drives that support three, four different policies and not have to worry about separating them. You can make them separate if you want, but you don't have to. Yes. No. No, that is not the case. Well, you do not divide the drive up when you're doing this. So what happens is if you have ever looked at, to get into a little of the technical details, if you look at how data actually is laid out on the drive on the local file system, for example, your object data is in a high level directory called objects. So we're going to have a sibling directory for that for each additionally defined storage policy that has data on that drive. So if you have your tuna fish storage policy, as opposed to your cupcake storage policy, then maybe tuna fish's storage policy index one will have objects dash one as a high level directory. So you're not carving up a single hard drive into multiple file system partitions and storage volumes. What happens is that you have one storage partition. Basically you can think of this as one failure domain. This is the hard drive itself, which is storing the data on top of it. And Swift is managing the global state of it. That particular failure domain doesn't have a global state. Across the cluster. So I'm kind of thinking whether these policies where you can use one drive to want to hurt that. So the talk about data placement and making sure that things are evenly balanced across the drives and does this hurt that because you might have one drive that has more policies on it than another. Yes, you can configure a policy that says, you can put one drive on all of your policies and overload that drive. But in general, this is not affecting the data placement. But as an operator, yes, you need to manage your sizing of drives. I think we have time for one more question in the back. So if I understand you correctly, the question is if I have an existing triple replicated policy that's globally distributed and I add on another reduced redundancy policy of two replicas, does that affect the data placement of the first policy? And the answer is no. Each of the policies because they are implemented as individual rings are actually fully autonomous and at impotent. So a change to one does not affect the data placement of another one. With the caveat that if you have one small hard drive that has, you know, every policy is gonna use this one hard drive. Yeah, that one might fill up more than one. It's only used for one policy. But that's more of a operational management system, not something as far as the general data placement. So I hope that you have a better understanding of how storage policies work and why they're awesome and why I truly think that they're the biggest thing that happens to Swift since the whole project was open sourced. It's really exciting. I want to say a particular thank you, especially to Paul for helping me today and also doing the lion's share of the work on the coding of the storage policies. Also Sam and Clay at Swiftstack and several people at Red Hat have been very instrumental in making sure that this is going to reach the light of day. So thank you for your time. We'll be here all week. Come to lots of the other sessions and have a good day. Thanks everybody. Thank you.