 Hi, folks. This is Mamu, the same from Intel. I'm going to be talking here about a performance benchmarking tool for object stores. I was going to do a show of hands question, but I guess that works only when there's a room full of people at a keynote. So I'm just going to skip right ahead. Before I actually go ahead talking about the tool itself, let me tell you a little bit about object stores. Not sure. I'm sure everybody here is familiar, but it never hurts to just make sure that everybody knows what I'm talking about. So the best way that I like to think of object stores is when I think of Facebook and the way they store their data. Traditionally, a long time back, we used to think of data as being stored in nice little rows and columns that would be a structured data. But increasingly today, a lot of data is being stored in an unstructured manner. These are your server log files, pictures, and videos, things that don't have or data that doesn't have a very well-defined format and data that you'd need to have retrieved on demand and many different times, depending on whether you're accessing it immediately, hot data versus data that you've kept at archive and you want to retrieve at some other point of time, like cold data. So the question that we try to go about solving at Intel is, as more and more data starts getting represented in this unstructured form, where you really don't care whether it's in nice little rows and tables, but you really care that the data is available to you, what are really the metrics that we are using today to make sure that the databases that we are maintaining, that the infrastructure that we are creating, actually meets the demand that the end users have? Is there an objective way of understanding what the performance of these systems are that we are building today? And how do you really compare one implementation versus the next? Maybe you have a couple of implementations due to different vendors. You've received two different codes as an end user. How do you really objectively compare whether the throughput is really important or whether the CPU is really important or if some of the HP is talking, whether the latency is really important? So what would be the really important metrics for me to capture so that I can understand what would be the best configuration to use? And that's what Cosbench is about. It's a performance benchmarking for Cloud Object Store. So with that, let me just get started with the real thing. So Cosbench is really the red highlighted items here. C-O-S-B-N-C-O. I should have done a red highlight on the S. So that's Cosbench. So it's a Cloud Object Store benchmark. And it's really what it says, right? Think of object stores. Again, object stores, think of Facebook. Think of object stores in the cloud, Amazon S3, Facebook, and so on. How do you measure the performance? This tool helps you get that done. We developed this internally. We spent quite a few months developing this. And now we have released it to open source. I'm not sure if any of you have been in the OpenStack session that was sponsored by Intel. There was a couple of slides that we talked about, Cosbench. This particular session, I just want to give you a little bit more of a view. For those who are really interested in the internals, we have a much more deeper session that's being delivered by the devs who actually created this. Great opportunity for you to go and talk to them and understand the actual architecture. I'm going to give you what I can in the next 12 minutes. So the key focus here is to measure the parameters that really matter for you. Throughput, the response time, and bandwidth. So let me just say a little bit about each one of these. So when you want to measure the throughput, it's really the number of operations that are being completed in a second. Really, how much data is going through? And are you really exercising the network and the system underlying with all of the get-puts and deletes? Now, with any of these object stores, you need to keep in mind that this is a restful API, just like Swift, like AS3, and so on. So you'd be doing a standard get-put-delete on all of these object stores. So how many of these operations can you get done? The bandwidth, again, how much data can you transfer through? And we also measure something called the success ratio. How many of these tests succeeded and failed? There are a couple of interesting problems that we managed to solve and a couple of interesting problems that we haven't been able to solve yet. One of the biggest problems we haven't been able to solve is, really, how do you clean up the test bed once you're done with the testing? And I think there are some scripts out there that help you do this. And so the thing to keep in mind is this is a tool that is evolving. The reason we have brought it out to open source is to get contributions from folks like you to also share the work that you have done and make sure that this is something that ends up both the end users, cloud service providers, as well as our ISV and OEM partners. So if you're still trying to figure out really what cost bench is and you've been using IOMeter, think of it as the IOMeter for object stores. So it's just taking the old concept of, here is my system. This is how I've configured it. Does it really meet my needs? And that's really what it helps you do. It understands what that is. So let me see if I can go to the next slide. Yes, success. So very quickly about the architecture and I'm not gonna pretend that I really am pretty technical about this. This is just to give you an idea of the way it is. From the ground up, we've built it to be sure, to make sure that it is very modular in nature. You should be able to take away pieces of it and we recognize Intel being a part of open source community for a long time. You wanna make sure that everybody has different pieces of interest in every tool. You would wanna make use of, let's say, the load generator, but not make use of the gooey part. So you should be able to take various parts of it and cobble together your own system if you so wish. More than welcome to use the tool as is, but we've architected it in such a way that you'd be able to take parts of it that you find useful and build your own tools if required. So the really top level, you basically have an admin console, which could be as simple as just this laptop. It's a web-based gooey that connects with the driver and allows you to, it's a set of a command and control. It allows you to specify the size of the get, the size of the ports. And you have objects raising all the way from 10 kilobytes to hundreds of megabytes. And you can specify what the interval of the gets and the puts is, how many threads you want, what kind of CPUs and so on. So there's a lot of configuration details that you're able to input. We have a gooey that we are working on. There are still some minor issues that you're working in terms of usability. So it is assumed that you know how to configure Swift, and this tool will not walk you through configuration. It's really walking you through the setting up of the test itself. And then the console, the admin console talks to the controller, which is really the brains behind this tool, which is really sending all the requests, the get puts into leads that you have, and funneling them off to the drivers. And the drivers are really where the work is happening. This is where the system is getting stressed. This is where all of the requests are being generated. The objects have been put or deleted according to your test specification. And all of this happens down below on the actual storage nodes. This is a standard Swift configuration. The proxy node decides where the request goes and distributes across. So the key is, if this is your system under test and this is the test tool that you're running, you want to make sure that there's a good level of decoupling between the two. At the end of the day, you want to make sure that when you measure something, you're not changing the results so much that when you take away the measurement, you start seeing some different behavior. So to that end, the architecture of this tool allows you to do that because you're able to take away certain components if you think that is causing a problem and then able to plug them back in. So that, well, by the high level architecture, in terms of the design itself, we've developed this in Java. So there's a lot of scripting tools. This is not the only benchmarking tool that's out there. But having looked at some of them, I do believe that this is probably the most easy and most mature to use. We've shared it before we released it to open source yesterday. We've shared it with a lot of our partners under NDA. And now you can go and download this from GitHub. And we've received some very strong and positive feedback. And we've been continuously working to improve it. And now the hope and intention is that as the community hopefully gets involved and finds it useful, we'll be able to do this better. No idea this was a build slide. Let me build it out. Yep. So without getting into the details, and this is a very top-level architecture, being developed in Java, we want to make sure that this is easy to debug and easy to use. At the same time, you also have an adapter layer that you're seeing over here. So right now, the tool supports Swift. It's been tested on the latest. It's not been tested on Grizzly. I believe the tests are ongoing. Excuse me. But it also supports AmpleyStore version 2.3.1, which is one of the more proprietary object stores. The adapter layer allows you to take this code and develop support for your own object store. If you're using a VMware object store, for example, you'd be able to take the adapter and create support for that. You'd still be able to use the same framework. But just to specific, if you're not using Get Puts and Delete, you'd be able to replace that. So some of the things that you're working on right now is we're trying to figure out how we can support CDMI, which is an interface that the Storage Network Industry Association backs as, let's see, a de-factor standard. Here in the US, I would say the de-factor standard is AS3, because everybody uses AS3. They want to be compatible with the AS3 API and has the Swift as well. So Get Put, Delete is the standard. CDMI is something that the SNEA Industry Association is standing behind, and we want to see how what makes sense to support that. Now we want to support AS3 also at the end. So the basic thing that I want you guys to take away is that we've made a start with OpenStack and Swift, because that's where a lot of the momentum is, and that's where a lot of the interest is. But by no means do we want this to be constrained just to OpenStack, Swift, because there will be other object stores out there. OpenStack, Swift is a great object store, but there will be others out there. We want to make sure that you're able to compare the performance across these, not just between different vendors of Swift, but let's say you want to compare the performance of Swift versus Ceph for as an example. So you want to be able to do that. And the workload definition is something that we are definitely working on, and that's very easy to configure in an XML script at this point of time. Right now it is what I call a synthetic workload. So you sitting in the lab think, OK, a web hosting company might generate a workload like this. Where we want to get to is to be able to specify workloads that are more representative of the real-life workloads with an actual CSP or a service provider would be encountering. So right now we have synthetic workloads that you can specify inside an XML file. We want to get to a point where we'll be able to specify real-life workloads. So that's work on, excuse me, ongoing. So this is how it looks like, right? This is the screenshot of the controller. As you can see, it's very simple, and that's deliberate because the intention is not to provide a really heavy tool. The intention is just to make sure that all of the functionality comes through, and you're able to get this first set of tests up and going. So you should be able to get this installed and running once all of the Java dependencies are taken care of within about 10 minutes. You should be assuming, of course, your Swift setup is running underneath. So as I said, this allows you to specify your mix of gets put deletes, size of the objects, kilobytes versus megabytes, the interval, what kind of data you want captured, and you will receive the data. And this does show the data either on the GUI console or inside server logs. The advantage of seeing this inside server logs is that for every request, you're able to then look at what was the latency between the requests. So all of the raw data is provided to you, and you can run your favorite analysis or statistics tool on top of it. We prefer to use Excel just because it's convenient. But if you want to use any other statistical tool, more than welcome to, all of the data is available. And this is an example of some of the data we extracted. As I said, we use Excel because it gives us some really good graphs right off the bat. And here's an example of a test load that you're seeing on a Swiss cluster. You can break it down as a broad graph or as a chart, however you wish to use it. And the last minute or so that I have, so we do have a demo of Cosbench. It should be here somewhere on B7. Just look for the Intel sign. Eric, I know, is doing a great job of showing the demo. So make sure you stop by, have a look at what the tool is, what it can do. And we'd love to get folks involved. We are looking for folks to start downloading this and using it. As I mentioned, we've got a lot of partners. We've already used it. But we really want this to be useful in the broader community. So if you get a chance, hit up the GitHub link. It went live yesterday. You should be able to get the user documentation, the source code, everything you need in order to get started with Cosbench. And that's it for me. Thank you. You can find me at the back if you have any questions. Thank you.