 All right. Welcome everybody. Thanks for coming out. My name is Vipula Sabaya from HP. And I'm Tim Simpson from Rackspace. Okay. Today we're going to talk to you about Project Red Dwarf database services for OpenStack. We'd like to talk about what the project is all about, how it fits into OpenStack, and give you an overview as well as how far we've come since the last summit. Okay. So today we'd like to ask a question, which is, why should databases be considered a first-class cloud resource? Because if you think about it, a database is just a program, right? And programs run on servers, and we have Nova already. So Nova can run them, and we're not talking about making like a patchy or PHP or something, a first-class service. We have tools like Heap that makes it really easy to automate deploying things like databases. So what gives? Why would we even be here discussing this? I think the important distinction is that databases are data. They're the most fundamental parts of your platform stack. And yes, Nova can store data. It does a really good job of it. In fact, Nova is so quintessential, it sort of is like the building block for everything. If you look at projects so far in OpenStack, they all came from Nova somehow. Cinder came from Nova Volumes. The project that used to start with a Q word came from Nova Networks. You know, and that's really great. I mean, Nova, there's so much that you can do being able to spin up virtual servers in the cloud. But sometimes many users don't really want that level of abstraction. What they want is a database, right? They want a virtual application rather than an actual server. And so this is all about the level of abstraction. We think it makes sense to sort of make a database a first-class citizen. So let's compare something that's very similar to databases, file storage. Sometimes just setting up a virtual server and storing files there is all you really need. So I think lots of people when you first set up a server, like probably lots of people in this room were in high school and they set up a server, what's the first thing they did? So they just stored files on it. And this works really well. But then you run into problems because you have to archive and protect all that data. And you have to really protect it from an environment that doesn't understand its true purpose, right? So to Nova, those files are just bits and bytes on a virtual hard drive. But to you there's something else. They're an abstraction and that's files. They're memories, right? They could be pictures you took during a vacation. They could be pictures of like a new addition to the family or something. You really want to protect that. You want to make sure that they're backed up. You want to make sure that you have instant access to them when you need them. You want to make sure you can access them over the internet. And databases are sort of in the same boat, relational databases. In some ways, yeah, they're just a program that runs on a server. But they're also data. They could be your entries on a WordPress blog. They could be contacts on your phone, right? And you want to make sure most people, when they're making apps, they just want to make sure they have access to that data. That it's safe and they can scale it up and down as they need to. Okay, so this is sort of the level of abstraction that RedDwarf has sort of been designed for. Simply put, it's a way to provision and manage database resources in the cloud as efficiently as possible. RedDwarf, it does a bunch of things. Some of it include automating administrative tasks like deployment, configuration, managing backups, restoring data. It's things that you don't have to do anymore in terms of dealing with databases. It lets you think of your data at a higher level. So the data that you're storing, you can think of it as just your blog post or the contacts on your phone. You don't have to worry about where it's actually stored. It also frees you from the mundane tasks so that you can worry about things that actually matter to your application rather than managing that data. So with RedDwarf, what you get is a managed service, not a service that you have to manage. We spoiled it. There's this thing that keeps coming up. Alright, there we go. Hopefully no one looked ahead. Alright, sorry about that. So what you get with RedDwarf is a managed service. It's a service that takes care of scaling, high availability, multi-tenancy, and also uses your resources effectively. These are things that you typically, as a deployer of a database, you have to worry about. But these are things that RedDwarf also takes care of for you. And for the best part, RedDwarf is fully open. All the code is available on Stackforce today. So from the server side pieces to the deployment scripts as well as the CLI to manage everything, you can find it all on Stackforge. Rackspace and HP, we've been working on this for a while now. And we have an API as well as an implementation that we're completely dedicated to. And one of the great things about RedDwarf is that it's built on OpenStack, for OpenStack from the beginning. That's why we feel, you know, the goal of RedDwarf has never changed. It's been designed to be the core thing or the thing that manages databases in OpenStack. And that's why we feel pretty strongly that it should be the de facto database service in OpenStack. Our API, if you look at our API, it fits nicely with other OpenStack APIs. And it's very similar to other OpenStack APIs. So it should be very familiar to you. So if you think about applications, right, application architecture, unless you're writing something on the desktop, in which case in the middle you're going to see where you're probably talking to like a cloud service these days. Or like a mobile app, you'll be talking to a cloud service. But if you think about a web application, you're always going to have an architecture diagram that looks a little bit like this. And in the middle, there's always going to be a database up there. And for sort of newer apps, it's going to be maybe like a NoSQL type database. But for most apps, it's going to be a relational database. And so really something that this adds to the OpenStack picture is this missing puzzle piece of having the ability to instantly spin up a relational database. So just imagine if, you know, Reddorf isn't an OpenStack right now, but just imagine if it was, how easy that would be. Okay, there's more and more methods out there to deploy OpenStack. DevStack's probably the most prominent one. I think, you know, most people who here has used DevStack to deploy OpenStack. And then there's like, if you go to the expo, like I know last summit, they were giving away like USB keys to set it up for you. So imagine if you set up OpenStack and you had access to Reddorf, along with all the other great services that it provides, you'd be able to set up your web applications very easily. You'd set up a server to actually run the code. You'd set up a database to handle all of your data. You'd back it up to Swift and then you'd be able to use the tooling such as Horizon to sort of see, you know, manage that data. And so we really feel that this is a piece that would add value to OpenStack. You know, it would fit really well. But as it is today, you know, if you set up an OpenStack cloud, you just have to sort of deploy the server yourself, install SQL on it, back everything up using Kron, secure it yourself. And, you know, he can automate this, but if you think about it, it's sort of equivalent to AppArmor, right? AppArmor is a really awesome program, but you're still responsible for it. You have to think, okay, I need to check this. I need to make sure that nothing's breaking and that everything's getting updated. Whereas if you just had an abstraction that says, hey, I want a database, from that point you're good. You just let RedDwarf do it for you and you have your database. Well, you know, we're not in OpenStack, but here's the good news. You can still use this today. So, you know, RedDwarf works as a, you know, if you're shocked. RedDwarf works in conjunction with the OpenStack, you know, cloud you already know and love. So, if you have access to an OpenStack setup, you can deploy RedDwarf to it. And it complements any install, regardless of whether or not you control it. Okay. So, how does RedDwarf complement OpenStack? So, RedDwarf leverages most of the OpenStack components, things like Nova, sender, quantum, we make use of all of that. We've got a fully functional REST API. We feel like it's been well thought out and we consider that to be the open API for database management in the cloud. If you look at our code, you'll see that we make a ton of use of the Oslo code base. So, the common code that's shared across other OpenStack services is also the basis of our implementation. What this means is that it makes RedDwarf very familiar, and it also reduces the learning curve for somebody new. We've also developed a piece of software that we call the guest agent. This is the thing that's first class on the database instance, on the compute instance that's running the database. It's the thing that's also responsible for managing the database software for the end user. And finally, from the beginning, we've built RedDwarf with plugability in mind. So, one of the biggest design goals we've had from the beginning was to support many types of databases. It's something we've been thinking about from day one, and at this point, we do have support for more than one database. One of the things that we like to think about when we talk about RedDwarf is I think we're at a point where we can think beyond just plain old VMs. We should be able to offer an experience that's at a higher level than a VM, so you shouldn't have to SSH onto a VM and provision software and manage things that way anymore. And that's kind of what we feel we've been able to accomplish with our REST API. So, our API allows the user to forget where their database is actually running, how many instances it may span, and maybe a replica which contains three instances. The user doesn't have to know that. The only thing that they have to think about is spending up a database using the API to spin up the database or managing it via the API, managing backups, creating schemas or, you know, additional users. One of the outcomes of such an API is that it actually, we feel it enhances the user experience. So, just as Swift lets you treat arbitrary data as objects within a container, we allow users to think of databases as resources that can be treated the same way. It's not just a VM running a database anymore. And yet, you know, there may be users that actually want to define level control. And for those users, we also have an API, basically the ability to turn on root users. So, if the way you roll is to, you know, be rude and do everything on your own, you can do that with framework. But, you know, that shouldn't be, we feel like it shouldn't be the only way. Another area that people tend to not think too much about when running databases is, you know, there's a ton of management that actually needs to happen. Things like optimizations that a user can use to secure the database when it's installed. With RedWolf, what you get is an optimal database every single time. So, whether you need a small test database for a QA or a large production database, you can be pretty confident that RedWolf is going to, you know, give you the right thing and it's going to configure it the right way. Whenever a database is created by RedWolf, it's also going to be secure. This is something that RedWolf does. It's something that you don't have to worry about. And, you know, one of the areas that HP, as well as Ratspace, that we've gotten pretty good at is optimizing the use of our hardware. So, you know, databases typically require a little memory and less CPE, right? When you run databases on standard VMs, you're probably not going to be able to, you know, run other types of applications along with databases and utilize the hardware in the right way. So, running RedWolf on that type of a deployment, you can, you know, optimize your fleet without, you know, having to turn the keys over to everything. And one way we actually use this at Ratspace is we have OpenBZ. We have a driver for a node that actually creates OpenBZ containers. So, with OpenBZ, there's security concerns, but you don't want to just give someone shell access to a container, right? And by using an actual API, we can give people access to my SQL applications, but without actually giving them access to the OpenBZ containers because there's security, you know, concerns there. So, you know, if you have an API for databases, it allows for other possibilities in the future. For example, you can actually have fully managed replication using an API like this. And you could also have, like, auto-scaled replication, failover. And these are the kind of features that, when you talk about just spending up a database on a no-bin instance, you know, it's very easy at first. But the further you get along, like, the more stuff you want to do, the harder it becomes. And so, people just want to set up a database, and they're not, they really don't want to be concerned with this thing. Having an API that just allows the ability to turn on things like replication or failover, it would really be tremendous. It would be very, very useful. We also can add stuff like cross-AZ region availability is something that, you know, HP is looking at. And the other cool thing is that if this is presented as API and makes it really simple for people to spin up databases, it would make it that much simpler to actually deploy OpenStack using OpenStack, you know, components itself. Because if you think about it, OpenStack is actually like that, a diagram that was up there where there's an infrastructure database that sits in the middle of everything. So if you could just spin up a red dwarf instance, it would actually make it even easier to sort of self-host it. Yeah, and if you guys went to the talks earlier today where, you know, I think it was OpenStack and OpenStack, or Nova Bear Metal, there's no, you know, there's no reason why red dwarf couldn't be the thing that's actually spinning up that database, the first database that Nova's going to use. Real quick, we're going to get into the architecture here. And hopefully this doesn't bore anyone, but if people out there have worked with OpenStack, worked with the code, you just want to show that red dwarf is very similar. So if you have, you know, if you look, basically red dwarf sits to the side over here. And on the right side we have, like, Nova and Swift, and, you know, all the good stuff from OpenStack. In OpenStack, you typically have these names that you're running. And in Nova, however, since that there's the message bus and the infrastructure database, which is sort of like the nerve center for the whole thing, they communicate between all of these processes. So with red dwarfs, we have the red dwarf API, which handles incoming REST HTTP calls. And then we have the task manager, which is just an RPC service that talks to the API for longer-running jobs. We have our own infrastructure database as well as message bus we use for communication between these pieces. And, you know, if you're writing this into VM, you can actually make it, those be the same thing as Nova uses. But if you're deploying it for real, it's probably good to set those up separately. Then red dwarf will talk to Nova and OpenStack on its using the REST API. And the reason we did this is this way you can actually set up red dwarf with OpenStack deployment that you don't have full control over. So it allows it to be very flexible and you can sort of set up red dwarf with your existing deployment. So to talk about the history of it, you know, if people here saw our talk at the Grizzly Summit, it was interesting because from the beginning we've always tried to make red dwarf something that's happening in the public. And we've been working with HP for a while now. But it was interesting at the Grizzly talk because we were very focused on the fact that we were both sort of using the core idea at red dwarf, but we would separate the daemons. So we'd have slides saying, hey, here's how we do stuff at the rack space and then people over here, they do something kind of different with HP. And it was sort of neat that we were able to swap those things out. But we were doing that to sort of meet a need, which is that we both had stuff we had to deploy with protection. Well, since Grizzly, work between the two companies is really, I think we've had more time for it. And HP's gone and they've switched to the public version of red dwarf. We started working on it every day. This is where all the work happens. And because HP got into it, there were some things about the public version, which we thought it was okay. We did the public version back in May. But it had fallen apart and it was rough around the edges. HP got into it and really started fixing stuff. And both companies started focusing on it. And now the public version of red dwarf is just easier to set up and use than it's been in a long time. There's been a lot of work that's gone into it. And this is where everyone at both companies is focused on. This is where all the commits that happen to red dwarf go. So we've gotten away from pretty much every issue that you might have seen before were truly dedicated to the code you see on Stackforge. Okay, so yeah, since, as I mentioned, since the Grizzly Summit, HP and Rackspace, we've both decided to converge on the open implementation. And since then, we've actually made a lot of progress. So at this point, we've got 16 full-time contributors that are contributing every single day. We've got complete integration with the OpenStack CI process. So every single commit that we make to red dwarf today is being gated by both unit tests as well as integration tests that were run against a DevStack-based install. So we have a live server that everything is being tested against before code actually ever gets merged. All of our code is hosted on Stackforge. We have the server-side pieces, we have the deployment scripts, we have the client, the Python red dwarf client. All these pieces are actually available on Stackforge today. We've also done a lot of work to integrate ourselves with DevStack. So because we're not an official project, you can't just go into your local RC and say, hey, you know, I want to enable red dwarf. But it's not that much more than that. What we've done is we've taken the local RSA approach. So you download DevStack, you download red dwarf, you copy over the local .sh and you run Stack.sh. What you end up getting is DevStack and red dwarf living in harmony. Everything is running together. We've also, since we've decided to go with the open version of things, we've changed our development process. So no work is actually getting done unless there is a bug or a blueprint filed in Launchpad. So the only way things actually get into the trunk is there better be a bug or a blueprint. So that's been a big change for us. Also, since the Grizzly Summit, when HB started looking at the open version of red dwarf, we realized that getting red dwarf up and running wasn't that straightforward. So we've actually gone through and with Rackspace itself, we fixed a lot of those issues. We made that process very easy. We've also, along the way, we've improved our documentation. So if you go to wiki.openstack.org slash red dwarf today, you're going to find a ton of information about running it, what it is. You'll even find design docs, things that, you know, as we've implemented features, we've made sure that we document why we made the decisions we made. We've also been busy working on features. You know, the two new features are the two that are actually are in trunk today. User level coders and rate limits. And we've got patches already submitted, they're not fully merged yet. So that's why the next two features are in progress. You've got the support, we've got the ability to do manual backup. So take a backup of your database and, you know, a bit of a point in time and restore it to that point in time. We've got notification events. So as red dwarf is doing things, it'll, you know, send up events to RabbitMQ. And we've also, you know, one of the things that we've improved a lot upon is plugability. When we first started, during the Grizzly Summit, the only database that was supported in red dwarf was Oracle's MySQL. One of the requirements for HB was that we run for Kona's version of MySQL. So we've improved code, we've made code a lot more pluggable to be able to take any database and plug it into red dwarf. And it should be fairly easy. So whether you want to put it in MariaDB or Postgres, at this point where, you know, the code base is such that you could easily do so. In terms of backups, when we implemented the backup feature, that was another big thing for us was, you know, we've got a couple of tools that we want to use for taking backups and managing them. But adding a third tool or a fourth tool shouldn't be that much more difficult. So maybe some of you are out listening to this and thinking, okay, this seems kind of neat. And it seems like they're working together, but how do I know this isn't just deployed? Like, how can I address this? Well, to throw some stats up, we have about 285 user tests. We have 222 integration tests that run in bait mode, and you were to pull down the code right now and just run talks. You could run all of these integration tests instantly along with the unit tests. And then of that same suite of integration tests, we have about 144 of those integration tests that are running in VM gating. And what VM gating means is that we actually set up a VM where we try to get everything working as close to real life as we can get it. So we actually set up NOVA for real. We call desktop, in fact, to set it up. We try to get all the actual features working to make sure we build an image that has, you know, a good guess on it. And we try to basically get it to work like it's going to in real life and run the test again. So we're able to run stuff really quick, but we also are doing our due diligence to check that it's, you know, going to have no bugs that might have been from the real concerns that connect or when we're interacting with those systems complex as NOVA. We have 16 developers right now, like people just said, and all of them are running a VM, right? And so they're constantly setting stuff up, and so these deployment scripts are running two different companies. Okay, people in different time zones now are actually working on this thing and making sure that it functions. So you're going to have more luck now than I think people have had in the past, like when they tried to download this, and it was just sort of the public version. And most of the API features, and along with major regressions, can actually get tested just when you pull it down and run stuff instantly. So actually, all the XML testing happens just when you pull it down Now, one cool thing is that if you, you know, from an open stack, we've had XML and the API sort of from the dawn of the whole project. But there's sometimes some grumbling that opens that XML isn't treated as a first-class citizen, but between the two APIs, like JSON might work really well and XML is kind of flaky. And we all like JSON here, I think, which is the reason for that. But I'd rather do it for trying to make sure that XML does get the short strip. Make sure that it works. We're fairly confident that the XML quality is going to be very high. And also, there's Pepe. Okay. So plans for the future. Well, right now we're on the very cusp of getting manual backups in. That feature is almost done. What we'd like to focus on now is automated backups. This is where having a cloud database makes a lot of sense, because you just provision a cloud database resource to sort of back this up to Swift. So you want to have these things backing up constantly and automatically and make it really simple. We also want to have point-in-time restore. So let's say that your WordPress blog gets hacked on a Sunday. You can go back to Saturday when you think that the database is still okay to restore from that exact point in time. Additionally, we want to look into replication. This is something that I think we've been talking about since we first even looked at having a cloud database type of resource in both companies. Replication is something that's really going to be exciting when it's paired with an Agile like Red Dwarf and it's made easy to do. Another thing is that if we're compatible with OpenStack, we want to make it very similar. So we have the DevStack integration. It would be really cool to get heap and horizon support as well. So being able to manage applications, stacks, and heap would be really pretty cool. And of course, if you could like see schemas and stuff in Horizon, that would make things easier for some people. We also want to support more databases. So, HP came in and they had Purcona and they did a really awesome job at it. It seems like it should be easy to add other things like MariaDB or even Postgres going forward. The core architecture of Red Dwarf was designed to make this kind of thing possible. And it seems like that's where we want to look at that stuff. And finally, it would be really cool if somehow we could provide all these to OpenStack users just very easily. So we want to look at getting incubated. Alright, so one thing that's pretty obvious, I think most of you agree, is with web applications these days there's a pretty uniform architecture. So you have your web tier that handles requests. You have some messaging tier that may be performing asynchronous tasks. And you typically have a database to store that data you care about. If you look at the OpenStack architecture it also falls in that same category, right? You've got the Nova infrastructure database. This is something that should come for free now. So it seems like every application out there requires a database. Why isn't OpenStack something that OpenStack doesn't seem to ship with a database sort of solution? Why is that the case, right? It shouldn't be an afterthought, we think. It's something that developers are going to need anyways. So it only makes sense to actually bundle it with the rest of OpenStack. When we wrote Red Dwarf we wrote it with that in mind, basically. We want to feel that we have a product that complements OpenStack. It actually enhances the positioning of OpenStack as a cloud. And it means that it's going to meet the needs of most of the developers because database is going to be a requirement for most developers. Red Dwarf is also growing. We're growing in terms of features. We're growing in terms of usability. And we're increasing our quality every single day. HP and Rackspace were both fully dedicated to the project. And we wanted to be successful. And we hope that Red Dwarf is the thing that's going to be the way to openStack. The project is very active with two companies on board. We feel like we have that critical mass to make these things successful. We're dedicating more resources. And we're trying to grow this project as fast as we can. If you're interested in this and I hope, I don't know, some people have become more interested in this talk, go to your hotel tonight and talk. What are you missing, really? Try installing Red Dwarf. Try to play it. We had it set up in a VM. Go ahead and test it out. We really think that it works a lot better than it has before. And we all compete back. This isn't super simple, but it's not rocket science either. And if you're very interested, join as a contributor. That can either be adding stuff or maybe adding bugs or new features. Get active on the wiki. Make sure that this project, if you're interested in relational databases, make sure that it goes the direction you want. So join us. Finally, we've always talked about the Pound Red Dwarf IRC room. Pound Red Dwarf on a free note is where we're home where we chat all day. We've talked about this from the very first time we ever mentioned Red Dwarf and OpenStack on it. One big change, one big change, is there's actually a lot of people there all the time. If you logged in right now, you would see members of both companies that aren't at the summit that are talking about things. Before, there's always, I think, a few people in Red Dwarf but there are probably bouncers. We're there all the time right now. If you have anything you want to ask about, there's no need to wait for these summits to talk to us in Q&A. Just get on IRC and there's people that would be thrilled to ask your questions. If this seems like, if you agree that relational databases wouldn't have a first class, be considered first classes in a cloud, if that's something that you actually think is a good idea, let your voice be heard. We're trying to get this incubated and if you support something like this, just let us know. Let the community know. Thank you for your time. We can hear you. You talked about relational databases a couple of times. Yes. How, if I'm just running Mongo or any other non-SQL database, all the same things that RedRock was providing? Yes. At that point, you're talking about making an API that can handle either a non-SQL database or a SQL database. Any data store really has all the same asks for the most part. Why is it really limited to my SQL? It's in play. When we think about this product, we want to make it so you can actually make it a bit richer, right? Not just provision and set it up, but also users and CMOS. And if you think of something like a non-SQL database, it's going to make that API very, it's going to give it a split personality. So it makes sense to sort of focus on making it for relational databases. Now, about a year ago, we had the San Francisco Summit. We talked about making RedRock something that could deploy a lot of other stuff. But it really seems like the sweet spot for this is relational databases. People on the team that are sitting here, they may disagree with that. That's just my view. I think relational databases is what makes those sense. If you try to just make an attraction for anything, it seems like that would get kind of messy. I would also say we should chat after we talk because some of us, and talk to this guy Daniel Morris, I think that as a group we're interested in what your asks would be whether we could accommodate something like that. I do agree that there is a lot of overlap. At least we use the colors. Something that's different from the API. Sure, and that's an absolute possibility. I would say to me, the APIs, when you talk about database services, the APIs only diverge of relational versus non-relational. It only diverges when you think about running XYZ resource in a VM versus running in more like a massively multi-tenant type of client. If I'm going to throw Mongo or Redis inside of Redworth instance, that's still very similar to me. Well, you have to back up Redis and Mongo. There's gratification for those as well. Now, yes, they're handled differently than they cover. The abstraction layer from the API seems very similar to me. Now, if you're thinking that you request resource or anything that happens on the back end, that's a very different API than some database management system that's inside of you. I personally think that it can be done, but that's my wishful thinking, but I'm going to fight for it. So, Daniel works with us at Rackspace on product, so if he says he's going to fight for it, I'm going to give him a spoke. Yes? Okay, so in the public right now we have a reference agent that is implemented in Python and it basically talks over RPC, over the message box. It's actually the same muscle code. So, it was actually patterned after the old Nova managers and when it you start up a new Redworth instance that's going to start a Nova instance and it's running this gas agent and it's going to receive these RPC calls. And so, for instance, if we resize the database, that in Nova could actually mean that it's going to make a copy like the compute instance and it's going to ask you if they look okay. One of the biggest challenges of the gas agent is just to get stuff from the API so it can list users and you can add new schemas, but it also checks the health of the actual app that's running to make sure that it's okay. So, in Nova, we actually had like a copy there and it asks you if it's looking good before you blow the old one away. Because we have this gas agent there we can actually see if MySQL is healthy if we can ping it and things are still working before we make that decision. So, that sort of is a job where you're more curious in like how it was constructed or just what it's done on the software. Oh, so when you first provision an instance it creates a user on the database called OS admin and it can only be used from that actual compute instance and it does all this communication of this user. The local database, yes. It's also something that's pluggable so, I mean, there's other implementations there's a possibility I mean, it can be written in anything we eventually also want to abstract out the way it communicates right now it's hard coded to the message bus and talks directly to the infrastructure database but, you know, obviously as well as other people feel that that's not the best approach, the quickest approach so those are two things that we want to abstract out as well maybe use other technologies that are more scalable. Yeah, that's something that we've been looking at a lot because it is sort of strange to use that same message bus for everything. And some companies may have a requirement that the agent not talk directly to the database that's already on the account that you now connected to the same database so I mean, going with the same plug-and-play kind of model we've got we would probably just recommend that another implementation be created that access and compute things via what we were talking about there's some file or something that might listen to that in that way so I mean yeah, there's definitely okay, well then you obviously know there's definitely other ways so I the reason I was asking is are there other companies that have that any concern? I think, you know, we're going with implementation at this point, we have implementation that requires access to the database to be able to manage it and if there is a requirement to not log into that database we'd have to have another implementation against the agent. You know, I'm a developer so take my estimation with a grain of salt but I feel like that's something that we could do right now you know, so the right space we're using is C++ agent and every time we get a feature done we make sure that there's something that's working in public and the reference agent and it should be very possible to make it so that it just wouldn't write it just doesn't prohibition that I was having a user and said it just pains it for the health because there's other times when the guest agent is useful you know, just take those ability to like, you know add users and see them as they take that away Hey, yeah, add a blueprint I think it's possible. So one thing that we do today is that when you resize an instance we actually select a different MyCNF input file that we install like so we stop the database resize it and then actually change the MyCNF file going forward I think that that's in you know, that's the idea that we're going to be able to do that in the future and we're going to be able to do that and we're going to be able to you know, that's the idea that we're going to be able to tweak things like that Actually, yeah, I've just got a blueprint right now to be able to change settings on the actual MyCNF Yeah, right Yeah, and I think that I don't think it's been fully decided whether we're going to let the user completely shoot themselves in the foot because I mean you can obviously exhaust the resources quite easily if you just set the connection count to infinity minus one You know what I mean? Assume 10% because maybe they put their buffer pool at 60 to 70 Take the remaining 10% of memory You can see in the Amazon if you look at the parameter groups they've got an algorithm that takes they're going to say the average thread is going to take about 10 to 8 and figure out how much the remainder of that memory will allow you to max out at and it won't let you go above that And I think that is something along those lines that we want to ask you about the RedNorke chain These guys in line Well, yeah Okay, so what's like What do we need in this? I guess last question What about when we're running RedNorke and my own data center on my own private one-minute loader session? That's impossible No, I'm just kidding, you can do that You pull down the code and you can totally do it Okay, and then after that Is Rackspace moving towards reference implementations for there? We are actually on that I think in production where a few will be in line right now Other reference guests? No, I mean the reference guests We're everything other than the guests Okay, thanks everyone Thank you