 Okay, so my name is Amrit, I'm the PTL for the Trove Project and I've been working on Trove since about Atlanta, Ice House, somewhere between Hong Kong and Atlanta if you will. And I work for a company called Tesoro, you can see the, that's the company I work for and we work explicitly on Trove and we have a product based on Trove. If you're interested in hearing more about our product, you should come to our booth, which is downstairs, this presentation is about the upstream project, not about our product. So before I go any further, how many of you in the room have used Trove in some way, shape or form so far? Okay, how many of you don't know what Trove is? Yeah, you're not qualified to do that. Okay, so I don't think, I don't know why this thing keeps going off by itself, but for those of you who are familiar with Trove, this stuff should be pretty straightforward. Trove is the open-stack database of the service project, it supports both relational and non-relational databases, important thing to know. For those of you who are familiar with database of the service in the context of some other clouds, Trove does a lot more than, for example, just RDS, it does a lot more than just RDS in the sense that RDS is strictly relational databases, Trove is relational and non-relational databases, and the important thing is that Trove is a framework which is highly extensible and supports over a dozen databases, the list is at the bottom. By way of history, development on Trove started somewhere in the grizzly Havana time frame, it got incubated around then and got integrated in Icehouse and has progressed very, very quickly after that. This presentation gives you somewhat of an overview of what Trove is, tells you some of the capabilities of Trove, some of the architecture of Trove, so you know what I'm talking about when I say these are the things which Trove supports and then looks ahead at what we're looking to do in future releases. Okay, let's make this kind of an interactive session if you don't mind, so if you have questions at any point, raise your hand. We've got about 50 minutes, rather than pushing all the questions to the end and finding we run out of time, let's take the Q and A right then and there. So, without further ado, one very important thing to remember about databases of service is it is not just about provisioning. Provisioning is an important thing. It is very important that a user of a database of the service be able to quickly provision a database. That is the first thing which everyone sees about database of the service. The old model of I want a database, wait for three months and you'll get one, just isn't going to fly, getting a database quickly is very important. But, unlike other parts of an application stack which are transient, which come and go in a production environment, the database comes and stays for a long time. And over the entire lifetime of a database, there are a lot of things which you have to do. Provisioning is just this one little aspect at the beginning. And the important thing about databases is more peculiarity about databases and the fact that they're long lived, is that if you screw up the configuration up front, you pay that penalty for a very, very long time. So you want to make sure that you get it right up front and then you manage it in a consistent way along its lifetime. That's what database of the service does for you. So how many of you have ever had to deal with a problem with the database at two o'clock in the morning because you forgot something when you configured the database the first time around? Oops, I forgot this in the config. Anybody? Okay, yeah. Yeah, you don't want to get that phone call. So you want to know for sure that when you hit the button, you say, give me a MySQL 5.6 database, it is configured exactly the same way every single time. Those are the kinds of things which provisioning gives for you. But once you provision it, databases have got important data. They run on a machine which is potentially compromisable. You need to make sure you're up to date with the latest security patches. How do you apply those in an automated way? Well, I have one database server, not very hard. I have 1,000 database servers, 300 running MySQL, 200 running Mongo, 150 running something else, all of them running Ubuntu, some of them maybe running Red Hat, whatever. I want to roll out a patch set to all the MySQL instances. I want to roll out a patch set to all the Ubuntu instances. How do you automate that? That's something which Trove can do for you. Databases need tuning, configuration parameters. Every database has got hundreds of these. How do you manage these configuration parameters in a consistent way across multiple database instances? If you're operating a service, a database as a service, within your organization, you might want to apply a single configuration parameter change to all 500 MySQL databases. Great, Trove can do that for you. At the same time, you want to limit the number of configuration parameters, your end users can change. Good, Trove can do that for you as well. So it takes something which is a well-known database like MySQL or Postgres or Mongo, and it converts it into something which is operable as a service, something which you can offer as a service and operate as a service. That's what Trove does for you. And it does this over the entire lifecycle of the database. From the time when you provision the first one, all the way to the time you tear down the application. And this is not just for single instances. This is a single instance of MySQL for development, a Mongo cluster for testing, Cassandra cluster for testing, or a production deployment of, I don't know, MySQL with Prokona X or DB cluster. All of these are supported. So if you notice the previous slide, we listed the databases which were available. This is alphabetically sorted. It's no other priority than AlphaSort. Single instances, in many cases, clusters. Redis as a cache or as a persistent store. Vertica for data warehousing applications. Our friends from IBM added support for DB2 Express as well, and so on. No matter what database you have, it's probably supported already. Okay, question so far, anything? The question was, any reason Oracle is not there? The reason is this presentation is primarily about the upstream project. If you're interested in support for Oracle, come see us in our booth and we can talk about it. Same thing for other commercial databases. So just a quick primer about, before I get into the architecture of Trove itself, this picture should be pretty clear to everybody who's been to any session on any OpenStack service, but just in case, every OpenStack service looks something like this. The service will expose a RESTful API endpoint. Every interaction to the service should be through that endpoint. Everything in here is the implementation of that service. There's a service, something which implements a service API, typically Whiskey or something like that. And then there are some components which actually do the things which that service wants to do. So Nova has schedulers and all of those things there. And all of these components chat amongst themselves over some message bus. In OpenStack terminology, this is typically abstracted as Oslo messaging. So this could be RabbitMQ Oslo messaging, wrapping it, some such mechanism. And every service has some amount of persistent state. It's stored in some database. This is typically a MySQL database. If you're deploying OpenStack, you might have NovaCinders with Glance Neutron Keystone sharing an infrastructure database. That's this one. They might share a RabbitMQ server, that one. These might be, these are typically going to be HA configurations, multiple instances of the database, a HA RabbitMQ cluster, what have you. But this is what a typical service looks like. And the important thing to remember is there's this very clear fence around the service. And that's there to indicate that the only way you're supposed to talk to the service is on that restful API endpoint. If this is Nova, another service should not go in and write stuff into this database, bad. If you wanna talk to Nova, talk to the API endpoint. Okay. This is a structure of almost every OpenStack service. Very, very small exceptions. One, we're gonna pass over that, that's a Cilometer, which has a different structure. We'll ignore that for the present. This is how most services are. Any questions about that? Okay. So, let's extend that picture and say this is what Trove looks like. And just to keep everybody on their toes, I kind of flipped everything left, right, sideways. On the right, the Trove service. In the middle, the OpenStack services we depend on. And on the left, some other stuff which we're gonna talk about in a second. So, first thing to remember, Trove is a service which depends on these other services. And this picture as of Newton needs one extra box here, which is Mistro because we now support scheduled operations and for that we use the Mistro scheduled scheduling framework. But let's assume there's a box here which says Mistro. Trove exposes an API, that's the API. When you want to talk to Trove, you talk to that little blue dot over there. The actual implementation of much of the hard work in Trove happens here in the task manager. And there's a message queue and some persistence state. A standard OpenStack service. Okay. If Trove wants a compute instance of any kind, it talks to Nova. If it wants a block store, it talks to Cinder. If you hand Trove a Keystone token, it'll validate it with Keystone. For networking, we use Neutron or Nova networking and we store images and glance. All the standard OpenStack services. So let's talk about a simple request to Trove. Trove is used for databases service. So what kind of things would you do? Give me a MySQL 5.6 instance. Give me a MySQL 5.6 instance. Now that's an authenticated request. So your client is gonna go, go to Keystone, get a token, hand an API there with your Keystone token, give me a MySQL 5.6 instance. And oh, by the way, I'd like it to be on a M1 large instance with 200 gig of storage attached to it. And I'd like it to be tied to this particular network. So Trove create, give it a name, flavor M1 large, size 200, and the other parameters you need. So the API service is gonna look at that, take your Keystone token and say, is that a valid Keystone token? So zoom it is. You want an M1 large. Do you have the quota to do an M1 large? You want 200 gig of storage. Do you have quota for 200 gig of storage? Let's zoom all of those checks pass. You'll get a response saying, great, your instance is being created, come back later. Because an instance creation could take several amount of time and that's not a synchronous call. Okay. Message goes over the message queue, the task manager gets it and is gonna process it. The task manager is gonna work with all of these other services and go and actually get you your database instance. So I said I want a MySQL 5.6 instance. So now let's talk about this side of the picture. Yesterday we had a hour and a half session about what these images are. But in order to get a MySQL 5.6 instance, one of the things you give Trove is a guest image which contains operating system of your choice. MySQL 5.6 and a little component of code they're called the guest agent which we'll get to in a second. So you package the stuff up, put it in glance and you tell Trove, you tell Trove over here, if anybody asks you for MySQL 5.6 instance, that's the one to use. And you can have another one which says it's a Mongo version whatever and another one for Percona whatever. You can have a whole bunch of images here. Okay, so far. Not if it's okay, shake your head, otherwise if it's not. If you don't move your head, I assume you're sleeping. Okay, all right, I know who is. So I requested a Trove instance M1 large with that image. Nova is gonna come over, pick this one up, Buddha VM. Okay, so this is now a virtual machine. What's on the virtual machine? The operating system I chose. An attachment to 200 gig of storage which is Cinder volume. The actual database, MySQL 5.6 and this thing called a guest agent. So what the hell is a guest agent? So Trove supports a dozen or so databases soon to be a couple more. And one of the things we support is a notion of say a backup. Trove backup create this. How many of you know what the exact command is to backup of MySQL database? Okay, do you also know the command to backup a Mongo instance? Cassandra, couch base? No, okay. So that's a database agnostic API. All you need to know is Trove backup create. Don't care what you actually have to do for the database. The guest agent does. When you need a backup, Trove backup create, the message actually makes it way over to the guest agent which will implement the database specific best practices for a backup. Okay, so far. So you've got a VM up and running. You have an application. Remember, the application talks directly to your database. Trove is not in the data plane business. Trove is entirely in the control plane side of things. How many of you are familiar with DynamoDB? No, one, okay. DynamoDB is in the data plane. Gets between you and the actual, between your application and the data. Not the case here. Your application talks directly to the database. If you're talking to MySQL, you're typically talking here on port 3306 into this VM. Which basically means that Trove would have gone to Neutron and created a security group, tied it to this instance and opened one and exactly one port, 3306. Similar port for Postgres. Some other port for Mongolia. Ports not included in that list, 22. You cannot SSH into that instance. That is secure, it's managed, what goes on in there is not your business. We don't want you to get into that instance. We want you to talk to the database. It's a managed database. If you're an IT person offering a database as a service, the last thing you want is your end user getting in there and starting to do random shit. We make it possible for you to put a very clean fence around it and say, you wanted a database, there you got a database. Go talk to it. Create tables, insert data, select query, do those kinds of things. Don't get an on port 22. We good so far? All right. So database agnostic stuff up here, database specific stuff implemented down here. We talked about a single instance. Now think about a cluster. Trove cluster create Mongo with six data nodes, three query routers, two config servers. Bang, it shows up there. And oh by the way, each one of those needs to have these flavors and this size of storage. Fairly large little JSON object comes along with that request, keystone validate. Good. Do you have the quota to do that thing? It says, you know what you do? The next thing which Trove's gonna do is actually get the task manager to go speak to NOAA and provision whatever the half dozen instances you need with the appropriate images. Once those instances all come up, there's gonna be a half dozen guest agents which are gonna talk amongst themselves and say, you're the config server, you're the query router, you're the data node. Tie it all together, make a cluster. And when that is all done, you have a running Mongo cluster. So you don't have to actually go and provision seven nodes yourself and tie them all together. Trove does that for you. Okay, those are some of the things which Trove will do for you. So with that as a quick primer on the architecture, let's switch gears and talk about what we have in Newton. So Newton was actually a fairly substantial release. Some Stakelytics numbers here for you. Fairly diverse community, 28 or so companies contributed to it, you know, 340 commits and a whole bunch of contributors from all of these companies. So I recognize several people here who are contributors so I'm not gonna embarrass you by asking you all to stand up, but it's fine. So what are some of the things which are in Newton? I talked about the fact that Trove is database as a service. So one of the things which you do have to do with databases from time to time is upgrade them. So we added support for upgradability with a new upgrade API. And the way in which we do upgrades is tied to the way in which we build guest images. So we have a database instance running. The upgrade of the database involves upgrading that running instance using the Nova Migrate API. Again, we're a consumer of Nova services. We're using Nova Migrate under the covers. And since this is a service, and since I told you we don't want people SSHing into the instance, we don't want you to have to SSH in there to go and look at query logs or Cassandra's log as the case may be here. So we added some capabilities to retrieve those and those are a whole bunch of usability improvements which we had here. How many of you here have used Nova from the CLI and had a problem when you create a Nova instance? Anybody? What do you see when you run Nova Show? You see a big stack trace with all the problems. We didn't used to have that in Trove, now we do. So when there's an error on create, we take the actual exception, throw it into a database. The next time you run Trove Show, you can see that usability improvement. Used to be much more fun debugging without that, but all these guys take away the fun now. One of the things which we did do was for integration with New Relic and a capability we had in the previous release called modules. As an operator, you have the ability to cause certain pieces of code to be injected into a running and a guest instance at launch. And you use that for example and in this particular case for New Relic. How many of you are familiar with what New Relic is? No? Okay. New Relic is a monitoring service for MySQL. If you have a New Relic agent on your machine and you're licensed, when you spin up a MySQL instance, it will pop up on New Relic's management dashboard somewhere. With this capability, you can specify a Trove create. Well, as an operator, you specify when a MySQL instance is launched, inject the New Relic module, which means that when you run a Trove created, it pops up automatically on your service dashboard. That's a new capability we had. And the last one, for those of you who've used Trove for a while, you're gonna love this. If you ever provisioned an instance and it got stuck in creation during the creation state, it was pretty much stuck. This is the equivalent to Nova Force Delete. We now have that. We support not only single instances, but also clusters. So we have some improvements for clusters, specifying modules during, we just told you what modules were, specifying those during cluster create. In the previous release in Mitaka, we added support for volume types. How many of you are familiar with this whole concept of volume types? Center volume types? Okay. So for those of you who are not familiar with this, CENTER supports this notion of volume types. Spinning disk, SSD. Spinning rust, SSD. Spinning rust is cheaper, not as highly-performance. SSD is more expensive, better performance. You can define them as effectively two classes of service. In CENTER. You can now provision with Trove instances and say, I want my volume to be 200 gig and please give me SSDs. You can now have the ability, therefore, to control who can get access to SSDs and who can get access to spinning rust. You can assign that as an attribute of the data stored version and say, this data stored version shall use SSDs or cannot use, you can basically enumerate the kinds of algorithms. And again, for clusters and for replicas. So we've talked a little bit about clusters. So for things like MySQL, master slave replication has been supported for some time now. Now we have the ability to support affinity and anti-affinity. Where do you want the replica to be placed? It is considered to be a very, very good practice to have your master and replica always on the same physical machine. Not if you agree. Okay, you're not supposed to nod for that question. Okay, anti-affinity is what you want. Okay. Our folks from IBM added some additional support for DB2 Express configuration groups. We didn't talk much about configuration groups, so let's talk a little bit about that. Every database has hundreds of configuration options. MySQL's got its collection of 100, Mongo's got its collection, Oracle's got its own collection. And you as an operator might decide that there's only 30 of the options for MySQL which you want your end users to be able to twiddle. But you as the operator want to pick another 40 and you want to send some specific defaults for. Great. Trove supports that capability in the form of a little template which you put on Trove and you say, when you provision an instance, blast this down to my.cnf or wherever the appropriate thing is on another database. And then there's a nice little way in which you can say, I'd like to change this parameter. Please, of course, everything goes with a please. And Trove says, I know that this parameter requires a reboot, so I'll let you change it but then I'll mark the instance as requires reboot. Or this parameter can be changed hot in which case it just continues as it is. We now have support for that in DB2. Earlier DB2 only supported offline backup now. It supports online backup. We added replication support for Postgres and I talked a little bit about quota management. It's important to remember that Trove's quotas are independent of the quotas on the underlying services. Trove has a notion of its own quotas. There's now an API to manage that. For those of you who've used Trove before, this was a little bit hard for you to change. You had to go and muck in the database a little bit. Now there's an API to deal with that. It is an admin API, though. So from the time when I started working on Trove way back in Icehouse till maybe Liberty and Metaka, the majority of the things we've been working on were things which I would consider to be table stakes. If you want to say you're offering a database as a service, you've got to have some of these capabilities. Nobody in their right mind will put an application in production on MongoDB with our clustering. Fair enough, right? There was a point in time when Trove did not support Mongo clustering. So we worked on things like that. Now we've got to the point where we've got, I think, a pretty good list of features that for the vast majority of people looking to use databases in their infrastructure, Trove has the capabilities which will meet it. So that's a long-sentence way of saying is Trove ready for production? Yes, I think it is. And I see in the room a bunch of people who are in fact operating Trove in production. We support a dozen databases at least. With Tesoro we add a couple more. And most of the features in this matrix, and by the way, when we say that we support clustering, it does not mean that we support clustering on every one of those 13 databases. So if we say we support configuration groups, like I said on the previous slide, we added support for DB2 configuration groups in this last release. So we're filling these out in this matrix, one data store, one feature at a time. We've done pretty well in that, and this matrix is now fairly full. Looking ahead, the kinds of things which we're trying to work on are usability improvements. So we've spoken with, right here at the summit, and when we speak with customers and prospects, I hear the kinds of things which people want to have. We also have contributors who are also users of the project. One of the areas where people said they have a lot of pain is building guest images. If you follow the OpenStack mailing list, about once every month there'll be somebody there who posts a question saying, I can't seem to get this to work. So one of the things which we wanted to finish in the Newton release, which we did not finish in time, was to make it easier for people to build guest images. The default way in which we build guest images is with this image builder. There is a set of non-production ready elements as part of the TRO project now. We're gonna package them up a little bit better so you can build development and testing images which you can start using, at least as a starting point for you to build a production image for yourself as your operating TRO, or as an end user if you want to use TRO. There are a bunch of improvements to clustering. I'm not gonna go into all of them here, but feel free to catch me after the session if you wanna know whether your favorite database supports any one of these features. As people start rolling TRO out into production, one of the things they talk about is we need support for HA, and multi-region support is one of those things which comes up from time to time. There's a blueprint and some code up for that right now in review, which we hope to have landed in the Ocata release. There is talk about support for additional databases. We can talk about more details about that. Primarily in the upstream, what we're gonna do in that area is to add support for different new versions of some of these databases, potentially not net new databases in the release itself. One of the things which we started working on a couple of releases ago was to have better integration with storage. For those of you who have used TRO for some time now, you probably know that the booted database instance has a single attached volume. This is not a best practices for most people. You do wanna have multiple attached volumes, and that's one of the things we plan to fix in the next release. Most of the existing deployments use TRO in virtual machines. TRO does work in bare metal, kinda sort of a little asterisk there. It doesn't work too well in containers right now. Both of these are areas which I'm particularly interested in, and if you happen to be interested in TRO in either of these environments, please definitely let me know. I'd love to hear what your requirements are in this area. So quick show of hands. Anybody here give a crap about TRO about bare metal or TRO about containers? Okay. One, two, three. Could we chat after this session please? Okay. And also, how many of you are scared about this whole thing about Python 2.7 going out of support and you really wanna get to Python 3.5 like soon? Anybody soon? Okay. All right. We do have to get to 3.5 at some point. I just wanted to know whether we're gonna have to do it in Okada or not. Sounds like the answer is no. The last one is just a catch all for a whole bunch of things. So between Ice House and now, when we've been making very, very rapid improvements in TRO, we've been making some decisions which usually come with the caveat. We know this is not the best way to do things. We'll fix it when we get to it. And that list is now fairly long and becoming a little bit troublesome. So there are some of these things which we're gonna have to pay down. Like the guest image creation thing. It was always one of those, we'll do those one of these days. We're just gonna have to pay those down. So there's a fair number of these things which we are doing in Okada. If you were in the session this morning where we talked about some of these, you know where these are, these have to do with the way in which we're doing ongoing CI based testing and things like that. Mostly of interested developers but maybe not so much to end users. So that's about all I had in terms of prepared slides. We have about 10 minutes which I figured would save for any questions at the end but I'll leave the slide up and let you look at it. If you haven't been to the Tesoro booth yet, it's like there, it's number 33 and I think they close at four o'clock. So if you get there really quick, you might even get a T-shirt. So with that, any questions? Question is how do people do charge back? So, okay, so when you provision an instance, when you shut it down, when you do any of these things, task manager knows about it. It emits events into salameter. Anything which consumes events in salameter can be used to do your charge back. So at this point, we don't do anything with charge back in Trove. You need to have a consumer down in salameter land which will pick up those things and deal with it. Not that I know of. I don't know of anybody who wants to add it in Trove and I don't know of any other service which, any other open stack service which manages its own charge back. Yeah. It generates the messages. You can consume them with whatever you want. So yeah, it's a good point. So if you're not running salameter in your cloud, you don't need two for this. The messages are gonna be generated. You can. So you remember I'd said, you know, every open stack service looks like this with one exception, which is salameter. All the salameter exception is interestful API. It's just a message. You just dump the messages. Other questions? Yes, sir. Do we support? Ceph is a backup store, yes. So the question was, do you support Ceph as a backup store? The answer is very simply, we support anything which looks like Swift. So if you have Ceph, I believe the answer is, yes, with Rados gateway. Is that right or wrong? Yeah. You'd raise your hand at some point. I don't know if that was a question or no. So the question is, is there any plan support for NDB cluster? I haven't heard of anybody who is explicitly Evan's interest in it. You are. So here's a suggestion. So this is a upstream presentation and the answer for some of these questions is gonna be, we would love to have support for NDB cluster. There's a pretty good framework for MySQL. There's a pretty good framework for clustering and extensibility to NDB cluster should not be a mountain of work. It's probably a question of putting the right pieces together in the right place. But I will caveat that by saying, if you do want support for NDB cluster, I think the thing which is gonna come right after you add support for NDB cluster is, my performance sucks, I need bare metal. So if you do want to add support for NDB cluster, sign up for bare metal support right now. Okay? If you'd like to do both. Great, and this morning you already signed up to do one thing so we'll just add it to the list and put it on your tab, okay? All right. Other questions? Thank you very much for coming. Still not too late to get a t-shirt.