 Thank you very much for that kind introduction. Hello, everybody, and welcome to the presentation today. You know, this is a interesting topic for me because I think this is something that has been very near and dear to me for actually a couple of years, you know, just being kind of involved in the Kubernetes space from, oh gosh, way back, I was at a company called CoreOS, we were doing lots of stuff, you know, and when I first saw what happened, you know, like watching a database go on top of this thing, I just thought it wasn't even possible. And then extending this today to like my understanding of how you can actually use Kubernetes in a way to not just service up like resilient, you know, systems, but to think through kind of what auto scale looks like. And, you know, what does it mean to have charge back and consumption models that are very different? So the talk today really is about, you know, how do you look at your own applications or services that you're building? And tease them out so that they could possibly become serverless. I'll talk, of course, about CockroachDB along the way. It is the app that I work on here at Cockroach Labs. So this is really kind of a combination of kind of database Kubernetes and serverless here over the course of the next, oh, I would say about 30, 35 minutes. And as the moderator noted, if y'all have questions, please, gosh, by all means, throw them into the chat, I'll be monitoring them along the way. And I'm happy to take questions along the way. I just really, I love questions. So I'm Jim, I am a principal product evangelist here at Cockroach Labs. On Twitter, I'm just James, I'm GitHub, I'm James. I have that at lots of different places. This is Intermediate. And I presented this conversation, actually I presented this deck about four hours ago to developer week in Europe or something. And I had a, there was a partner on here with me, my friend Rain. So that's why the stuff on the right hand side, the pronouns are we, but it's I here. I'm not a database expert. I work with a lot of really amazing intelligent people and I learn a whole lot from them. And I can talk about it up and down in lots of different ways, but man, expert is a wholly other level. I'm curious about this stuff. I love tech. But honestly, I think the concepts that are presented in this presentation and some of the other talks that we've done around distributed systems, I really think is cool. And I think these are the concepts that I'll really define in our future. I've been involved with the Linux foundation and the CNCF for years. And I really feel they're on the cutting edge of a lot of this stuff that's going on. I think that's why a lot of us are attracted to this. And I looked to these governing bodies as shepherding these things so that they do end up in our future. So I think again, I think our host here. So this is some high level context and high level concepts. Like I said, I'm gonna be talking in the context of Congress database, but I'd really like this to hopefully help you think about how you kind of think about your application or service and what goes where in terms of, how do you get scale out serverless type architecture for your app. So let's just start with the word serverless. Now, when I first saw the word, I kind of was like, oh, that's kind of a weird name. I think it's infrastructureless, honestly, because I think of infrastructures. I guess everything's running on servers somewhere, but there's the things that we do. But I think to get a sense of what serverless means for all of us as app developers or just people who are building things, right? I'd like to go to the past. I grew up with this rotary phone over here on the left-hand side. I didn't like friends who had a lot of zeros in their telephone number because it was really painful. I don't know if some people probably have never even seen those things, but a telephone is a telephone. And long came wireless technology for our house and then really for broad cellular technology, right? And so the whole wireless principle was applied to the telephone. The telephone is still the telephone. It's just wireless now. It still does the same function allows us to communicate with people. It's just wireless now, right? And so we think about the concept, the concept like the core item hasn't changed. However, the way in which we deliver it has. I guess, yeah, I don't know. iPhones and our smartphones are a little pretty far away from phones, but in fact, I hardly even use my phone as a phone anymore. So it loosely works. Electric is another kind of way I think about this. You know, I think you look on the left-hand side, it's like, I understand this engine. I could troubleshoot it. I could work on this probably with my brother. He's a bit better at engines than I am. And you look at this middle one, well, there's lots of servo motors. And, you know, like this is the kind of my car now. I don't have no idea what's going on there. And then on the right-hand side, well, that's the motor and electric car. I have no idea what's going on. It doesn't matter, a car is still a car. Like to the user, to your audience, the people who are actually gonna actually engage with this item, car is still a car. It gets me from one place to the other, right? It has steering wheel, it has four wheels. All right, maybe it won't have a steering wheel eventually, right? I mean, auto-driving seems kind of interesting and fun, right? So we're taking electric and we're applying it to the principle of a car, right? Or the core item of a car. And, you know, I argue that the same thing is going on in the serverless. And if we think about serverless, and what are these principles that were a time, that were tying to our applications, I think of this as a paradigm, right? And it really comes down to, you know, little or no manual server management. And there's always gonna be some level of management when we deploy applications, right? You know, automatic elastic scale, built in resilience, because if it's a serverless thing, it should just be there. And if I have to actually engage with the thing to make sure that it's actually fault tolerant, that's an issue, right? It has to be always available and has to be instant. And I think this last bullet is probably the one that I think is changing the minds of a lot of people around serverless and really coming down to changing the way that we use and buy software as developers and the way we actually, you know, run these things. I love the concepts of consumption-based billing. You know, what is my monthly bill? I don't wanna deal with, you know, buying a license for something for a year. I only use it for like two months out of that year. It's wasted compute, especially in the day and age of us having, you know, these large bills to the large, you know, cloud providers. How do we actually minimize and become very efficient with our compute and only have like kind of consumption-based, you know, rating and billing for around these things? And I think, if I think about those four kind of core principles of serverless, that's pretty cool. And I think we've come a long way with serverless. Serverless, you know, we have these like these compute platforms, you know, Fargate, Azure services. I really love Google Cloud Run. I think it's pretty cool. You can, you know, take any containerized app and just throw it at Google Cloud Run and it'll scale that thing up and down. It's pretty slick, y'all. I'd go check it out. But it's kind of just managing the infrastructure underneath. And then there's this emerging group of companies that are doing, you know, serverless functions. Again, you know, the big three, I'll offer that. But then I think about, you know, Cloud for Workers and Netlify and these sort of things. I'd love Netlify. I think Netlify is super cool. Where it's more event-driven. You know, it's a function that a developer can call. And it just basically spins up, spins down, is used as needed. It's just a function that basically, as you code, just sits in a directory. And I think these serverless functions are pretty, pretty cool. And I love the concept of serverless functions and the concept of lots of things that we do. And then there's these kind of, these larger serverless app development kind of platforms that are starting to emerge where I think they're putting it all together. They're getting compute, the functions, the whole thing. And I think, you know, start thinking of like the modern IDE and what is that gonna look like? You know, do we get to the point where, you know, we're kind of doing some of these things? I do believe that this kind of next generation of app development is happening. And it's really because I think some of the things that we've seen over the past two years of building serverless and using this. And the value it can provide to people who use these things. Now, when you start to deal with data, things get a little bit more complex when you, within a serverless environment or without, right? Like when you start to deal with data, you have to start thinking about, you know, great, I'm resilient and I'm fault tolerant, but to what level? What's my failure domain? Is my failure domain a region? Is it just an AZ? Is it an entire Kubernetes cluster? How do I survive the failure? Cause these things do go down, right? How do we scale these things so that they can actually, you know, if it's serverless, I don't want to be serverless within a single region. I mean, you know, how do I expand beyond regions, you know? And for databases, you know, the transactional guarantees get really, really tricky here because if I have, you know, a person in New York writing a record and somebody in Sydney writing the record at the same time who wins, how do you, how do you deal with these kinds of conflicts? And so there's things you have to do in the software to actually mitigate these sort of things. But if you take this and you take those serverless principles and now let's apply it to, you know, the database. And then that's what we're going to use as our example here. Well, in a database in the old world, you know, we sharded database, you know, we had, you know, two instances of a database, you know, multiple shards, so we got scale. We were doing the synchronization between two sites. Maybe there's a primary and a secondary one and one region, another in secondary region or maybe I need coverage in Europe. So I have, you know, this sharded fold database thing going on in US West and then I got something over in Europe and I'm doing synchronization and it gets really complex. In this modern world, this becomes more of a distributed platform, right? And we start to apply these principles to this distributed mindset so that, you know, each entity is kind of part of the bigger whole. And that's how you get serverless because scaling big massive pieces up and down, down to zero, it's near impossible to do that. It takes, you know, 10 people or even two people to do that. That's too much. Serverless is about automation. It really is. And it's how do you automate these things? And so when we think about the concepts or the core primitives of our application, how do we break them down so that we can scale them separately? And that's really the trick behind serverless. Now, in a serverless database, again, the data creates big challenges. How do we auto-scale? Putting data close to users so they have low latency access to that and thinking about these transactional latencies, right? Ultimately, I feel like serverless in the term should be infrastructureless because these are the things get really complicated. You know, scaling across regions, these sort of things are really tricky. I think ultimately the end game of serverless is about, I think, you know, it's the value to the user, whoever is using these sort of things. And for developers, I think it's about starting instantly. It's about eliminating the need to manage upgrade, operate, no operations, right? I don't want to deal with scale and I want to completely eliminate downtime. I don't think there's a developer in the world who wants to even think about any of those things on the top of this slide. I think what we want to do is just build. You know, I think we want transactional. We want data to be guaranteed consistent. We want it in real time. And I think ultimately the end game of this, at least for us at Cockroach Labs, when we think about a serverless database, you know, if we really think about that and you just have an API against that database with endpoints around the planet, could the database be something simple as a SQL API in the cloud? I don't even have to think about any of these things. And if we remove the stuff across the top of the slide and we start to just build and code against the database just as an API, things get really interesting. And I think that's the, like to me, that's the ultimate vision of kind of what what serverless can do for the database. And so, you know, think about that. What could that mean for you and your application where you're just some blank blank API in the cloud, right? And that's, you know, for us, it's a SQL API in the cloud. Now, let me actually, I'm going to actually give a little quick demo here, you know, why not? I mean, you know, I'll go out and I'll end here. So I'm going to go to cockroachlabs.com and you could spin up a cluster using our serverless instance right now. I just need to log in. I just hit the sign. Oh, I'm already signed in. And so it's really as simple as create a cluster. We have serverless or dedicated. Dedicated is a, you know, full instances where we have SREs managing it, it's private. Our serverless instance is multi-tenant. You choose a cloud provider, choose a region. Right now we're AWS and Google cloud. And you set a spend limit. Now this comes back to the consumption-based billing. And as consumption-based billing becomes kind of more important, understanding consumer behavior around it is actually pretty important. We've gone a long way to think through this. And for us, it was like, well, let's just put limits in place so that, you know, you're never going to spend more than $10 a month if I put $10 in my spend limit or zero. It could be completely free forever. And for us, that means, well, I have to throttle usage of the database over time. What does that mean for your application? Because I don't think anybody really likes to get a bill at the end of the month where some service just ran wildly for, you know, 20 days and I get charged, you know, thousands of dollars, right? And so I think as we build these things, we as developers need to be cognizant and aware of what people really want. And for us, you know, having spend limits was a big deal. And then we just name the cluster. So I could just create a cluster. It's going to be ready in a couple of seconds here. And what's happened in the background, I'm going to show you exactly what's happened. So I'm going to create a user, generate and save password. Yep, great. And I'm going to hit next. I have a cluster up and running. And I have, you know, I have a complete set of connection strings. I can use a general connection string, parameters. I can use the CockroachDB client, all these things. Basically what I have is I now, I have a Postgres cluster. I have a Postgres instance, basically. It's a CockroachDB serverless instance. We're wire compatible with Postgres. So any of those kind of Postgres tools you would use or drivers, they should just be able to connect to this. And so you're just going to use the connection string or parameters. I use parameters when I'm using a bunch of different tools or just use the CockroachDB client. And now you have a complete relational database that is resilient. It's going to do auto scale. And I'm going to show you what it does underneath the cover. So quick demo basically is it's pretty easy to get basically a CockroachDB cluster up and running. There's lots of more information on our website about what CockroachDB does, how it's different than other things. But ultimately I think it is the simplest form. It's kind of like Postgres, but automate all the scale, eliminate the concept of downtime, online schema changes, upgrades, all that. That's really kind of the big stuff of what it does. So that's the quick little demo. I encourage you all to go out and check out a cluster. So it's pretty easy to get going. Okay, so let's talk about architecture. Let's drive into kind of how we did what we did. And this is really the top of this. So a database is really comprised of three layers. There's a language, there is a SQL execution layer and their storage. Every database really is kind of developed and thought of in this way. And really ultimately, any one of our applications can be thought about like this, right? Like there's an API with my application. Okay, sir, is it a UI? Is it just some basic, you know, rest API? I don't know. The language, like that's how you communicate. It does things and it stores data to disk. Literally a database is just like every other application. They're a little bit more complex. They're not easy to build. I'll tell you that right now. But they're all kind of like this. Now in distributed systems, you know, I think there's databases on the right hand side. You know, we have a distributed database. You have to start thinking a little bit differently because, okay, so the language is still the language, storage is still storage, but actually in storage, am I storing data across multiple different places? Because remember, this is a distributed system. This is no longer a monolith. This isn't like one server. We're using many, many, many, right? And so we have distributed storage. We have to think about how we get data to that. That's replication and distribution. Distributed execution becomes very important, right? I can have a single instance of Postgres on distributed storage. Well, I still have a single instance. All my rights have to go through there. I haven't solved the scale problem, right? So how do you distribute that for both reads and writes? Well, in our database, or how do you distribute your execution for, I don't know, whatever you're doing in your application and starting to think about these concepts is actually pretty important. I think the shift towards distributed systems is where these things are happening. So the divide of where you cut your code apart to actually make something serverless is really between execution and storage. Because if we think about serverless and what we're doing at the storage layer, things need to persist. They cannot be ephemeral. Like they need some sort of consistent existence, right? And you need to make sure that those things are up and running at all times, right? Whereas at the execution layer, I'm not relying on something that needs to be, like it's in that item, right? I'm just, for us, we're just executing a query. We're gonna use storage, right? And so taking away and understanding what's ephemeral versus what needs persistence, that's the divide in your application. So look through your code and figure out what is those lines? Do you need to rearchitect things? How is this gonna work? For us, we had to kind of rearchitect and tease these things apart. Luckily enough, it was pretty modular in our code. So it's fairly straightforward to do that. But that's the trick to really thinking through things that are serverless. I've talked to lots of people building serverless applications. This is the same pattern emerging across the board. And I think it's kind of one of these things is fairly simple once you see it. Ephemeral versus persistence, scale ephemeral, scale persistence in a different way. And I think that's the trick, right? So let's get into the storage layer. In the storage layer of what we're doing, it's a little bit different than some other databases, right? Like in Cockroach, and I kind of have to go through kind of how Cockroach works, but hopefully this is valuable to you as well from an architecture point of view, just show what we had to do. Ultimately in Cockroach, everything, all the data is stored in a KV. It uses KV underneath. Now we translate KV to relational, so it's SQL. But ultimately you don't need to know what's going on in this logical key space, but every table is basically this huge monolithic logical key space. It's all ordered, right? And KV allows us to do some really interesting things. And it's all ordered is the key thing. This lexigraphically by key is the trick here. And so what we do is we take all the tables and we convert them to KV. The K being a table name and index, the key and then a column name. And then the value is the value for that particular column. So let's just show how that works. Here's a quick table, it's the dogs table. I have an ID, I have name, and I have weight. And I have a bunch of entries. Now this just looks like, this is just a table. This is a simple DDL, same thing in Cockroach, relational. If you're familiar with SQL, you get this. And this is what my data looks like. I could select all from dogs. I'm gonna, this is what I'm gonna get back. Underneath the covers, what we do is we actually, when we actually store the data at disk, we're gonna take, and we're gonna translate each row into these KV pairs. So okay, here's the table name is dog. My primary key is 34 for that, the ID, right? And then for the name column, I'm gonna have Carl. Okay, for the weight column, I'm gonna have 10.1. And I just go down the line. And so we're translating records into KV. Now, if you look at the ordering of this, right? Dog 34, name, weight, dog 78, name, weight, dog 94. So we're ordering them. So this basically all gets encoded. It doesn't look like this dog slash 34 last name. We're actually encoding these things so that we can actually get really, really efficient sorts. So underneath the covers. And we know exactly where to insert things into this ordered lexicographic pair. Now that's pretty cool. It's what allows us to actually understand where tables are when we start to actually move them around. Excuse me, I got a frog in my throat. Excuse me, everybody, sorry about that. So what did we do to make this serverless? Well, if we're gonna use shared storage underneath, what if everybody had a dog table? I wouldn't know which dog table belonged to which person. So what we did is we just basically added a tenant ID to the key. And so now this is Andy K's dog table. So what was my cluster? I forget, I don't know, sparkly kitten or something. I forget, we come up with these funny names, but that would belong to that particular tenant. And so what we do is we're just appending that to the front of that. So now in this huge lexicographically ordered KV set, all of the records for that particular tenant are together. And then inside there, the tables are all kind of sorted and then the keys and all the stuff, right? So basically what we're doing is we have a complete understanding across everybody. And we're using one big shared storage layer, but the tenant key is what's allowing us to understand which parts of this data belong to who, okay? Let's see here. There was a question that came through. What is the most appropriate naming convention to label nodes on a Kubernetes classes? For instance, controller and responder. Well, gosh, I don't know, Emra. I'm not, that is an interesting question. I don't know, I'm sorry, I'm gonna have to, I don't know that I don't know the answer to that. Sorry about that. I wish I did. I don't, like y'all, I live in this architecture of the application layer. I don't get too deep into like the, I understand how Kubernetes works, but managing it is complex and kudos to all of you who do that, but maybe somebody else knows, sorry, Emra. All right, so that's great. We just talked about the storage layer. We had to actually think about what we were doing from the storage layer. We then had to think about, okay, distributed execution, we're already doing that, but how do we tie that to a particular tenant? How do I make sure that, you know, Jim's execution is just mine, right? And this sort of stuff, right? So, okay, let's go back to what we're doing in Cockroach. And with this big, huge key space, and we added, you know, we added this whole like the tenant to each of the keys and this sort of thing. And then basically what we're doing is we take this big, huge KV range, right? And we, or this big, huge KV sort, and we cut it down into these ranges, which is basically these 512 megabit ranges. And think of this as like the way that we shard a table. We're just taking these ranges and they're small enough so that we can move them around really quickly. And we can write them to different physical nodes or different nodes that are running in pods in a Kubernetes cluster, right? And basically that's how we kind of distribute the data. And that's a kind of a key concept in Cockroach. Now, if you're gonna do this, we actually have an index so we can have to find these ranges, you know, so the first range lives on these servers and whatnot. And so we have an index structure that allows us to locate these things. It's very much like a B tree, if you're familiar with what that is. If you wanna check out how we do this, our code is completely open. You go check out our Git repo and actually check this out. Underneath the covers though, we're using something called Raft. I hope many people on this are familiar with Raft. If you're not, Raft is a really interesting concept as a developer and anybody who's kind of thinking microservices and distributed systems, if you're on this webinar thinking Kubernetes, definitely kind of check out Raft and see how it applies to what you're doing. But Raft is a distributed consensus algorithm. Now there's another one called PAXOS. Some people use that as well. And what it does is it's distributed system and it's gaining consensus. It really allows us to get atomic reads, atomic writes and consistent reads and what we're doing. Now, what is Raft? Raft is basically, it's an algorithm that manages replicas of whatever it is you want to actually manage and write, say, to disk. In our case, the reason I had this little blue thing, well, this is that range that I had before. This is the second range. So we basically break up our ranges and then manage them using Raft. Now we create replicas of these. So we have three copies here. Within Raft, you want three or five or seven or nine, 11, the odd numbers because you want quorum. So when I write to a Raft group, I want two of three of these to commit. I want two of threes to say we're good and then say everything's good because a third will come along or three of five. So it's a quorum-based transaction. So we get this quorum and actually make all these things work. Also in Raft, there's this concept of a leader. It's a special replica and there's two followers. All the rights, all the commands kind of go to through the Raft leader to the followers and it basically manages to make sure everything's good so that you only have to ask one Raft leader. It's like, great, is this good? Yes, it's good. And I could acknowledge back to the application whatever it's doing. It's going to have this kind of authoritative up-to-date answer for what's going on within that Raft group, right? This also allows us to get atomic replication. So we're going to make sure that this is going to happen. Like the whole transaction is going to happen or not. It's going to coordinate with other Raft groups because I'm doing transactions across maybe hundreds of ranges because maybe I'm inserting 10 records or I'm doing a select across different things and doing joins. Like we can do this kind of cross shard transactions as well and that Raft leader is going to manage that. It's going to help with consistency and make sure that the atomicity in acid actually gets committed. Okay, so if you want to learn more about distributed consensus, you want to learn more about Raft, there is this really wonderful website. I haven't been there in a while. I'm pretty sure it's still going on. Somebody probably would have told me but the secret lives of data.com, I'd go check that out. It's, they give a great kind of visual representation for how Raft works much better than I could ever do. So we use Raft then to actually distribute data, so each Raft group, right? There's three of the copies. It's the same data, it's that range. We're going to distribute them across three different nodes in a system in cockroach DB, right? And so that's what we're doing. We take the first range, we write it, we take the second range, we write it, we take the third range. What this allows for is kind of scale. I can scale, I can add more nodes so I get more capacity. All of these nodes could service reads and writes. I'm highly resilient. If I lose a node, I still have two of three of the copies left, right? And so that's kind of the special nature of what we're doing that's helping us at the storage layer. Okay, great. So how do I kind of pull all this together and now make this serverless? And this is the trick of how you use Kubernetes to actually do all this, okay? But one second before I get there, there was a question, Debanju writes, is it possible to allocate a network volume as persistent storage to these nodes? Cluster, on-prem, open-source cluster and network volume EBS, S3 or EFS. Yes, you're gonna have persistent storage and you're gonna use basically, in Kubernetes, if I presume this question, you're gonna use the storage class and you'll just make a PV claim from inside that pod. And then you'll use stateful sets within the context of the storage nodes like where you need to persist. Use stateful sets to actually maintain that persistent connection between that pod and whichever volume you have actually connected to. And then so each of the storage like here within this context of this, this is us deploying a database. So this could be any application on a Kubernetes cluster. What you're seeing in the bottom, that color is this, this is just a pod and this blue box is the application itself. They're all communicating to each other. And so if I wanted storage, I'm just gonna take a, I'm gonna use storage class PV claim. I'm gonna use, you know, stateful sets to actually attach that to each one of these pods so that I could actually persist that to launch you. So it's a great question. Just me a while to figure out the storage class when I got first introduced to Kubernetes, how this works. So for us, you know, this is one single logical database across multiple different pods. Every one of these things can take reads and writes. This is just SQL. You can do cross shard queries. You can do, we can actually deploy this across multiple regions of multiple Kubernetes clusters. We can have one single logical database which is pretty cool, right? So you can, like your failure domain can actually go up to entire Kubernetes clusters. This was a database that was built literally on the same exact principles as Kubernetes. This wasn't like take a database and do auto sharding and try to make it work in this context. Now this was basically built with the same kind of principles and that's why it works really well within this context. And using Kubernetes to actually build this out as a serverless cluster is pretty cool. And so what we did is we said, hey, look at, let's look at our, let's look at our stack, our layers within our product. And we basically took this kind of the execution and how to find the data and the language. And we put that in one version and then we took the other side and it was the storage of that data and that's separate. And so what we're doing is we're running a completely separate cluster of CockroachDB that's doing storage only. And each of those kind of pods in the bottom, they have their own PV claim. They actually understand where that storage is. And I'm just communicating to them. They can, they can service requests for basically getting data. And remember this data is spread across all my users across all their tables is now distributed across, okay, this is just three nodes running in three pods, but this can be hundreds of pods. And the SQL execution layer just works with this, with the storage layer to basically find that data. So lots of different data and users in the bottom and then the SQL, the execution at the top. And the SQL execution part is ephemeral. So here's a virtual coster for a particular tenant. We have three execution pods and it's just accessing its data underneath. And I can scale basically the execution now get really, really efficient. So tenant one needs three different execution pods because of their bandwidth, right? Tenant two only needs one, tenant three needs three, right? And it really depends on the load that that person has. And that's a key concept in with serverless, right? Scale up, scale down, scale to zero, right? Make it really efficient. So what do you get charged for? Well, for us, it's how much storage and how much compute you use, right? How many transactions? And that's really allow, we're able to scale these things separately, which is a key kind of concept within serverless as we think through this. So how does this look? In Congress to be serverless, we have like the storage and replication nodes. We have more than one in each of AZ, but we're doing a lot of them. We're actually working on making this across multiple regions. So you'll have this across multiple AZs and then regions as well. As beta right now, it's within a single region. You spin up, so tenant one is living on three different availability zones. So if I lose an availability zone, I still have other two other pods to actually go against. Again, it kind of improves that resilient nature. So tenant two comes in as one and then tenant three. And so we're just basically randomly deploying them across different AZs. Now, if you're gonna do this, you need to be able to find what it is is your compute layer, right? Where is this thing? And so, we built a piece of software which is basically a proxy, which understands where my tenant is. So if I come and ask any one of these proxies that are in any one of these AZs running, it knows it has a manifest basically. It says, hey, tenant one, here's where you can find tenant one's execution layer and route that query or route that request, whatever you're doing an application to that particular user, right? So we tease out the ephemeral compute from the persistent storage and then you need to find things and we built a proxy to do that. Really as simple as that to actually build out a serverless thing. Now data also scales, transactions scale as well. So volume of data will scale at the persistent layer down here in storage or replication. We're just gonna keep adding capacity there, right? But the ephemeral compute that goes up and down based on the traffic, right? And so we needed to accommodate elastic usage, deal with spikes and spin things down to zero. And this is all done via these kind of ephemeral kind of SQL pods. So we built this capability in autoscalar. And autoscalar basically is looking at, you know each particular CPU for these pods and understanding know what are the usages over the last five minutes. We're looking at peak CPU usage and all within the context of a particular tenant, right? And so if a tenant needs capacity we spin up new capacity. If they are not using it, we spin it down. If it falls to zero, we'll completely remove the tenant from the cluster. And so that we aren't using compute that we don't need to pay for. So how does this look for? So here's our diagram from before, you know we've teased out the two layers. Tenant one, my capacity, my steady state capacity is average here up on the left hand side. It requires three ephemeral pods to deal with that amount of traffic. But at some point I get a spike. I don't know, it's like five PM and a bunch of people come online and I'm getting a lot of orders come in and it's going up. And so what we do is we have these things that we have in each cluster and there's a cluster in each availability zone. We basically have these unassigned hot pods. They're basically instances of my ephemeral compute later that are just unassigned. So the proxy doesn't even, they just know they're there but they aren't assigned to a tenant. And so what I do is I basically take one of these hot pods in the proxy and the manifest that it says, hey, assign this to tenant one and start using it. I have just scaled capacity like that. Because these things are already up and running it's up really fast. I don't have to worry about the time it takes for the pod to start and then start the application. All these things is just running an unassigned. And that's what we did. Now when things return back to steady state I spin that thing down. And if it goes all the way, right? It's like middle of the night nobody's asking for any transactions. We scale that all the way down. So the auto scaler is actually doing this in the background. But then most importantly, say a query comes in it wakes up that tenant. And so we're able to spin up a SQL pod very quickly. It's under a half a second and it's for that first query that comes in. I know we're trying to drive that down to like, under a hundred milliseconds under a tenth of a second. And so right now that's kind of a big focus. How do we get these not running really fast? Another big problem you have to deal with in the serverless world because well people want these things to actually come up really quick. And so what we're doing is again we just take this hot pod we don't have that latency of spinning up the Kubernetes pod instance. And we're assigning that to a particular tenant the proxy now understands that all the proxies know where to go. So if a query comes in against any one of them it knows where to go get that tenant, right? And so that's basically how it works. So that's what we've done to basically take something what was a distributed system and fairly elegant but then tease out this kind of ephemeral compute side from this persistent storage layer to make our application serverless. Now it was fairly straightforward for us because while we were already a distributed system so like kind of applying these principles were a little bit easier but I think if you start from scratch thinking about this like as you build your next application how do you kind of tease these two things apart and in the architecture of application makes a whole lot of sense because you kind of prepare yourself to apply the serverless principles to whatever it is that you're building. And I think that's the trick and for me the big takeaway. Again, you saw me create an instance y'all please go feel free to go out and build your own cluster it's pretty easy to sign up. We're gonna give you a free cluster up to five gigabytes of storage a month 250 million request units which is pretty ample. A request unit is just basically an abstraction of a query. All queries are not similar like select star from where and join blah, blah, blah that's much more query than select star from customer, so that's the way we do that and it's free. So you can go out and do that right now. Request units are interesting if you run out of requests units we don't actually, you don't run down to zero so you have no capacity we actually reserve enough capacity so that you have enough for the rest of the month so we basically kind of throttle it so like this last scenario is easy you have 8.64 million request units if you use everything on your first day you have that amount for the rest of the month, right? So it's fairly good from that point of view. So what's the chargeback model for this cockroach DB? The charge model is basically you can add a gigabyte of storage or 250 million request units for additional I think it's a dollar per, I forget what it is you have to go to the website and look at that but it outlines all of that but it's free up to that certain level. So okay, so that is all my presentation there weren't a whole lot of questions I hope it was valuable to everybody I do hope to see people live I'm super excited to go to QCon and be live with people in a couple of weeks I'm also really excited to go to Valencia I would not, I would be lying if I didn't say that you know the the Congress Labs team will be there we'll have lots of engineers who can talk through these things with you we can talk about our database as much as you want as well we'll be giving away things all these things if you're thinking about going I really look forward to seeing people again live at QCon it's my favorite two shows of the year are QCon Europe and QCon North America so I'm really looking forward to getting back with everybody so with that, if there's no other questions I wanted to thank everybody for taking the time today I hope this was valuable to you you know I hope I did a little bit of a balance between us talking about a database but kind of how we thought about going through this serverless journey I really hope it came I really hope that was a value to people so all right well thank you everybody I really really appreciate it oh that was my other presenter this morning that's my friend Rain, oops I didn't fix this slide so thank you everybody I really really appreciate it and have a great day. Great thank you so much to Jim for his time today and thank you to all the participants who joined us as a reminder this recording will be on the Linux Foundation YouTube page later today we hope you're able to join us for future webinars have a wonderful day.