 Welcome to How to Deploy a Cloud Native Database on Kubernetes. Today, we'll talk about how to use persistent volumes with each cockroach database pod, how to use replica sets to manage a cluster, and attaching an application to cockroach DB. My name is Lindsay Hooper, and I'm one of the Postgres Conference organizers, and I will be your moderator for this webinar. I'm here with Jim Walker, VP Product Marketing at Cockroach Labs, and Chris Casano, Solutions Engineering Cockroach Labs. Jim is a developer who has spent a large part of his career helping people understand complex emerging technologies. He's passionate about distributed systems, data, and open source technologies. He's a startup, a holic, and loves building apps in new communities, and he's helped build companies like Hortonworks and CoreOS. Prior to Cockroach, Chris was a Senior Solutions Engineering Manager at Cloudera, a Solutions Engineering Manager at Hortonworks, and a Principal Sales Consultant at Oracle. Jim and Chris, thank you so much for being here. With that, I'm going to hand it off, and you guys can take it away. Enjoy. That's great. Thanks, Lindsay, and welcome everybody to the webinar. Yeah, today we're going to talk about Postgres a little bit. We're going to talk about running relational database on Kubernetes. And I think the little bit about my history, thank you for that great introduction, Lindsay. I've kind of been in the Kubernetes space for a couple years working within companies. I was early days at CoreOS too, and well, not early, but I guess in there working through that whole process. And one of the areas where I saw some applications being held back from being deployed in Kubernetes was really this relational, the operational data. These workloads, like the system of record workloads where I need a database behind the application. Stateless applications in Kubernetes, straight forward, it's pretty simple. I want to compute out there. And so for me, finding CockroachDB at the time a couple of years ago, I was like, wow, this is cool. This kind of fits this new world. And so that's what we're really going to talk about. So first of all, welcome, Chris. I just did with my quick little thing, but thanks for joining me, Chris. You're out there, buddy. Just wanted to make sure your mic is working and you're there. Hey, Jim, can you hear me? You bet. Awesome. Say hello to the crowd, Chris. Hello, crowd. Great to meet everyone. Really excited to talk to you today. Awesome. So thanks, Chris. I'm going to give just a quick background. We're going to, a large part of this will be Chris in product and hands-on keyboard and command line and all that one now. Just wanted to give everybody kind of a quick context before we got into that. And as Lindsay noted, please do ask questions. I'll be monitoring questions in the chat along the way. And I'll try to throw those in either in line in conversation that I'm having with Chris or explicitly at the end or explicitly as a question that I'll lob in to us to actually answer that. But please do participate, because we love that. So I, like I was saying earlier, I kind of jumped the gun on my intro. I really love Kubernetes. And I love the whole move towards distributed systems and orchestration and everything that's happening. I'm proud to be a member of the larger Kubernetes community. I think I find it to be an incredibly great group of people. Not just some really interesting technology, but I think it's some really amazing people. And I've made some really great friends. One of them that I've been lucky enough to meet a couple of times is Kelsey Hightower. And if you're in the Kubernetes community, you probably know Kelsey because they'll keynote things like KubeCon. And he's just been a really big proponent and really an evangelist for Kubernetes just across the globe. And probably one of the smartest guys in the entire community. And he gets this stuff really well. His experience has lent him to understanding distributed systems and lending that sort of thing. But I think when I first found Cockroach, I was talking to Kelsey about us. And he's like, yeah, that's the database for Kubernetes. Really because it kind of fits the architecture. What's the underlying design and the way that Cockroach DB was really built from the scratch really is well aligned with Kubernetes. And it really comes back to the history of Kubernetes and the history of Cockroach and the alignment that they have. If you're not familiar with Kubernetes and the project and its origin story, really it is a derivative of something that is used internally at Google called Borg. Borg being the back end kind of orchestration and platform in which all their services run. And so Google took some of those concepts and open source the version or kind of a version of that called Kubernetes. Now, at the same time at Google, back in the early 2010s or so, they had actually built a database called Google Spanner. And Spanner was really built for Borg. And there were a lot of concepts and contexts in that that's kind of built for that. And so what is the open source database distributed database that actually works with Kubernetes? And that's exactly what Cockroach DB is. If you look back to the Google Spanner white paper that came out and oh gosh, I don't even know the year right now. It's like a lot of the same core principles that are in that paper is exactly what Cockroach DB was built. So the alignment of Cockroach DB with Kubernetes is very much like the alignment of Spanner with Borg. And so really, this is a database that was built for this new world. Now, there's lots of things that we're going to talk about that Chris can talk about we get into. We often get asked, hey, do you guys have an operator? I think for a while we were kind of like, we don't really need an operator because we actually piggybacked on a lot of the core capabilities and the core features that were found in Kubernetes because we were so well aligned. We weren't legacy tech trying to work in this new world. We're tech actually built port. So we have since built operators and whatnot. I know where Red Hat OpenShift operator certified. We're building out operators to help people manage these ongoing clusters. But install and rolling upgrades and all that stuff pretty straightforward. And so it just lives along the lines of this unique distributed architecture that Kubernetes is giving to people. At the core, Kubernetes is really two pieces, a control plane and something called a pod, where the pod is used to coordinate back with a control plane and basically allows you to scale applications up and down and make sure that applications are going to be always on and you're going to meet certain SLAs from a compute point of view. But it has challenges. It has challenges being geo-distributed. Running a Kubernetes cluster in a single data center is, I'm not going to say straightforward because I think there's probably some Kubernetes people who might freak out by that. But it's a lot more straightforward than trying to run these things across multiple different data centers because there's things like network storage and compute that get really, really complex. But the core concepts in Kubernetes are really fantastic. And so Cockroach is really architected to take care of some of these things. At the core of what Cockroach is doing is a distributed system. We are a true, if you're a student of distributed systems, we are a true shared nothing single atomic unit distributed system where there's some core concepts in distributed systems where each individual participant in a global compute strategy, whatever that's going to be, in this case a database, is its own atomic unit. It's a self-contained unit. And it doesn't need other participants, to actually accomplish what it needs to accomplish internally. It can do things better with other participants. But the single atomic unit, which is CockroachDB, is really critical. But we also implement something from our architecture that is we are shared nothing, which means we don't share concepts or data or certain things with other instances of the database. And that's actually really, really critical. It's a very different way of thinking about a distributed or cloud database than some of the other things that are out there. Like an Amazon Aurora is just a bunch of instances, a Postgres, one write node, and a bunch of read instances on top of distributed storage. That's shared something, that's shared storage. And that's how they actually are able to accomplish kind of sharing. But it doesn't, when you start to actually perform rights, when you have distributed storage and distributed reads across a planet, how do you actually control these things? How do I allow somebody to write a record in New York and Sydney at the same time, who wins, right? And so CockroachDB takes care of some really difficult challenges in distributed systems when it comes to a database. Number one, in a coordinating and having consensus for all queries and transactions, allowing for transactions to happen at every single node because we are the single atomic unit. I don't have different write nodes and read nodes. Everything is like, it's a full instance of a database, just as you would think of any instance of a single node database, right? And so how do you actually coordinate transactions? We actually implement a distributed architecture and we're writing data in triplicate to the database. We're writing different nodes, different instances of the database. So we can be resilient. So if you lose a node, you're still gonna have two copies of that data. But what's interesting is when you start to deploy these nodes in different places, there's no more manual sharding of the database either. We actually automate all that. I'll go through that a little bit. And then we can do something really unique in that we can attach data to a location. We can do something called geo partitioning, which is kind of a unique characteristic of CockroachDB. Now, we've also done this in the open. We aren't tied to any particular hardware. We aren't tied to any particular cloud. So this is inherently multi-cloud. I could spin up a node of CockroachDB on this laptop. And as long as I have a TLS connection with a cluster that's running in AWS or Azure or on-prem, I can connect to that. And this laptop that I'm standing in front of will be a participant in that database. It will look like one logical database to the end user. And what that means is that every single node in this entire database is a gateway, is an endpoint, if you will, if you think like that, into the rest of the database. So I would have complete access to the data. It would start sharding data here and we would have a bunch of data going on. So ultimately, underneath the covers, we use something called Raft. If you're not familiar with Raft, it's a really cool algorithm. If you're in the distributed systems, there's two cool things to really think about. Raft and PAXOS are kind of the core distributed consensus protocols that I think are really interesting. Basically, we're using Raft to accomplish a lot of what we do. In Raft, what we're doing is we're actually writing data in triplicate across multiple nodes. On the right-hand side, you can think of this as like, this is a table and we're writing it in three ways. And we have three replicas of that data. That is a Raft group. Now we use that to basically distribute data and to ensure basically consistency with across a cluster. Raft is also important. There's a concept in Raft called a leaseholder. And that is like the leader of the three replicas, right? And that leader basically has basically the system of record for that little group of records. That's shard of data, right? And it will basically always be sure. But it's also coordinating to make sure that transactions are committed and reads are happening in the right way. There's a lot of capabilities in the leaseholder. I don't want to get too deep into the capabilities of the architecture. We have a couple other, I think we did an architecture webinar with the Postgres community already and we get deep into this. And I want to get to Chris so he can actually show you all how this thing actually works. But when we place data into a cluster, one of the unique things you have to think about with CockroachDB is when you implement the data, when you implement the database and you're actually defining schema for tables, you need to take into consideration two extra things beyond like the normal indexes, right? You got to think about resilience and latency. Resilience is what do I want to survive? What kind of value or domain do I want to survive? Like where does my data need to be located so I can survive a failure of a disk or a rack or a data center, a single node? In this case, I'm gonna do single node. Maybe I can actually fail a region, right? So I have one copy of data in one region, two copies in another region. I can actually survive the value of a region at that point, right? So there's things that you do at the table level that will be implemented at the row level for your data that is actually on. So here's a dogs table. We're gonna just gonna start writing shards of this data, ranges, if you will, at least 64 megabit chunks to different machines. So we're gonna write to node one, node 304, node one, two, and three, and then we're gonna put the red range out into these other nodes. We can now survive failure of a particular node because I'm always gonna have two copies of this data. Now, we're distributing the data here based on storage, but we can actually also place load based on load. So maybe there's a hot range, a hot shard, if you will, for the old time. But if you're getting too much access to that data, the database is actually smart enough to understand that I have too much load in a particular part of the table. So let's just isolate that shard or that range onto nodes that have less utilization so that we can actually optimize for load as well, which is actually pretty important from a performance point of view. We can also place data in particular locations. People love this because it helps us allow, it helps us comply with jurisdictional privacy laws. I can have all German data live in Germany. I can have certain data live in different jurisdictions. And so as long as we can tag a node as living in a particular region, the database itself is actually gonna automate this. The database is gonna take care of partitioning the data and making sure it physically lives on servers where it's designed to live. We do that at the table level. We do it when you design the table, when you design the database, but we can also do it on the fly in production. We can make changes to all of this in production, which is pretty cool. We also can rebalance. So say we throw in a new node into this corner or cluster, what it's gonna do is basically redistribute things. So no more manual sharding, it's just automated the complete redistribution of data. So you don't have to think about any of that, but we can also survive a failure as well. So we had this cluster of five nodes, one of the nodes went down. It's smart enough to actually understand because of RAF and these replica sets. So basically I had some under-replicated replica sets and Chris will show you this in the product. It actually redistributes those and makes sure that you're actually always gonna have three copies of that data. So the database itself is actually doing a whole lot of things, but it's aligned very well with Kubernetes because the same way that Kubernetes is built to scale, is built to be resilient, is built to automate all these capabilities, the same kind of core concepts that are in CochristDB. So without further ado, I wanna actually get into the demo itself. So Chris, you still there, buddy? I'm here, Jim. Awesome. Did I miss anything? Was that all right? I think you're spot on. It's gonna be a nice setup for the demonstration here. Awesome. It's like we planned it, buddy. So you wanna take it away here, I'll stop sharing here and let's get into some code here and let's get into some product. Okay, great. So I'm just gonna give a quick explanation of what you're gonna see today. I've actually documented this entire demo. So if you want it to go do it yourself, you can go to my GitHub page. There is a repository there called cockroach-eks and the entire script is there as far as how you can set up your own environment and run the demo that you're gonna see today. What we're gonna show is how you can deploy a cockroach using a stateful set in Kubernetes and we're gonna do this in a secure fashion. So we're gonna create a secure cluster, deploy it using a stateful set and we'll also be able to deploy a really simple Flask application that will connect to the database. So you can see how an app and the database can work with each other in Kubernetes. And then we'll do some resilience testing. So we'll actually remove a node from the cockroach database and you'll see that we can still continue to transact on the database itself. There's a quick checklist here. These are all the things that I had to do to kind of set up the demonstration. I'm not gonna go into each one of them individually but really simple and easy to do. Again, this is using Amazon EKS. That's our Kubernetes cluster that we're using here today. It's not local on my laptop or doing everything out in the Amazon Cloud. Again, we're gonna go create that certificate authority, set up our cluster and we'll connect to our cluster, deploy an app, connect to the database and then I'll show you some of the resiliency pieces. So without further ado, let's jump right into it. Come out of the full screen here. Oops, forward one too many slides, let's go back there. So we're gonna start with the end state. And like I said, this is not a fancy app at all. We're just trying to demonstrate to you how this works. But I have a really simple application that's built in Flask using SQL Alchemy. SQL Alchemy is an org that you can utilize from a programmatic perspective to connect to a database like CockroachDB. And all we have in this application is a to-do list. And it's basically what I'm gonna show you today. First, I'm gonna show you how to create a certificate authority. Then we're gonna go to create a Secure's Cockroach database, connect the Flask app and test resilience. So just to do a quick demonstration, you know, maybe the last thing we'll do is answer questions. So I'm gonna put another to-do item here, right? We'll create that entry. All right, so you can see that we have actually added something here in the application. So if we go to the database, let's just go confirm that that information's there. So I'm gonna, I have a cube kettle or cubectl or cube control, whatever you wanna call it. This is our way of connecting to the, you know, the Kubernetes environment. And I'm gonna spin up a Cockroach client so that I can go query the database. So let me just do a select on my table so that you can see the information that, you know, you're seeing through the application, right? So here's the same information that you were seeing through the application. Here's a new entry that I just added called answer questions, okay? So that just shows you how the application's set up. What we're gonna do now is we're actually gonna kill this entire thing. We're gonna start over and I'm gonna show you how we get to the end state. So what I'm gonna do is I'm gonna remove everything. All right, so I like self-driving demos. I think we're in the age of automation. So I try to use a lot of scripts to kind of automate some of these things, but this is actually gonna go into the EKS environment. And it's actually gonna remove, you know, my Cockroach database is gonna remove the Flask application that you saw. It's gonna remove the stateful set that we created. It's gonna remove all those things and then we're gonna come back and recreate it. Okay, so while that's removing and dropping all those items, I'm gonna give you an idea of how you can go about doing this yourself. The first thing you'll wanna do is, you know, take a look at how you can set up a stateful set. You can go right into our documentation, you know docs.cockroachlabs.com or you can go right into our GitHub repo. And if you go to this URL that you see here, it's in the Kubernetes section, bring your own search. There's a YAML file or a config file that shows you how you can go about doing this. And it's pretty straightforward. What you're doing at first is creating all the security that you need to set up wire encryption for Cockroach DB. So you go through a series of commands where you create search for the nodes, you create search for the users, you're then adding those search to EKS. So there's a couple of commands here that you can see where you're adding the search right into EKS. And that kind of gets you all set up from a security standpoint. From there, you can apply the stateful set. The stateful set is gonna deploy Cockroach DB. It says, hey, I wanna set up with these persistent volumes of where I want to keep my data. I wanna set up with a load balancer so that I can balance the traffic to all the nodes within my pod. And then start up the Cockroach service. So it's really that straightforward. If I come back to here, we can go and check to see if we've removed everything. So I'll just do a quick command here. We'll just confirm that everything is gone. Yep, you can see here, I just have my secrets that are there. And yep, we're all good and ready to go. So I'm gonna run a demo right now that we'll bring up the environment, a piece at a time, and then I'll show you how that works from a command perspective. All right, so I'm gonna kick this off. You can see the script is gonna go ahead and add the stateful set. And in just a minute, it's gonna, once it goes through, I'll connect to the database. You can see all the items that are running. Okay, so here's what we created. We have a stateful set that was added to Kubernetes. Right now it's still bringing up the pods. You can see my secrets here that I've added before. I didn't have to remove them. They're already there, didn't make sense to remove them and add them back. We have a database that's getting created. So you can see that everything's just starting up. I'll do a test here again to make sure everything's up and going before we kick off the admin UI. But we have a three-node setup and then we also have a secure client so I can go and talk to the database. Again, all the wire encryption is gonna be set up between each of the containers within these pods. And the last couple of things. So we have a couple of services that are here. Again, for load balancing and understanding DNS within those pods. So that's all created through the scripts. Okay, so from here, I'm gonna go click return. It's gonna bring up the admin UI for us and I'll give you a quick walkthrough of the environment. So right now I'm in the admin UI. I can make this a little bit bigger for you. You can see that we have three nodes that are live within the database. I can come over to the metrics too and I can see what's going on, what queries are running, what statements have been run before and so forth. You'll also notice that if I go to databases you'll notice that I have the to-dos databases here that I was querying before, which is kind of interesting. And you might be asking, well, I'm used to Kubernetes being working with stateless apps and if I lose a pod and bring it back I don't always have my information with me. Well, this is where the concept of a stateful set changes that whole dynamic. With the admin of stateful sets it allows us now to actually persist not just the data but also the network identities and the data volumes for pods. So even though I blew away all my pods and then redeployed the stateful set those persistent volumes are still within Kubernetes and all I have to do is reattach them and you can see that I have my databases still here intact. So something that's rather unique with stateful sets. Yep, and great for stateful types of applications like databases. Okay, so that's my, here's my cluster, my three node cluster that's up and running. Let's go back to my self-driving demo here. The next thing we're gonna do is we're gonna deploy the Flask app and I'll show you how this works as well. So I'll click return. This is gonna deploy three nodes within Kubernetes. And I think you're familiar with this screen before. Here's the application that we've created before. So you can see that this is up and running as well. Okay, you know, actually while we're here I'll just show you how what that application looks like. Let me just come to my hello app. Yeah, here's the, here's how the configuration works. Again, this is a, you know, a Flask application that's using SQL alchemy. All I really had to do here is just prepare the item you see here. That might be a little small. If you can't read it, I'll quickly explain what this is. This is basically the URL to connect to the database. You can see that I'm connecting as a user max roach. Here's the load balancer that I'm connecting to as well as the certificates that are, that makes my connection secure. And then there's other things here as far as, you know the code for the application and so forth. But that's all connected to the database securely. So I'm doing good. So if we wanted to do an update here, so we created the certificate authority. You know, we created the database. We connected the Flask app. We can update that, you know that just transact in the database. The last thing we wanna do is we wanna remove a pod. So what I'm gonna do now is I'm actually to kill one of the, excuse me, I wanna kill one of the cockroach containers, which is running, you know, one of the cockroach nodes. I'm gonna kill that and I'm gonna show you how we're still resilient within this setup. So I'm gonna click enter here. All right. And it's gonna go ahead and delete the pod. I'll add a new item. I'm gonna say test this out, right? And we're still able to transact. And if we go back to the admin UI, you'll see here that, you know we actually have a node down, right? We killed one of the nodes, the node is down. And what's nice about cockroaches is that it's very self-aware of what's going on, right? And it will also go ahead and start repairing itself. So part of the stateful set will say, you know I have X number of replicas I wanna use for the database. And you know, so if a node does come down the node will spin back up. And cockroach is also aware of, you know where it's under replicated. So you might have just saw this really quick, but it showed me that I was under replicated because I was lost a particular node. What's great about cockroach is that it's pretty patient. It realizes, hey, there's something not right here. It will wait to see if there's a connectivity problem or something as far as getting that node back connected to the cluster. If it over a period of time, it doesn't get connected, it will start replicating that data to make sure that those range sets that Jim mentioned are continued to be resilient and have a replication factor of three. Hey, Chris, so in this case, basically you killed the pod but Kubernetes was smart enough to get the pod up and running again because that's what Kubernetes does, right? And basically by spinning that pod back up it knew basically cockroaches there, great. So cockroaches use stateful sets to basically get state of that killed off container pod and restart, correct? I mean, so I guess what I'm saying is like it's super aligned. Very aligned, right? So that's nice about how, like you said, Jim, having a shared nothing architecture, it's very easy to scale out and then also be able to handle failures which are typically happen a lot with containers. So that pretty much brings me to the end of the demonstration here. Why don't we go to maybe questions, Jim? Yeah, so there's a couple of things, Chris, that I think are kind of unique here. And number one, there was a question here. The UI that you have in front of you, I'm paraphrasing. Is this available in the open source version as well? Yes, so the admin UI here is a part of the cockroach core version as well as the enterprise version. Right. So, and then I guess I'll ask the question that we got last time, Chris. It's because this is self-contained. Do I have to spin up extra hardware to actually run this admin UI or how does that work within Cockroach DB? Yeah, great question. So you can run the admin UI from any of the nodes. So what's really nice about this is that, let me see if I can bring it up here. Yeah, so here's all the nodes that I have in my particular cluster. And there is an HTTP address that you can add here. It's typically always 8080, but you can set it to something else. But you can connect to that UI on any of the nodes. So what's nice about this is not like some of the traditional databases we have where you have an admin tool that's separate from the database. It needs to have its own HA configuration. It needs to have a shared database. It needs to have two separate applications running. What's nice is we could just take advantage of what we have as far as how the Cockroach database is built and use our own dog food as far as serving up the admin UI and the metrics. Right. And I think that's huge, right? It's kind of one of these things where it's, it comes back to this whole like, is it so self-contained atomic unit, right? And that, you know, every node is the same exact binary deployed across the entire thing, right? Actually, there was a question here, Chris, and I'm just actually just this great question. So can you describe briefly how database, how clients will interact with services endpoint that ultimately gets access to the data in one of the nodes? And how much awareness do the clients have to have about where pods are living and further where data is residing? I think that's a softball, but I love that question. Yeah, so you can go in a lot of different ways here, which is great. That's what Cockroach gives you is a ton of versatility, especially from the locality perspective, which Jim was talking about earlier. So, you know, I can have this database span multiple regions, right? And from an application standpoint, if my applications co-located with my, you know, with each of the different regions, all I have to do is connect to the local region where that database resides to connect to the entire database. So I don't have to connect, you know, even though my app is, you know, I might have an app set up in Virginia. It doesn't need to connect to the database that's in California. I could just connect right within Virginia. And I still have connectivity to the entire database that could be spanning the globe. So it's, you know, I think we can all, all the patterns that you could do with Cockroach, you can make things super complex if you needed to. But under the, you know, under the hood, it's really simple. You have an application that connects to, you know, connects to a database. You can use any of the Postgres drivers that are out there to make the connection really simple. What's nice too is that even with the Postgres drivers, you can fail over to other nodes as well. And what's great about that from a Cockroach perspective is that, like Jim mentioned, every node is a gateway. So it doesn't matter which node you're connecting to. You have the same view of a database, no matter if you're connecting into node one or to node 15. Right, and that allows us to do some cool things, right? Like we can start to move data around a cluster to say, follow a user, right? Let's follow the customer and move data into a different region because they started accessing data from Germany or whatever, right? Because we want latency to be, you know, really low for them to actually think about that. The application owner doesn't have to worry about that whatsoever. They don't have to know, you know, from a compliance point of view. If I want to access records that are of, say, German and privacy laws are real, you know, real strict, well, I can have that data live on a German server. I don't have to ask a German endpoint. I don't have to ask a German gateway, right? A node that lives in Germany. I could ask anything, like I was saying, I could spin up a node right here, right? And access that data, right, Chris? That's correct. Yeah, so the locality is key for a lot of different use cases, whether they're regulatory or if you want to really lower latency for your application. Right. And one customer put it really good to position it really well. It's almost like a, cockroach is almost like a CDN with transactions, right? Where you can bring the data closer to your users, where your users need to access that data. That's really what we do is like being able to lower the latency where your users need to connect to the data. Right. So let's talk a little bit about transactions, Chris. And I think that's one of the, we just did a webinar last week with our friend Keith and I was talking to some of the differences between Cassandra and cockroach. And a lot of the difference there really come around transactions and how data gets distributed and how we deal with these things. We got a question from somebody on the webinar and was talking about transaction isolation. I know we do serializable isolation. We could talk about that. But if a transaction is happening on a row and a new one comes in at the same time and that row hasn't been committed yet, let's do single row. Let's do single transaction. Let's not even get into multiple transactions or multiple rights within a single transaction. Like it's not gonna get into that. Does it wait or does it, what happens, right? What's the protocol when a transaction has happened and a conflict comes in at the same time? Yeah, it all depends on the type of conflict, but because we're serializable, everything gets ordered as far as how transactions are gonna occur. So if a transaction is happening currently, there's an insert or a right transaction that's happening. The next transaction that's coming is that is gonna have to wait, right? And there'll be a retry that happens. So that once that other, the beginning transaction completed, the next one will happen. Which great is for consistency purposes, from a right perspective, it's perfect for applications where you have really strict consistency types of use cases, whether they're financial ledgers or payments or metadata. That's where a cockroach shines for those types of use cases. And there are things that we can do to, sometimes you might have very hard read types of applications that you might want to have the capability to read stale data. We have a concept called follower reads where you can have data that's read locally as opposed to going all the way back to the lease holder. That's gonna be handling all that transaction traffic and be able to read basically points of time. I think within the 20.1 release, we'll have, you can do a stale consistent read from about five seconds ago. So if you can tolerate that sort of stainless of a consistency read, you could do that within cockroach as well. And Chris, those limitations are at the table level, correct? Yes, so you can do, yes, follow reads at the table level. It's basically, we call time travel queries. So at the table, you'll just put in your statements, basically notation called as of system time. And you'll say as of system time five seconds ago and then your query will run. Cool, so let's talk a little bit about, actually somebody had a follow up. What about reads? Do they still need to wait for others to complete? We don't have to wait for reads, right? It was just as long as the records are committed, it would just be writes that a read would have to wait for. Would do reads wait for writes to complete or how does that work? Yeah, the reads will wait too. If there's a write intent actually happening on that particular range, it's gonna wait too. So it will retry, right? And then once that write is committed, the read will perform. However, like I mentioned with the follow reads concept, if you don't wanna wait, you can always, like I said, do a stale consistent read if you wanted to. Yeah, so it's a good segue into the next topic. I got a question here too. What about compatibility with SQL and just kind of SQL coverage that we have? Can you comment on, I mean, we're on a Postgres webinar here, I guess I'm leading the witness a little bit, but can you comment on just a compatibility with Postgres and then SQL syntax? Yeah, great question. So yeah, we utilize the Postgres wire protocol within Cockroach, so what does that allow you to do and why is that beneficial? Well, it allows you to use any of the Postgres drivers that are out there to connect to Cockroach, which makes it very ubiquitous as far as all the types of applications that you'd wanna connect to Cockroach, whether you're using Java or Hibernate or Spring Boot within the Java world, or if you wanna use Go or Python, it's very easy to go and connect to Cockroach. We made, I think, a really great decision as far as being able to support the Postgres wire protocol within the database. And then on top of that, the dialect is very similar as well. Not every single command that you can run in Postgres, you can run in Cockroach, but all the SQL compliant types of statements that you're looking to run, typically will run on Cockroach. They're just a handful of features that Postgres has that Cockroach doesn't, things like stored procedures and triggers. But for the most part, the dialect is very similar. Right. And so is this compatible with other tools as well? Like say, I'm using tools on top of Postgres, Pygiammon or like Liquidbase and some of these other things, like how is the compatibility with those things? I presume fairly well, right? Wire compatible in syntax. Yeah, if you can utilize the Postgres drivers, it's very easy to connect to third party applications. Okay, cool. And then I think there was another question that said, so is this on top of Postgres, not MariaDB? I think there's a, and just I'll take that one, but I think what Chris is talking about, yeah, we implemented the wire protocol, right? So we're gonna speak that language, I guess, if you will, if you wanna call it that, we're gonna speak that protocol. And we have re-architected the entirety of the database. While this may look like Postgres to an application developer, the entire SQL execution and storage and everything else under the covers beyond the interface is a new database. This is not built on top of Postgres, this is not built on top of MariaDB. And we did that because we actually wanted to be a cloud database, right? We actually wanted to re-architect from the ground up to be truly cloud native. And that's kind of one of those things I think is a little bit different from what we do. And then some of the others, right, Chris? Yeah, I mean, it's a complete rewrite from the execution layer, right? That's correct, yeah. Underneath the covers, it's completely different from, you know, Postgres or Maria or MySQL or any of the other legacy relational databases. However, to a developer, you know, it's gonna look a lot like Postgres. When you connect to it, you feel like you're in a Postgres database. Yep. So should we be pooling connections with something like JDBC or does that just throw a wrench in the internal consistency that Kakarot is trying to achieve across nodes? No, you can absolutely use connection pools. That's, you know, that's certainly welcomed as far as how you want your apps to connect to the database. Right. Yep. And it comes back to the whole thing, like this should work the way that you work. I think that was kind of one of the design principles that we had, right? Yeah, absolutely. I think the biggest difference that I've seen, you know, if you think about the history of databases going from relational to NoseSQL, you know, now to distributed SQL with Kakarot's DB is that, you know, the past couple of years because a lot of NoseSQL stores kind of gave up on consistency or having eventual consistency, a lot of times you're building those checks within your application. Right. And then within Kakarot, you don't necessarily need to do that. Right, it's like, right, they built these NoseSQL databases. Like, they took the guardrails out, but by taking the guardrails out, you had to put the guardrails into your application if you actually wanted to accomplish any sort of transactional consistency. Right, but they took the guardrails out so they could get speed of access. And so, like, I think for a system of access, stuff like, you know, the Cassandra's of the world, great tech, right? But system of record, they, you know, there's issues with transactions. And I think that's what I'm seeing a lot of, so. Yeah. So, one other question here, and it was in the context, and I'll go back a couple of questions, but you mentioned something interesting in that, you know, stored procedures was one of the things that is not in our syntax. How do you get past that, right? Like, I think stored procedures are part of, you know, what some people are doing. You know, it's definitely older tech. I guess when I was coding, I was thinking about them. How do you get past that in a database like this that isn't allowing for stored procedures? Yeah, great question. And I wrote my fair share of PL SQL in my heyday. I'm sure there's still plenty of them floating around out there. But yeah, you know, we don't, we don't support stored prox. And if you think about a lot of the design patterns of how applications are modernizing, especially using with ORMs, you know, a lot of logic is moving from the database back into the applications. I think what, where we got caught up, you know, historically is that we'd have, you know, logic built in databases and stored procedures and then we have logic built in applications. And then you, you know, you have competing, sometimes in big enterprises, competing teams for those things, right? You need DVAs to manage the stored prox, app developers to manage the application logic. And sometimes that gets a little messy. Where we've seen the modernization going is a lot of that logic is existing within applications now. So we've seen a lot of our customers especially ones that have, have done legacy database migrations is actually move those stored procedures into microservices, which is a pretty common pattern now. Right. Yeah, I think that's all over the place, right? It just makes sense. I mean, using the database to do some sort of distributed execute, like from that layer, like from that kind of complexity, doesn't make a whole lot of sense, right? I mean, let's use the database for what it means, right? So what about performance, Chris? And, you know, how do people run their own kind of performance benchmarks? You know, what have we published from that sort of point of view? I think everybody always thinks about performance when they think about a database and you know, being distributed and doing transactions and all these different things is often difficult. So how do we talk about performance in the context of Cockroach TV? Yeah, great question. So we have, we actually have some great docs on performance. One thing I want folks to know about though is that a lot of folks spend time creating workloads. We have something within Cockroach TV. If you go to our documentation called Cockroach workload and this is a great way to actually just go to sniff test your setup. We have a whole bunch of different workloads that we've prepared for you. Folks might be familiar with TPCC. It's a very standard benchmark that's utilized out in the database realm. We also have YCSB and a few others. And once you set up your database, like I said, go run one of these tests. You can run a sniff test to see how you're performing. It gives you an idea of the efficiency that you're running at, the ops per second. You can look at the metrics in the database to see how things are performing. It's a great way to go just see how things are. From a performance perspective, we're actually really fast considering that we're a geo-distributed database. You know, things in signal region, you're pretty much bounded by speed of light. And that goes for single region and multi-region. You know, as you start building multi-regional clusters, speed of light is something you definitely have to consider. And then what we'll do is we'll work with you to figure out, hey, what's the performance and resiliency trade-offs that you wanna make? And then we could figure out, hey, this is the best way to set up your database so that you have low latency and then a high degree of resiliency within your database setup. But those are usually the two trade-offs. Just like you have trade-offs with latency and throughput, when you're setting up a cockroach cluster, you're thinking about the trust between resiliency and latency. I mean, excuse me, resiliency and yeah, and latency. Right, so last question, they kinda slowed down here. How do I get started, you know? So, and let me just blow that out a little bit, you know, can I run this outside a cloud environment? I know you were using a cloud environment. Can I run it all local? How do I get started with CockroachDB? So there's two questions in one there. Yeah, so you have two ways you can go do this. You can go ahead and download CockroachDB and go off and going. We have a whole bunch of tutorials online that you can run through. We also have Cockroach University. We'll walk you through how to utilize the database. It's really simple. It's a single binary, you know, you drop it, use your local bin and off you go. It's really, really simple. And then you can also use CockroachCloud, right? So this is something that we offer now that it's an always-on service. You can go to get CockroachDB, go to CockroachCloud, sign up, and we'll give you the connection URL to go connect to your database and you can start there as well. Like I said, the documentation is terrific to just go and get started for creating your own account, creating your own cluster. It's all here to get started. Cool. Well, awesome. I don't see any other questions coming in. I know our friend Tim Vale, who actually is a Cockroach employee is out there answering to everybody. A couple of questions there. So thanks, Tim, for that. You're the best. Thank you, Tim. I know, right? Yeah. But with that, I think that's all we had. I think I had like one last slide. So just as a follow-up, Chris mentioned that our documentation is really world-class. I've never worked at a company with documentation as good as ours. I mean, the whole fact that we give you workflows to run on this sort of stuff, if you're gonna get started, you're gonna start running and start playing with Cockroach. Getting it via CockroachCloud, it's pretty simple, pretty straightforward. Anybody has any issues? We also have a Slack community of which people use to get self-support as well. So I think it's a CockroachDB.slack.com. I think that's the community. But we've got a lot of stuff out there about running databases on Kubernetes. I mean, this is kind of part and parcel of what we do and really critical to the way that we run. And so there's lots of information out there. You can go out and please do check this stuff out. If you're interested in talking to us, we're happy to talk to you as well. Just reach out through the website. It's really the easiest way. And as Chris said, you can download and start using it today or you can actually spin up in CockroachCloud. So Chris, one more slide. And I think this is, I love what you do here. I think the replay link will be sent out to everybody. And then Chris, again, what is in your Git repo? Okay, yeah. So it's the entire demo that you saw today. So it has the setup, it's creating a cluster of EKS. It has setting up the CockroachDB stateful set. It will deploy the Flask application that you saw. And then it has a script to do the self-driving demo that you saw today. Cool. Awesome. So I love demos that give you code as well. So thanks for doing that, buddy. And thanks for being on the webinar today. Thank you everybody for joining. And Lindsay, do you wanna take us away? You guys did a fantastic job. Even I learned something. So thank you so much. Thank you everyone for joining us. Thank you CockroachDB and Chris and Jim for just having a wonderful webinar this morning. I hope everyone stays safe and sane during this crazy time. And we will see you on the next one. Thanks everyone.