 Welcome to today's PostgreSQL World Webinar, Yugabyte DB Distributed PostgreSQL, why and how. We're joined by Frank Pashou, developer advocate at Yugabyte, who will talk all things Yugabyte and answer questions like why another database instead of an extension. When can Yugabyte be substituted for standard PostgreSQL and why? What is Yugabyte not a good choice and PostgreSQL is the best fit? I love that you're gonna get into that. And what is distributed versus sharded when it comes to scale out? My name is Lindsay Hooper. I'm one of the PostgreSQL conference organizers and I'll be your moderator for this webinar. So a little bit about your speaker. Like I said, Frank is a developer advocate for Yugabyte, a PostgreSQL compatible open source distributed SQL database. He's also Oracle ACE director, Oracle certified master, AWS data hero and an Oak table member. So with that, I'm gonna hand it off. Take it away. Thank you, Lindsay. Yeah, so you introduced the topic. I will be quick on the intro. I'm developer advocate for Yugabyte. It's a beautiful SQL database. I will explain what it is, but especially because it's PostgreSQL compatible and we are in a PostgreSQL conference. I will focus on the differences and the similarities with PostgreSQL. What is similar to PostgreSQL? What is different? And also we can go into the detail about how it was built with part of PostgreSQL and part of new development. And if there are any questions during the talk, then do not hesitate to ask them. I can adapt the content to your questions as well. So about myself, the most important is that you can reach me on Twitter or by email, but I'm a lot on Twitter, but I'm also easy to find on LinkedIn. So do not hesitate if you have any question later. And I will start directly about why PostgreSQL compatibility for a new database and something that probably everybody has seen in the past years, the PostgreSQL popularity. So this is the database ranking engine which is based on popularity in social media, stack-over-flow question, et cetera. Basically, we see the old databases at the top, but not growing anymore. And some are growing, especially PostgreSQL, it's really growing in popularity more and more. And I've also put a few of the distributed SQL databases, those who started with Spanner and CoCoachDB, TIDB, YugabaiDB, knowing that most of them are going to PostgreSQL compatibility except TIDB, which is more my SQL compatibility. So basically, PostgreSQL is a popular database. And I would say the most popular for OLTP because we don't need to forget that my SQL is still very popular and still growing. But we see more PostgreSQL when it comes to OLTP to complex applications, complex queries, joins, foreign keys, constraints, et cetera. And also PostgreSQL is the, if we compare with my SQL, so the two open source databases that are very popular, PostgreSQL really follows the SQL standard where my SQL doesn't. And this is also important because besides being PostgreSQL, if you follow the SQL standard, the gap with other databases is minimal. So we cannot say it's the best open source database, but it's probably the most popular for OLTP and the best when you want SQL compatibility. And PostgreSQL has a lot of features built in, but also with the extensibility provided by PostgreSQL where people can add many extensions. So yeah, this is about the popularity and then it makes sense. I will explain later how you got by started, but it really makes sense to follow PostgreSQL compatibility today when you build a new database. And this is not only when building databases, it's also what we see with the cloud managed services. I've put just four pictures. You can see the colors for AWS, for Google, for Azure, and even Oracle is now talking about a PostgreSQL compatible service on their cloud. Basically, all cloud vendors now have realized that they need to go to PostgreSQL compatibility and propose many managed services that are more or less compatible with PostgreSQL. For example, AWS started with RDS where you can run PostgreSQL, which is very similar to the community PostgreSQL. And they have also Aurora, which is a bit different in the storage, but still very compatible. Google has the same kind of offers. Cloud SQL is mostly the same as the community PostgreSQL. Other ADB is more like Aurora in AWS. And Google Spanner started as a distributed SQL database with very limited SQL. And now they are going to PostgreSQL compatibility more and more adding features that are PostgreSQL compatible. And Azure also, they renamed it different time. It was called hyperscale. It's now called CosmoDB for PostgreSQL, but they also have multiple services based on PostgreSQL. And even Oracle has announced a PostgreSQL service, which is not yet there, but it shows that even a database company like Oracle realized that they need to provide a PostgreSQL managed service. So why being PostgreSQL compatible? There are many two big reasons. The first is the familiarity for users. And I will take just an example. I don't know if it's large enough, but if you, we build a new database with YugaBite. And if people search for something like how to find the first rule for each group buy, that's the kind of question we can find on Stack Overflow for SQL database. If you search that on YugaBite with the YugaBiteDB tag, you have no reason because YugaBite is quite recent and there are not a lot of questions. But if you search the same with PostgreSQL, then you have a lot of results. And I took the example there with the most voted question. So that's the first reason being PostgreSQL compatible. The first reason is the familiarity where people can use the knowledge they have or the knowledge base in Stack Overflow or blogs or anything. Everything that is about SQL because it is PostgreSQL compatible, you have the same answers for one and the other. And maybe I can show you, so this is the Stack Overflow page about this question for PostgreSQL. And I've seen they even show an example in DB Fiddler where you can run examples like you run the test case and you see it. And this is PostgreSQL 13 because at the time the question was there, it was 13, you can see the same in 15 and you have also YugaBite. So DB Fiddler is not yet with the latest versions of YugaBite and this one is a bit longer to start because it starts in a container. But basically you can verify that exactly the same SQL DDL and DML that was the answer for this Stack Overflow question is exactly the same running on it. Okay, and let me just go back here. So the familiarity of it. Before being compatible to run application, being familiar, not having to learn a new language, that's something important and this is why people are going to PostgreSQL compatibility. Familiarity with the syntax but also all the ecosystem and tools when you have a tool or a framework that is compatible with the PostgreSQL dialect, then probably it will work on all PostgreSQL compatible databases like YugaBite. And we benefit from it every day. For example, when there is something new, a new tool or new application for PostgreSQL, just try it on YugaBite and that works. And the second reason for compatibility is portability. If you already have an application running on PostgreSQL and you want to test how it works on a scale out database like YugaBiteDB, you can just change the connection string and because it is compatible, the protocol, the syntax, the query, the bit of year is compatible, then it's easy to migrate. Migrate or simply test it. And in both ways, you can also start to go on YugaBite and then finally realize that this is not what you wanted. You don't want to distribute, you don't want to scale out and you want to go back to PostgreSQL. This works in both ways. And between PostgreSQL and YugaBite because it's compatible, but also when you come away from another database, it's quite similar because it's the SQL standard and also PostgreSQL is very similar to a vehicle in some behavior. So those are the two big reasons for the compatibility. And then a question can be why being just compatible, why forking PostgreSQL, I will explain the architecture. YugaBite uses a fork of PostgreSQL and adds a different storage to it. And the fork means that when we want to follow the PostgreSQL versions for the compatibility for the new features, then we have to merge it. So why forking and not just using PostgreSQL or adding an extension to PostgreSQL, it's quite difficult currently because a distributed database is very different from a monolithic one. And even if PostgreSQL is very extensible with extensions, there are many places in the code where things must be done a bit differently. This is a short diagram that shows the PostgreSQL part of YugaBite where you connect to a PostgreSQL back end and basically what this back end does is the rewriter and then the query planner and then the executor. And of course there is the optimizer to find the best plan and the main interface with the distributed storage of YugaBite, the main interface is in the table and access method, but they're also part of code in different places because you do not plan, you do not optimize an execution plan in the same way when you know that data is distributed, for example, I will take other examples. So basically PostgreSQL has a good extensibility, but for the storage option, it is a bit limited the places where we can plug a different storage. And basically there are two interfaces provided by PostgreSQL when you want to store data in a different storage, other you are at the high level with the foreign data wrapper. So it's not really plugging a different storage, it's really plugging a different database. And there is a lower level that says the table and index access methods, which can be used when you want to store the PostgreSQL a bit differently, but this is really low level and not ideal when the storage is really different as a distributed one. And for example, there were a project in the PostgreSQL extension to plug a different storage, to do ZIP and do like Oracle does. And the project is now dead, it was probably not very easy to plug a different storage. And then that's the reason why in the current situation, you got bite, but also all the other databases that propose a different storage for PostgreSQL, usually they fork it to be able to have hooks at different places more than the places that are currently there in the code. So not enough hooks for specific storage. This may change in the future, there are more and more databases trying to plug a different storage to PostgreSQL and maybe at some point the PostgreSQL community will add more hooks for those, ideally the same hooks for all those projects. There is also another thing where when you build a new database, you don't want to inherit many problems that come from the current storage of PostgreSQL and vacuum transaction ID wraparound are the most known issues that are encountered in production because of the current way PostgreSQL is storing things. And to explain that, I will take an example from an article written by Rick Branson. The title is 10 things I hate about PostgreSQL, but it starts the article saying that PostgreSQL is a really good database. It's just PostgreSQL is really good, but there are 10 weaknesses. And he listed them, I will not go into the detail here, but you see this transaction ID wraparound, you see the vacuum and other things. I'm saying that to say that the main weaknesses in PostgreSQL are all located at the storage level because of course it's more difficult to change things in the storage for a database that is there for many years and decades. And basically if you look about the components that are concerned by that, the vacuum is really a problem, a source of many problems. Hip tables basically PostgreSQL does like Oracle does by default, stores table rows in a hip, not in the primary key. And then all indexes, including the primary key is a secondary index. Most other databases are storing the tables within the primary key and have secondary indexes. And some problems come from there. Also many of the problems listed in this article are about higher ability. This is something that we want to solve with scaling out and storage. Basically saying that the idea of having a fork is really to change the storage, to add a new storage that can scale out without the legacy of the problem that comes from an existing database. So now the real question, do you need to scale out because PostgreSQL is quite good as very good performance and everybody knows that when you are a very big company you need to scale out to multiple servers and even multiple regions. But for most enterprises, people just think about the performance and say, oh, everything runs on one server, I don't need to scale out. But scaling out, so running a database on multiple nodes gives many benefits. The first one is availability. If you run on multiple nodes where there is replication between the nodes, then if one node is down, everything continues. If you run on a monolithic database like Postgres, if the node is down, you have to fail over to a standby database. And then this takes time and this has many problems. You have to reinstate a new standby, et cetera. So high availability, even if you don't have a big database, even if you don't need high throughput, you may still want to be resilient, especially in the cloud where it can happen that the network is called for a few minutes between availability zone, for example. And that's for failures, but also for upgrades. Currently, if you want to upgrade a Postgres database, you have to stop the application for several minutes or hours, depending on the size and the way you do the upgrade. When you have multiple nodes that are replicated because you can take one down and start it with the new version, then you can do rolling upgrade while the application is running. So that can be a reason. And cloud native, the meaning of it is that when you deploy on the cloud, for example, if you go to AWS, you have regions with at least three availability zones, you want to benefit from this infrastructure. You have three availability zones where the latency between them is low, one millisecond. And then you don't want to use them just as a standby, having all the connections, all the data processing in one availability zone and two standby that are there just in case we need to failover. With this kind of infrastructure, we want to distribute the load over the three availability zones. And also there is this idea from the cloud. If going to the cloud can be expensive if you go there in the same way as you did on premises, like provisioning big instances for to be sure that the peak of workloads during the year will work with that. When you go to the clouds, you want to benefit from the elasticity. When low with low resources, most of the time, and when you have a peak of activity can be a peak in the day or a peak in the year, then you want to add more compute power or more storage if it's about the volume of data. And you want to separate it. You don't want to change completely a server with its storage. So that's also the idea. And the extreme idea of it is to providing a serverless service where when you don't use the database, you don't use any compute. Of course, the storage will be there because a database stores data persistently, but the compute you need it only when you run query, when you connect and you run query. So if you have develop instances for developers who work only during business hours, you can shut down the compute during the night. And the best is that it just starts when it is used so that you don't have to schedule in advance when it runs. So those are many reasons to scale out. And another one is geo distribution because today you may have an application with a database that serves users not only in one region and for latency reasons, but also for data governance, you may have the need to store European users in a region in Europe, US users in US, et cetera. And this can be quite complex if you build different databases for that. So when you have a distributed database that can scale out where you can add multiple nodes, you can put nodes in different regions and tag the data so that it goes to the right region depending on where the users are. Okay, and also there are some people who want to go on the cloud but hybrid between multiple cloud vendors or with some nodes on premises, some nodes in the clouds. I will go fast on that, but do not hesitate to interrupt me if you have questions. I'm going fast, there are more information in the slides but I also want to do a little demo or take questions at the end as you want. Basically, if we take the history of SQL databases and the need to scale out, it started with no SQL because scaling out with all SQL features and by SQL features, I mean, I seed transaction for in case secondary indexes, that's not easy. So the first attempt to scale out was no SQL databases, which gets rid of many features, the data dictionary, the optimizer for in case, et cetera, but can scale to multiple nodes. And then started some new databases which realize that going to no SQL without all those features makes the application a bit more complex because the things like transaction that the database doesn't handle, then you have to do that in the code and the new SQL trend was to use monolithic databases but do shorting on top of it to be able to distribute it. And the good example is CytusDB where you decide some tables are replicated on all nodes and some are shorted, but each node is still a Postgres database which can have its standby and then it becomes complex if you have multiple nodes. And there are also other database that we can also call new SQL still using the monolithic Postgres for this example but with a different storage for different purpose. So that was, for example, Aurora where the instead of storing data in local file, it stores it in the distributed disk storage across availability zones. So you still have a monolithic database, only one place where you can connect to read and write your data, but the data is replicated into multiple availability zones. And recently, Neon is starting the same idea but open source also to provide serverless applications and clone facilities and all that. And then there is distributed SQL and that is the big difference. So those names are decided by marketing but the big difference is that instead of taking monolithic databases and changing the storage or adding shorting on top of it, the idea is a bit the opposite, building distributed storage and putting SQL on top of it. And all the examples comes from Google Spanner where Google had a lot of applications in no SQL in big tables and realized that it was quite difficult to maintain them and started to add SQL feature to a distributed storage and then go coach DB and then you go by DB where the decision was really to reuse Postgres so SQL on top and those are the four well-known, TIDB is more MySQL compatible. Spanner started as limited SQL but now it's going to Postgres compatibility. Go coach is protocol compatible and some syntax and you go by it really reuses Postgres. And about compatibility, it's quite important. It's easy to say we are Postgres compatible but there are many levels and I will put the four levels like that. The first one is using the same protocol so that people can use the same drivers. The second step is the SQL syntax being compatible maybe with some additions. For example, when you want to distribute data on shard in UGAbyte, you have some keywords that can add some information about sharding, hash sharding or hunt sharding but the syntax is compatible so that an application, a SQL statement written for Postgres can run the same on UGAbyte or whatever. Then there are features and triggers, thought procedures, all those features that are there in Postgres that maybe you don't choose everywhere but at some point you will need one of those features and this is really where we're using Postgres gives all those features without additional development and then the runtime compatibility. So this is really the behavior. Even if your SQL statement runs the same on another database than Postgres, that doesn't mean that when you will have multiple users reading and waiting to the same data, that doesn't mean that you will have the same behavior with a concurrent access. So locking isolation level, the I in ACID compatibility means having the same semantic for those isolation level than Postgres. And another level is the extension compatibility where even if you are really similar to Postgres, do you accept all the extensions that have been built for Postgres? And then probably there are some extensions that are at the SQL level that can be accepted and those that are at storage level, if you have a different storage, then it is different. So basically the best compatible with Postgres is Postgres itself. Any other fork will have less compatibility. Even the managed services like Cloud SQL in Google and RDS Postgres in Aurora, even if they're mostly the same as Postgres, there are a few things that they have to change because it is a managed service. They cannot give super user access and all that. But the best compatible is of course Postgres. And then the less compatible is in the distributed SQL databases is Spanner because they decided later to add Postgres compatibility. But they're working on that. So currently they have protocol compatibility and some syntax compatibility. So I put that in the middle of it. CodeCroachDB is quite the same. They have driver compatibility, some syntax compatibility, but not all features. For example, you don't have stored procedures in CodeCroach. Maybe they will implement it, but that will be one by one. And that's different from Postgres. I will explain the picture later. That's different from Yugabyte where because we reuse the Postgres code itself, then all those features like store procedure triggers come as is. We just have to test them to be sure. And also as the goal of Yugabyte is really to be Postgres compatible, the runtime behavior, the locking, the isolation level are really compatible. So you can really take an application and run it without any change. This is an excerpt from a paper from CodeCroach that explained the CodeCroach wire level compatibility. They explained that they use the same drivers for the ecosystem using the same tools. But basically, it behaves differently from Postgres and require intervention in the client-side code, so the application. For example, the isolation level, if you do not support read committed, which is the most used in Postgres, if you do not support it, you can use serializable isolation level, which is really good in terms of anomalies because you have no transaction anomalies. But the problem is that you have serializable errors and then the application has to handle the retry. In Yugabyte, we provide read committed for those applications who expect that there are no serializable errors. And going fast on how Yugabyte was built, I already mentioned that it was built with a distributed storage and on top of it, Postgres, basically the funders have been working in many database vendors like Oracle, but mostly also on HBase and Cassandra at Facebook. And they started on the Google Spanner paper building the distributed storage, not only storage, but transaction. Basically, so what is called dogDB is a distributed key value transactional. And they started to open the Cassandra-like API, but doing more than Cassandra because fully transactional, because secondary indexes, et cetera. And of course, it was not written from scratch. Basically, data is shorted, distributed, and each short is based on HuxDB. And the replication is done with the Haft consensus protocol to be able to replicate in sync to multiple nodes. And this was based on Apache Kudo. And then a few years later was plugged the Postgres, the SQL API using Postgres. So taking the Postgres code, forking it, changing what must be changed and plugging, so that instead of storing locally in local file, the Postgres code calls the dogDB APIs to store in this distributed engine. So basically, Yugabyte is deployed on multiple nodes. Each node has a dogDB node with part of the data, but has also a Postgres endpoint, which is a stateless Postgres. Everything that has a state that must be persisted goes to the dogDB layer. And the picture about it, so the marketing side of it, the user side of it, is having one logical database with all SQL features, but behind multiple nodes storing it and replicated and in sync with the Haft protocol and being able to distribute that across multiple regions if that is the need. And the architecture of it, this is the dogDB, so the distributed transactional storage layer doing automatic sharding, so based on the primary key of the table or the index columns, this is automatically distributed. There is load balancing to be sure that all nodes do the same work and have the same auth data of the authspots and transaction across it. So some transaction, most of SQL transaction will update the table, check a foreign key, update secondary indexes and those are in multiple nodes. So distributed transactions are managed there and the Haft protocol and on top of it, Postgres is just one API on top of it. I mentioned that there is also a Cassandra-like API and maybe one day another one, why not? If there is a need for it, it can be my SQL, it can be anything, but currently the Postgres API is the most popular. And the idea is that this distributed storage is just nodes started on VMs or containers anywhere and you can have nodes in different cloud vendors and premises, et cetera. And the Postgres application, the application written for Postgres using the Postgres driver connects to the Postgres endpoint in each. Okay, now if we want to look more about where in the Postgres code, this storage is plugged. So the best is to look at it because you can buy this fully open source like Postgres and if you search in the Postgres code, you will see a sub directory which is the Postgres fork and in this directory you can search for is YB relation. A relation in Postgres is a table or an index and if it's a Yugabyte one, so a distributed one, then things may be done differently and usually you will see this hook with if is YB relation calling a different code than Postgres. For example, here we are on the table scan, scanning YB table and here you see how Postgres really think that all tables are YB table. Actually Yugabytes tables are not YB tables but in the scan, we call a different function and the different function is in different files. Where there is a YB scan and YB LSM, I will explain quickly. We don't store tables in YB tables and indexes in bit trees. We store everything in LSM trees. I will explain a bit more about it or we will see if we have a question but I will go fast on that there. But this is different places in the code where we have different storage to scan, to read but also to modify data and those call the distributed storage. If you want to see it when it is executed, this is a flame graph showing the different stack. Let me just put it here if I can download it. So the flame graph basically shows all the different call stacks in my system. I focus on D1 from the Postgres endpoint and I've put some callers to show what comes from the Postgres code itself and what has been added by Yugabyte. Basically in Postgres, when you connect, you are there, you are in the Postmaster and there are a few things where Yugabyte does something differently but the big one here is a wrapper where the idea of it is that when you are in the distributed system because of clocks queue, the different clocks between the servers, there are some cases where you cannot guarantee consistency and then you have to read again at a different read point. I will not go into the details but basically that's something that you don't have with a monolithic database and then Postgres doesn't have to handle it but in the distributed system, there are some places where you have to restart it and instead of sending a serializable error to the client so that the application restarted, the code itself can do it in some cases. So this is an example where Yugabyte adds some code if something can be restarted automatically then it will be started automatically and then the executor code from mostly Postgres and that's some point, for example, this is the insert, then it calls the Yugabyte code that will build the buffer to send to the different tablet servers to distribute the data. So I will not go again too much into the details. Tell me if you have questions about that. I will either answer some questions or do a demo or do the same at the same time. A bit more about the raft consensus and LSM tree. So the Postgres layer sends operations to this distributed transactional key value data store that is called DogDB and basically those are the new values. So either you read for a key, the primary key or the index key or you update some colon values and there you have the new value and it must be timestamped to order the transaction operations and there is a logical clock that is a clock consistent in the whole cluster and this goes to the raft protocol because you don't want to write only in one place. It goes to one, if you update one row, it goes to one short, but this short will be replicated. So the short we call them tablet and it is replicated to multiple tablet peers and this is a raft group where you have a leader and follower and the raft protocol will guarantee the atomicity of just writing in different places. And it is stored in the LSM tree, log structure merge tree where we just add new values quite similar to Postgres, but in a different format so that we can do MVCC multi-versioning and the most interesting thing is that this LSM tree different from the battery is happened only and first it goes into memory and when the memory is full for a tablet it can be about 100 megabytes when you insert or change it's full and then the memory is flushed to a file, SST file and this file then is immutable. So you accumulate files with the versions of your rows or column values and at some point you accumulate them and you have some kind of garbage collection but very lower level than vacuum so without looking anything in the background that will compact them to remove the intermediate versions. Okay, then let's go for a demo and the idea of the demo is to run on Docker. I have Docker on my laptop here and I will run the command data in the slides. Basically, I have a distributed database. I start one node but my goal is to be resilient and then have multiple nodes. So replication factor three, I need at least three nodes and here I'm just starting them. You see the different containers and I'm nearly running the same thing the only difference is that the second one and the first one joins the first one. So the YB0 is the first one and I join them. So that we are in the same cluster. I can see that from Docker and we also have a web console, sorry, local host. So the web console on local host, why local host? Because I forwarded the port of this web console to my laptop. So here I can see the three nodes for the master. I will not go into the detail. The most important are the tablet servers. I have three nodes that can run queries and store data and for the moment I have no data in them. I will add data and because it is PostgreSQL compatible I just connect. So I also forwarded the port 5.4.3.3 here. I can connect with PostgreSQL clients, PSQL and we have some additional functions. For example, one that shows the different nodes when you are connected. It's quite convenient. And if I want to see where I'm connected to one node here, I'm connected to YB0. And I can connect to another one because I forwarded the port to different port on my local laptop. I connect to another node. I can see the listener address is another node. So that's the idea. I see it as a whole database but I can connect to any node and this distributes the connection and then the memory used by the connection and the CPU. Now let's create quickly a table. So I create a table with a UID. This is why I use the PostgreSQL extension PGA-Crypto. I create a gene index to show that everything you have in PostgreSQL is possible and I will insert some data. This inserts UID and the text that tells me where I was connected when I inserted it. So here I was connected on YB1. If I connect on another one, for example, and I insert YB2, I will connect to the first one I tunneled. I think if this one I insert, okay. So YB0 and now if I select from my table, so I connected to different places, inserted a row and finally I can create and I see all rows because logically it is only one database. It's not like the shorted database where you are in different places. So I can create and create in one transaction. This is a transaction and now I will run this kind of insert in a loop, inserting new rows to show you the activity we have. So if I refresh the tablet server list, first I can see that I'm using some storage there and I can see reads and write operations. I'm doing only inserts, those are write operations. Even if insert has to read some data, those operations are the operation between the Postgres layer and the distributed storage. And here we write, I refresh and you see that I'm writing. And quickly I will show you the most important there. So yeah, must also show you that this table demo has multiple tablets. Those are the charts based on the hash on the primary key. So a row can go to a different chart and this is the raft group where there is a leader and followers. When you read and write, you go to the leader but it is distributed because some rows have their leader in YB1, some rows have their leader in YB0 and YB2. Okay, and now what I want to show you, so let me just show the application running and I will just stop one node there. And you can see that here my application is waiting because I stopped it. There were some leaders in this node. So the first thing that Sugabai does when it detects that the node is down is elect a new leader in the other nodes. And then there is also a TCP timeout that is configurable here. You have seen 15 seconds where the application was just waiting to get connected to the right leader, but waiting, nothing failed, no error message. This is much faster than failover to a standby database where all connections have to be initiated. And if I look at, so here if I refresh the list of tablets, you can see, so I stopped the YB2. You can see that I still have some tablet peers there, but only followers because the node is down. All the leaders were reelected in the available nodes. And REST protocol basically waits for the column so you can write to the leader in YB1, get acknowledgement from YB0. And even if YB2 is not answering, everything continues. And now if I just start it back, so I'm simulating unavailability zone failure for a few minutes, for example. So quickly the node is back, will catch up the latest changes. And we will see quickly that it will even elect new leaders on it. So not yet the case, but if I refresh, I expect that you go by it automatically as rebalanced the leader to go on YB2, not the case yet, but that will come just checking that the application is still running. When you scale out, there is no problem. And we will see that YB2 will take, of course it's a bit longer when starting on container, actually I wanted to pause it rather than starting, but here it has to start and then we will see it running. And so I can see writes everywhere, then I guess that some followers in YB2 have been elected as leader. Yeah, for example, this one. So that's the cloud native part of Yugabyte. You have operations or failures at cloud level without any operation in the database. The application continues, the data is still available and the load is rebalanced. And I will quickly show the scale out of it and then see if there are some questions. So let me just open another terminal so that we can still see the application running. So here in a loop, I'm starting three more nodes, three, four and five. So they are starting and I will see them. So you can still see the application running, of course. And if I look at the tablet servers, you see that those new nodes were detected. So YB5, 4 and 3. And at some point, they will take also written writes. So here I'm simulating the case where I'm running low because I don't have a lot of activity on my application on my database. And then there is a peak of activity because it is Black Friday or because the customer carrier sends so some mailing to the users and they all connect. I see that the activity is increasing. Then I quickly double the size of my cluster adding more nodes. And if I refresh, I can see write operations on all nodes including the new ones. How is it possible? If I look at the tablets, I can see that those new nodes, so YB5, 4 and 3, those new nodes now have some followers and even leaders. So that's automated by the database. A new node is detected, then automatically a new follower will be created on it to be able to rebalance the data. And when you have followers rebalanced, then some of them will be elected as leader because the leader is doing more work. This is where the reads and writes are happening and you want to balance that as well. Okay, so basically the two things that were there the possibility when you are in replication factor three to have one node down, if it is a cross three availability zone, one availability zone down, if it is a cross three regions run region down and the application continues just hangs a few seconds for the leader election if it has to read and write from this leader. Okay, so if you want to test it, that's quite easy. If you have Docker, of course there is documentation about everything to run it but you can run a three nodes cluster and play with it stopping nodes as you want and connecting to it just with a Postgres application, you can connect with the beaver, you can connect with anything Postgres. Okay, we are nearly after one hour. I don't see any questions actually. I don't see a lot of people. So probably most people will see the recording of it. Let me just summarize quickly what we have seen. The reasons for a distributed SQL database can be Yugabit or anything. Scale out for elasticity and resilience to failures. The reason for Postgres compatibility no need to learn a new database, a new syntax and benefit from the ecosystems of tools or knowledge base. And the reason for a fork rather than implementing new features are just benefiting from all the Postgres features immediately with the best compatibility and same barrier so that you can build a new application that can run on Yugabit or Postgres but also use an application that has been built to run on Postgres and test it or run it in production on Yugabit. And the main difference with Postgres because it's not equal to Postgres, the main difference is that you have to think a bit more about the data model. Always think about the data model but if something is not optimal in a single node server then it will be worse in a distributed database. So think about the primary key, how you hash it or range it or short it by range and think also about the new access pattern. The index only scan is very efficient in Yugabit. More than Postgres, we have skip scan so there are also more possibilities even if the same queries run, you may think about the things that are added for performance in Yugabit and don't worry anymore about vacuum. The garbage collection of the SSD file is completely different, different level, no consequence in transactions ID and locking. And when it can be an alternative to Postgres, it's not for everything, mostly LTP. Currently it's mostly optimized for OLTP. There are some analytic push downs and there will be more but there are some complex query group by that we'll have to get all data and aggregate them by the Postgres layer until all that is pushed down. So for the moment, not for data warehouses, more OLTP and the analytics that you do in OLTP. Cloud native environments, everywhere, VMs, container, et cetera. And multitenant also because you have no, you can scale and you can also dedicate some nodes to some data, to some partitions and your distribution when you want to run a database with users everywhere. Okay, so we are after one hour. Thank you very much for being there. If you have questions later when watching the recording, do not hesitate to contact me. I'm developer advocate. So my goal is to let you know about your Gabbard, answer your question and also get your feedback if you use it, if you test it. I'm interested to know what works and what doesn't work. Okay, thank you very much. Brilliant. Thank you so much for this wonderful presentation. We haven't gotten any questions in but we do have a comment that says, great session. Thank you. And I think that sums it up really nicely. So again, Frank, thank you so much for your time and the thought that you put into this. To our attendees, thank you for spending a little bit of your day with us. And I hope to see you at future Postgres World webinars. Have a good one now. Bye. Bye-bye.