 Okay. Hello. Hi everyone. Thank you for joining my session. I really appreciate it. I see a lot of people. I wasn't expecting so much. So my name is Miguel Araujo. I'm a software engineer at the Mexico middleware and clients team at Oracle. And for the last few years I've been focused mostly in a bit cluster, this new project. Most specifically I'm leading the implementation and design of the admin API, which you'll get to know what it is. So the agenda is short, has to be quick, only 20 minutes. So I'll start with an introduction for about the new cluster for those who are not familiar with it. Then I'll move to talk about the shell and the admin API, which is the main topic for this. Then I'll do a live demo split into two, about setting up a cluster to show you how easy and fast it is. And then I'll show you a quick demo about the automatic failover. Okay. So, you know, the cluster, a little bit of background. It's a known fact that pretty much all organizations require their most critical systems to be highly available. Most specifically data. They cannot afford to have their data, they have data loss or data not available. So high availability is critical. And how do you achieve it? I mean, if I ask around, pretty much all of you will say that is replication. That's absolutely correct. So high availability, critical factor, replication, common solution. My SQL has had great support for replication for quite some time with a classic master's sleep replication. But people is looking for other solutions, for built-in HA solutions with everything integrated. And recently, my SQL has provided group replication, which is replication plugin update everywhere. So you can set up multi-master setups. It's virtually synchronous replication. And the most thrilling feature, at least for this use case, is high availability. So in group replication, servers are parts of groups. And there is the notion of group. There is the notion of members of the group. So when an instance joins a group, all the others know about it. When it leaves the same and so on. This allows group reconfiguration. And we also have distributed recovery. So when the instances join the group, they automatically recover all the missing data. This is all powered by a group communication system. It relies on group communication primitives. And it's in-house implementation of the famous PAXOS consensus algorithm. However, this looks really promising. And it is. But it's not that easy to set up and maintain. You need some technical knowledge about it. And then people wonder how to configure the applications, how to integrate all the components. So you have your full stack HA solution. And this is exactly the vision that we have for In-N-Db cluster. There is a single product, MySQL. No external components with high availability and scaling features back then. Providing an integrated end-to-end solution that is easy to use. That's one of the main goals of In-N-Db cluster. It's a usability. So three main characteristics. It's a fuse with built-in HA. Another box solution with everything integrated. And this sets the base for scale out and high performance from now on. So going bottom up on the stack of the components of In-N-Db cluster, we have the data servers running the MySQL and the group replication plugin. Then we have the application servers with the router. That is the other component that I didn't introduce yet. So the router is a very lightweight middleware component. It provides transparent routing between your connections to the servers with the data. So it hides all the TCP ports of the servers behind one port for read writes and another for read only. And to set up and manage everything, you have the MySQL shell with the admin API. That is the API to set up and configuration of In-N-Db clusters. You have the clients and you have the full stack HA solution. So again, one product, MySQL, full stack HA solution and easy to use. So next topic, the shell. What is the shell? The MySQL shell is a brand new interactive multi-language interface for MySQL. It supports development and administration for the server. And it can be used to perform queries and updates or whatever like you did with the old client. But on top of it, you can also do administration operations. And for that, we have APIs implemented in the shell, which we call scriptable DevOps APIs because you can do development and operations. And one of the main goals of the shell is to be an unified interface for both developers and DBAs. On top of that, it's very intuitive and it's very easy to use. So features, recap. Multi-language support. We have JavaScript, Python and SQL in the shell. You don't have to restart the shell. You can just switch between languages. You can do both interactive and batch operations. And with the development APIs, we support both documents and relational models. So I don't know if you're aware of it, but I've already talked around here about the document store. The shell has support for the document store with a fluent API. That is the Dev API. And we also have the support for the classic relational model. What about the admin API? The admin API is an administration API implemented in the shell, which allows you to create and manage clusters. And the main goal of the admin API is to hide the complexity of configuring, provisioning, orchestration of NDB clusters. It's very simple, very straightforward. The goal was always to be very usable and usability is always a top concern. So it doesn't require my SQL expertise, but at the same time, it's flexible and it's powerful. You can go from the basic commands and basic operations to more complex ones. It's available currently in both JavaScript and Python in the shell. And we'll see in the future. So I'm talking about how easy it is to set up and do everything. So let's do a quick demo to show you how to set up a cluster. Okay, so let's start the shell. Can you see it? Yeah, it's cool. So this is how the shell looks like. You have this eye candy. You have auto-completion. I can show you the online helper with the commands, general stuff. And what I want to show you is that in the shell, it automatically initializes global objects. So the DBA is the one for where the admin API is implemented. Then you have mySQL, mySQL X, that is the document store, the dev API that I was talking about, and other objects. So what we care for now is the DBA one. You can see the auto-completion. It's quite cool. You can call the online helper and you see the, maybe this is too big. You can see the available commands of the admin API. So for this demo, I have three virtual machines. Linux running on three different hosts, so simulating a real scenario. And all of them have mySQL 8 running. So it's in IC1, IC2, and IC3. So let me show you how the shell looks like. So I'll establish a connection to IC1. So I'm going to show you. You switch to SQL, just like this. And you can run commands, for example, version, hostname. OK, so play normal SQL. So let me switch to JavaScript. So let's go on and actually create the cluster. So I'm connected to the first instance. IC1, that's where I will start my cluster. That's what we call the seed instance, the first one. And the first thing we'll do before creating a cluster is to verify if the instance is valid for another cluster. By valid, it means that in order for group application to work, it needs some specific configurations, which not all of them are by default right now. So for that, we have a nice command that is the shaking since configuration, which can remotely shake it for you. So let's check IC1. It's fine. It's valid for another cluster. So let's move on and create the cluster. So to create the cluster, we have the DBA create cluster command. Let me show you the online helper for it. So the create cluster command has several, OK, the syntax has several options. It provides a dictionary of options. It has plenty of options. So by default, the DBA cluster starts in single primary mode. So there's only one instance that can accept recent writes and all the others are read-only. But you can also set up multi-master. You can adopt an existing group application group that if you set up manually. And add YP white lists and so many other options. But I'll go for the defaults. So there's only one mandatory parameter that is the name of the cluster. I'll call it FOSDM. And let's create the cluster. So cluster is up and running. If you notice, I send a variable to it. So that's our cluster object. So if you can check now that you can execute, you have more commands. So the ones only related to the cluster object, you can now execute others. Add instance, check instance state, describe, disconnect, dissolve, so on. You can check the status of the cluster. So we output JSON for the cluster status. And you have several attributes, cluster name, the default replica set. We call replica sets to group replication groups. Each cluster must have at least one. This is the default one. You can see that by default, in the cluster is single primary mode, like I was saying. So the primary is the instance where I created my cluster. And you have the topology. There's this object with the address, the mode, the read write. And the status is online. And it's everything fine and up and running. So next step, add more instances to the cluster. Notice in the status text, cluster is not talented to any failures because we only have one. We need at least three instances to, for an identity cluster to be tolerant to one failure. And let's go on and next step, add another instance. So for that, I'm going to check how is the configuration of the second instance, 306. So on purpose, I changed the configuration of this instance just to show here. So non-valid. Okay. So the command is telling us the instance is not valid. Some issues were found. And for example, the bin log checks some, the current value is CRC32. And it needs to be non. The bin log format, it's statement. It has to be row, which is by default in my SQL 8, but I just changed it to show here. The transaction write set is off and it should be XX-64. And this is the same output just in JSON. The command is telling us fix the instance or we cannot use it in any cluster. But luckily we have a command to fix the instance configuration remotely. You don't even need to log into the instance. Which is the DBA configure instance command. So I'll call it on the instance. And the command is telling you the options that needs to change. The current valid and the required valid. It prompts you if you accept the changes. You say yes. And okay, the instance is now configured for the cluster. There's a warning saying that one of the... It's saying that you're required to restart the instance and that's because of the transaction write set extraction. That you need to restart the instance in order for the option to be changed. So let me switch to SQL mode. Establish a session to the instance. And now I'll run the restart command. Which is this beautiful, amazing command that we have in my SQL 8 that you can remotely restart an instance. It's really cool. Let's just run it. Let me reconnect the session. We're good. And just to prove that everything is fine now, I'll call the check instance command. And okay, it's good. So let's add it to the cluster. Okay, add in the second instance. I'll add the third that is running on IC3. And let's take a look at the status of our cluster now. Okay, so you can see that we have the three instances on the cluster. First one, the read write. The other two, read only. IC2 and IC3. And the status text is telling us that the cluster is online and can't tolerate up to one failure. So our cluster is up and running. And you see how easy it is and quick to set up a cluster using the admin API. You have several other commands. You can see the status of the cluster, but just the topology. You can use the describe command. It's a simplified version of the status. It only has the description of the topology and so on. So cluster up and running, all good. What's the next step? Set up the router. So let me... So the router has an option to automatically bootstrap itself to be configured for a specific NDB cluster. That's the bootstrap option. You have to specify the URI for one of the members of the cluster. I'll use IC1. And you can also specify an output directory where the cluster will be installed and it's a self-contained directory with the configuration for the router and the start and stop screens. Okay, so the router is configured. You can see here that read-write connections go to localhost 6446, read-only to localhost 6447. And you also have the ports for the X plugin. So let me start the router. There's a start script for it. It should be up and running. There it is. So I'm going to connect to the read-write ports of the router 6446. I'm going to get my cluster with a get-cluster command. You have to connect it to one of the instances of the cluster. We're connected to the router, so it automatically redirects the connection. Here's our cluster. Now let me switch to SQL and show you host. So you can see the router is redirecting to IC1, that is our primary in the NDB cluster. So if I connect to the... to the read-only ports of the router, I can... Let me show you. It will redirect the query to one of the read-only instances. In this case, IC3. So this is it. The router is running. NDB cluster is set up and running, and it's quite simple and straightforward to set up. This is just a basic setup. So we have time. Yeah. Automatic failover. Let me show you very quickly. So for example... I'm going to kill one of the instances, for example, the primary. That is IC1. So let me SSH to it. I'll just stop the MySQL service to show you what happens when the primary goes away. So let's reconnect. Let me connect to the read-write part of the router. Get again the cluster object. Check the status. Let's take a look. So primary, IC2. A new primary was elected. IC1, that was the initial primary, is gone. I stopped the service. A new one was elected. It's now IC2. And you can see here IC1 is missing. Missing it means that it's registered in our metadata, but this is not up and running. And the other two, IC2, became the new primary. Another read-write mode is set. And the other is read-only, as before. So also the status text changes and it's telling us now that the cluster is not tolerant to any failures, and one member is not active. So quick simulation. Let me bring the service back up and running. And let's check the status now. That was quick. So the instance that was gone is now online and is back to the cluster. It automatically rejoins the cluster and we have automatic failover. And that's it. So quick summary. In any cluster is the built-in HA solution for MySQL. It's a full stack high availability out of the box. It's very easy to use. Usability was always and will always continue to be a top concern. And for that we have the MySQL shell with the admin API that it can, like you saw, be used to configure an administrative cluster very easily. Brings together developers and DBAs. It's only one tool for both. And it's powerful, flexible, and secure. If you... Those will be online, so if you have the links, if you want to go from here, I really recommend to check the user guide because it gives you a global overview of the administrative cluster and it points you to the right directions for download. So it explains how to do a basic setup, then more advanced setups, troubleshooting, and so on. The shell user guide, the API reference manuals, and as always, our blogs keep being updated with the news and fun stuff. Okay, so questions. Yes, this one? Yes. Yeah. Yeah. We will... Up. We will... Eventually. Yeah, but it's in the plan. Yeah, definitely. Yes. Empty node. What do you mean? Sure, yeah. And you have the distributor recovery, so you have to go into the recovering state that is fetching the missing data and eventually gets in the online state and it's up and running. Yeah, but only if you have all the binary locks on the machine. If not, you have to start by a backup. Yeah. Yes, of course. Yeah. We use set persist. That is a new feature of my SQL 8 that allows you to... I don't know if you know about set persist, no. So with set persist, we run the command remotely and the configuration is persisted in the server. So behind the scenes, the server creates... Don't go too far. The next talk will discuss about set persist. Stay there. Peter will tell you what it is. There is one great last question. Flapping. What do you mean by flapping? We don't deal with that. That's group replication. After all, if it has too many issues, we say this is a network problem for this one. It gets rejected for the group. Thank you.