 What solution of when you face with a big table, big records? Okay, for instance, at this moment with the proliferation of web, the application is growing huge, it's growing as a very fast pace. When you open, when you put your application on the net, at first maybe it's only 20 concurrent users or probably 10 GB data. But then after that, what is happening is that it grows a lot. Maybe in the next week or so, it already grows your application or your database, already grows 50 or 100 GB. That means you're actually hosting a lot of millions of records in tables. So what is the solution for that? You know why? Because whenever any connection, whenever a lot of connection accessing one table, a big table, it's going to be slow. It's a recipe for a slow performance. Why? Because locking happening everywhere in the table and the index size of the table is huge. This is the solution for that. I'm going to share with you. Racing the bar, okay. So like I mentioned in the web, our application grows a lot at a huge pace. Now, coming back a little bit, just a backtrack. I believe you guys know that in 2010, MySQL comes under our code. So not every one of you. I believe at that time, many of you feel, hey, MySQL will be gone maybe in a few months. Raise your hand if you think about that. Okay, thank you for your honesty. Anyway, what happened is after five years, we are still here. So there's a reason why we are still here. And the reason is because many big companies already adopting open source technologies and we cannot avoid it. Companies cannot fight the wave, the big wave of open source coming. So we have to adopt, we have to embrace. So MySQL has been strong in the web application. We are strong in the open source application, right? And it's only right that Oracle embraces us as well. So this is just a brief picture of our artwork. We are releasing products regularly every year, okay? Now in application, because we are facing with such a huge growth, the application needs to be responsive and agile, in which in this case, we need some sort of high availability, right? Meaning that if you have one single server, if you have one single database server, what happens if that server, what happens is that server down. You can't think one thousand and one reason why that server can come down. Can you think of one faulty in the hardware, you know, electricity drop, all these many, many reasons for one single server to go down. So we need high availability. If we have redundancy, one, you know, master ends at least a clone of one server. So we have two servers. So one server down, the other still serving the data. The entire data is still running, your application is still running, right? Next is scale data in user groups. When coming to a huge data growth, your data is growing by leaps and bounds. Now, what you want to do is you want to scale accordingly. You don't want to buy one big server, 32 sockets, one terabyte of RAM, to be able to handle your application for the next seven years, right? It's a huge initial investment for you, isn't it? You want to start with small. You want to start with a low-commodity server, right? Low-commodity server like two sockets, maybe 30 GB RAM, and then grow and grow and grow according to when your application grows accordingly. So your database, your investment grows according to your application grows, right? OLTP and analytics, you need to have some sort of OLTP and analytics functions as well, right? Like a big data, it's the trend that big data is coming into the technology, right? So this is for you to help you to be able to make informed decision in a faster manner. Elastic as well for server-provisioning because we are adopting the cloud. Okay, sorry about that. Elastic means that we are adopting the cloud. It actually can provision the server from the cloud easily, right? Okay, so this is the technical definition of MySQL Fabric. It's extensible and easy to use framework for managing a farm of MySQL server. So it's basically a distributed database. Distributed database, right? And this is what it gives you. Okay, first of all, are you all aware? Oh, no, sorry, not aware. Do you know the term sharding, right? Sharding is used in a lot of places. Okay, sharding means is that say if you have 100 million records in one table, you split it, for example, into 10 servers. So each server have 10 million records, right? What happens is that after that, in each of the server, because the workload is also split, the data size is also split, right? So it's easier for you to manage the performance of your database because everything is split, including your workload, right? So that is sharding. Then the second function of MySQL Fabric is edge high availability. Now, all of you have already known that one of the major functions of feature of MySQL also available in community is the replication, okay? You guys are aware of replication is, right? Master and slave, the slave is the clone, right? So the slave will pull all the data from all the data changes from the master. And this is the high availability that we use in Fabric. How we contribute, what is the value of Fabric in the replication is that normally replication is done when replication, when a master is down, the slave should take over as the new master, right? Now, normally with replication, that is done manually. There is no other way you have to do it manually. You have to do a change master command and all this stuff. But with Fabric, it is done automatically. It means that the Fabric process will actually detect whether the master is down or up or down. When it's down, it immediately promotes the slave into become the new master. And then the rest of the slave will connect to the new master and pull the data from the new master. So that is the major two functions. The additional function is that, okay, as you know, one of the reason why people use replication is that to split workload, right? The character of web application mostly is 90% of them, 90% of the workload for web application is read, right? Take, for example, Google's, Wikipedia, Amazon. Most of the time, when you go to Amazon, you are click, click, click, looking, looking, searching, searching, searching. And then at the end of the day, you might not buy anything, isn't it? Right? And now, by nature, the workload of web is 90% read. So what happens is that, normally, people split, people have the master and then a couple of slaves to actually handle the read. So the master is free from any read operations. The master only handle the write operations, update, insert, delete. That's it. So the slave move to, the slave that can handle, handle the read. Now with, without fabric, without fabric, you have to direct, direct manually your read and write from the application into the slave or the master, without fabric. But with fabric, you can act, it will be directed automatically to the slave. You can set it that way. Right? Okay. At this moment, at this moment, okay. Fabric, fabric is work together with the connector, master connectors. Right? The connector has to be fabric aware. Has to be fabric aware means that you have to download the latest version. Right? At this moment, the one that is supported is phytonjavaphp.net.net and C is still a lab. Sorry. Yup. So, take note of this phytonjavaphp.net. Yes. Anyone need the USB? Want to copy the file? The zip archive. How does it archive? Because it's like trying to use it. No. It's okay. One wants to use it. The gzip, I think. The gzip. It's just BOM. Come again? It's a BOM file. Is there a problem opening that? Anyone have problem opening the zip except the gentleman? Okay. Let me continue first. Right. So, now the data. Okay. When we talk about data, we put data in the storage. You can be it sand or storage area network, shared storage, right? Or local disk. Anyway, you need storage to put the data in. Right? Now, normally on an advanced layer, like if you have a high level of storage, right? The storage itself is able to do, you know, redundancy. It's able to do high availability. The storage itself. So, they do, they sync each other. If you have a high level of storage, that is, that costs you a lot. I don't even want to share the price. Okay. Right. Now, normally with MySQL, for us, we use high availability on the, like I mentioned to you, we are using replication. That means on the application layer, on the database layer. Right? So, we don't use the storage to actually sync each other. We just use the replication. Now, but when we talk about HA, it's not just enough to only secure the HA for the storage. You have to secure the application server as well. Right? Your application server needs to be more than one. Because if that's application server down, you are gone. Right? So, you need to have at least two application servers and then probably have, probably put a load balancer on top of the application server. Any load balancer, be it software or hardware will be. Now, over here, we need access to data is where your database software is. The database, you need the database software to access the data in the storage. Right? Okay. There is, okay, but now what we add here is the routing in between each other, between the data. Okay? The data, base of that is the MySQL server. Right? On top of that, from application server to the database, we need, normally people connect directly from app to the database. Right? So, we need a routing function, but if you connect directly, what happens if the network is down? Then you are gone. Your application is gone. So, you need a redundancy as well. A routing from the app to the database server. Right? So, this database, this routing, we can actually strengthen it to cross each other as well. So, it increases the high availability. Right? So, just a little bit of replication type that we have in MySQL is that, in MySQL, the default is asynchronous, asynchronous replication. What it means by asynchronous replication is that your master, the master server, will not wait for the slave to be updated. So, it's the responsibility of the slave to actually pull the data from the master and update itself. Right? That is the default. But starting from, okay, starting from MySQL 5.5, we have what we call semi-synchronous. Now, semi-synchronous is when you receive a transaction, your MySQL master receives a transaction. It will send the change into the slave file system or relay log first. Right? It will send the changes to the file system of the slave server. Okay? And then it returns back, say, okay, acknowledge, and then the transaction is done. Right? So, this is in a way making sure that the changes whatever happened is already in the file system on the slave. But take note is that these changes has not been, it's not been replayed in the database yet. It's only on the file system, on the log, on the relay log. So, what the storage database needs to do is that it needs to pull the data from the relay log and replay it in the database itself. So, that's why we call it semi-synchronous because it is not fully synchronous. Right? By the way, but if you want a synchronous replication solution, that is another, that's a different solution. It is not replication. It is what we call MySQL cluster, the one in the middle here. MySQL cluster. That is another product of MySQL. It is also available in community edition as well if you like to give it a try. Okay? That one is the only solution that can do active, active within MySQL. Active, active solution means that you have two servers, active, active. One down, the other ones still active. Your application only need to redirect to the active one. Right. So, now, this is the framework of Fabric. So, the Fabric controller is the one that keeps the data. Right? It actually, the Fabric actually keeps the data in one small database, MySQL database as well. So, it uses MySQL database to keep, to store all the data of the routing. Right? This is in the process as well. Now, as I mentioned just now, it is using the connector. PHP, phyton, java.net connector. What happened is that all the information of all the, the MySQL Fabric structure architecture that you already set up, it will be routed and it will be updated into the connector cache, your MySQL connector cache. Right? So, when it is updated, every time there is a change in your architecture in the Fabric, say if there is the note down, right? And then it will be updated in, the status will be updated into the Fabric process, into the Fabric store. And then it, Fabric store will actually update your connector cache. That's the, and this is the, actually the entire architecture of your Fabric, the HA group. Right? Okay. So, you notice that all the, like, that in this case, we have a connector cache, right? Now, at this point, any questions? First. Okay. There is one thing that, that you might, you might want to ask. What happened if the Fabric store down? Because the Fabric store is the, is the one, is the, is the database that is the store where, is the, sorry, is the place where they put all the routing information, which is master, which is slave, and how many, how many servers do you have and all this stuff, which one is read-write, which one is read-only, all this stuff. What happened is it's down. Don't worry. Don't worry. Your application can still access the data. Why? Because all the information already in the connector cache. As, as long as, as the information is there, you don't have any problem. It will still be running. And then, when you bring up the, the Fabric store again, it will just join again and you don't have any problem at all. Right? So, this is the, the effect of the failover. So, we are using replication here. Primary is basically, basically the master, the one that receives all the writes. Right? This is the term that we will be, we will be using in the hands-on workshop as well. You will, you will notice that. Secondary is the, is the replication slave. It will, it only accepts read-only transactions. Right? And then, failover, when, when we have primary, when the primary is down, or the master is down, the slave will automatically tick, will, will become the new slave. Sorry, the new master. And it will all be managed by the Fabric process. Right? Over here we have a high availability group concept. In, in this, what it means is that in high availability group, you have, you have, you have one master. Only one master. Right? But you have, you can have a number of slaves. Some slave, you want to start it to become a slave, only slave. Slave that is ready for you to be, to, to take, to take over when the master is down. On the other hand, you might have a number of other slaves that you only assign for read-only transactions. So, and this is part of the one, one group HA. So, you, you, you make it different. You can put it that way. You can set it that way. Later, at this point, maybe you guys, a lot of, a lot of question mark in your, in your, in your mind. But later when you do a hands-on workshop, you will see. Okay. When we define a group first, we define a group first, and then we add server to the group. This is the HA group that we talked about just now. And then after, and then we promote one server to be the primary first. When we set up for the first time, right? And then we activate after, after everything is done, after we put all the, all the master, sorry, all the server that we want to be part of that HA group, then we activate it. We activate the anti-HA group. So now the fabric will monitor the entire HA group. Okay. So, right, so scaling, in this case, we actually scale horizontally, horizontally. So we don't need to, we don't need to invest a big machine in the beginning, but we just start small. Guys, you, you are, you are doing startup, right? Most of, some of you are doing startup, right? Any one of you doing startup here? Right? Okay. You don't want to invest, you don't want to invest a million dollar machine, isn't it? You, you run, right? Doesn't make sense. Might as well just buy a commodity server, HP, Dell. It's only cost probably 5, 10,000, right? That's it. So, and then you, when your application database grows, you just add more server, add more server, add more server. Done. So you, when you get the income, you buy more servers. Easy as that, right? And as you can see, right now my scale, the latest version is 5.6. The next one is 5.7. And at this moment is still alpha release. Soon, maybe next month, it will become beta release or release candidate, right? We are actually aiming to launch 5.7 to be ready for production use by October. Okay. As you can see there, that is the difference in performance between the different versions. You can see that how fast my scale 5.7 as compared to the current version, 5.6. So, when it comes out, you will be happy to see a performance increase a lot. So, just a little summary of the benefit of Sharding. Okay. The benefit of Sharding is that we can do a right scalability, handle more rights because at first you have one server. You add more server, more server, more server, so you have more, right? Then that means you are actually handling, you can handle more rights. Now, large dataset, database too large. So, since database too large, definitely it's a recipe for slow performance. What you want is you want to put more servers to manage the performance better. And of course, the improved performance. Okay. Right? I don't think I need to read it. Sharding architecture. This is the architecture of Sharding. So, over here you can see more clearly of what, how the architecture is. Okay. At first you have connectors. This is the application, this is the connectors that you put in the application server. Right? Right? So, as I mentioned earlier, the fabric store or the fabric node here will actually be talking regularly with the connector in the application. It will update the entries for the information in the connector cache to make sure that you get the latest routing information. Right? And then on this side, you have all the shards and a global group. Okay. So, right. So, this is all the shards. Say, this is all the big tables that you have. Say, you have 100 million records. You split it across different shards, different servers. Right? Right? So, okay, the global group is the group where you want to put table that you don't want to shard. Right? You definitely, in your schema, definitely there is some tables that is small. Say, it's like reference table. Maybe, in terms of the employee, you know the employee department table, EMPDPT table? Maybe department table is reference. Right? Small table. Where you have it, you have a transaction table that 100 million records. But there is some small table that only have 20, 30 records. Do you want to shard it? Do you want to split it across a lot of shards? Doesn't make sense. Right? You want to just keep it that way in one server. So, you put that table in the global group. All this table is accessible from all the, for all transactions. Right? Okay. This is the HA group. Like I mentioned, this is the global group. Primary secondary. We have primary secondary as well. HA group, we have primary secondary, which is master and slave. Master and slave. And all this is the extra replicas that I mentioned. If you want to have more ability to take up more read operations. Okay. I think I spent too much time talking sharding setup. Okay. This one, you will be doing it. Okay. You will be able to do more operation in terms of maintenance. You can actually move the shards from one server to the other. From one data, the data from one shard to the other shard. Right? You can actually also split the shard because your data is going to grow. Right? In one shard, maybe that one shard is already too big and then you want to grow. You want to split it into two servers as well, another server. So, you have to split the shard. Okay. Everyone already has your file. Anyone still need the file? The VM? Okay. Cool. You're cool? Right. Anyone still need the USB? No? Yes. Everyone cool? Okay. Right. So, first of all, open the zip file. You can open the zip file, right? It's not corrupted, right? Cool. Okay. So, in this case, what you need to do is if you run Windows, just run the VirtualBox Windows. If you run Linux, then you can run any Linux version that you want. Okay. Just install the VirtualBox first and then we need to do an import. When it is done, when you click on the VirtualBox, it will appear like this. Without this. All these are empty. When the first time you run VirtualBox. Okay. When you see this already, you just import. Sorry. Yeah. Import. Just import. Okay. And then choose the OVA file that you just downloaded and that you just copied. Next. And then don't forget to re-initialize, make address, and then just click import. That's it. It will be importing and it will probably take a few minutes depending on your laptop. And probably most of you are already using SSD. It should be fine. Anyone have any problem installing? No? There are a couple of things that not in the slide actually. I just found out in the VM that we need to do. There will be a number of replication troubleshooting steps that we need to do. Just a little bit. The good thing is that you will get to know how to deal with the replication issue like this. When it's done, when the import is done, please start it. Just click on it and then click start. Stay running. Okay. Fine. No problem. Stay running? Right. Right. Oh. Yeah. So I just downloaded this. Okay. Sorry about that. Right. I should put that in the next slide. Yeah. Because actually most of you guys are using Macbook, right? Macbook. Okay. Right. Yeah. Next slide. Yeah. Next slide. Yeah. Next slide. Yeah. Next slide. Yeah. Next slide. Yeah. Next slide. Yeah. Next slide. Yeah. Yeah. Yeah. Yeah. Yeah. So in demo settings, change it from post-molation to the next slide. Yeah. Yeah. Okay. Yeah. Yeah. Yeah. Okay. That's done. Okay. Everyone that's already done with the import can just access it to music route and one and the password. One, two, three, four, five, six. Sorry. Okay, after you're done with this page, this slide, stop first. We are going to need to do some replication fixing. There is another, guys, in the application, in the apps folder just now included in the zip file, there is a program called Putty. Use this to access the VM using the IP provided. Use this because if you are typing on the VM window itself, you cannot copy and paste. You need to type everything. If you're using Putty, you can copy and paste the command. You don't want to waste time typing. For those of you that are using Linux, you don't have issues. You just have to SSH to the VM. Now, if you have done this slide, you definitely will have seven MySQL instances running in there. Seven MySQL instances in one VM. It will take some time when you start it for the first time, when you start the script. Those are the instances that we are going to use for the fabric. One of them, which is 1.3.3.06.4, is going to be the fabric store. It is actually in your zip file, in the document folder. MySQL fabric managed setup. Managed setup is basically to set up the environment and to set up the admin password for your XMLRPC connection. And then the start, of course, to start the fabric process. Come again? You want? Easy. Now, what we need to do here is show slave status. You will see the status of the slave, the slave process. These two should be running. If not running, then your application is broken. In order to go there, you have to do this first. Just connect. Use MySQL client. Connect to 3.3.07. One of them is, this is the global group slave. The thing is that we might not have enough time, but we just do it as fast as possible. So, when you connect to this, both of them are yes? Or one no? Yes, sure. Sorry, guys. Hopefully, I don't really think that we have enough time to adminize already some other session updates. How many people are able to do this before you decide to do this? Because we have tried to use a new client, something like this. We really like to communicate with people in another session. I mean, there are more ways of doing this. And we can also continue with our next workshop. Okay, we've only just one more thing. Yes, sure. Okay, the rest of them, when you do it, when you just follow the instructions, it's okay. But this step, you need to do it first for this VM. Because I used this VM before, and then I found that there are some things that are not done right. Okay, what you need to do is, over here, you should see some error if your slave is not up. I would say do this. Okay, check this first. The retrieved GTID and the executed GTID, are they both the same? Right now, are they both the same? If they are the same, then that means you already executed the entire transaction for this. If not the same, then you might get an error. You know what? It's not enough time. It's very hard for me to go ahead and use this. Anyway, it's possible. Can you post full or part of your slides? Then people, if they wanted to follow themselves, so they need you to explain everything. Is it useful for people if you distribute the slides to... I have inside the... Okay, right. Maybe explain if they can do it on their own, or give you contact information if they want to ask you questions. Sure, sure. If you want to give a talk on this, we can also organize this. Guys, okay. So for now, since we don't have enough time, so I would say I will just give my contact. So if you have any problem running the instructions that I give you in the slides, just let me know. I'll get you to the steps. There are some things that need to be done on the replication. We need to... Basically, we need to insert a blank transaction first in all the slides. Basically, that's the idea. But it's just that we don't have enough time at this point. Right? So sorry, guys. Sorry, sorry, sorry. Right, so sorry, guys. Thank you very much to join this session. I hope you guys enjoy the zip file that I gave you. Let me know if you have anything and any information you have at home.