 but we have worked on about three years. So we recently finished Myrox deployment in our biggest database called the UDV. So I'm talking about the deployment stories and what we are planning to next. So first I'm going to talk about MySQL at Facebook, then Myrox overviews, and how we deployed in production. Then in the last, I'm going to talk about futures. So at Facebook, we have a very big database called the UDV, which is a user's database. So in UDV, we start almost all Facebook activities. For example, when you like or you like or comments to somebody's post or making friends. So these Facebook activities are stored in MySQL database. So MySQL database is massively sharded. So apparently single MySQL instance is not enough to serve millions of Facebook people. So we have a huge record cache in front of MySQL, so which is called TAO, that cache tier mostly hits the lead request hitting TAO, but if cache misses, then it goes to MySQL database and also all right requests goes through to MySQL. So if MySQL is slow, then this affects the people's experience at Facebook. So UDV, so MySQL database has to be fast enough, so we take care to make it fast and reliable. And we don't MySQL run on cloud, so we don't have AWS or Google Clouds to back up for any other monitoring capabilities. So we do by ourselves like automated master failovers or recoveries or provisioning on backup from the store. So these common DBA activities are automated. And last point is, this is the main motivation of why we started MySQL project. So we use pure flash storage. So PCI Express flash drives. So apparently how the disk is slow, that's too slow for serving our users to running on UDV. So we replaced this pure flash years ago. So the pure flash is fast, but it's still pretty expensive, especially on the server grade. So server SSD is pretty expensive. And we are pretty much constrained by space, not by CPU and IOPS. So saving space was pretty important for us. So what is MyLogs? So MyLogs is MySQL database on top of RocksDB. So RocksDB is a Keyvalu store, which is a LSM log structure and merge database originally created by Google named LevelDB. So LevelDB was originally created for lightweight kind of embedded applications. So Facebook was pretty much impressed with the LevelDB and we wanted to use on the server side. So we forked from LevelDB and added many features to run on back ends like a multi-threaded background operations. So MyLogs is basically using RocksDB as a storage engine. So we are heavy users of in-level compression. So we use in-level. The purpose of the development of the MyLogs was using RocksDB storage engine to replace in-level. So we created RocksDB as a storage engine. Then all of the existing MySQL clients can use MySQL database without changing application code. So it's normally when migrating applications, then migrating client code is very time consuming. So we focused on switching back end engines without changing applications. So everything is open source. So this is published as an open source project. And it's distributed from Mammarillie and Percona. So MyLogs initial goal at Facebook. So in in-level, we use in-level compression in main database called UDB. So as I said before, so we are pretty much constrained by space. So I believe CPU usage was pretty low, like 20% and IO utilization was also pretty small. So SSC was very fast, so it doesn't use much IO capacities. But from space point of view, it's pretty much constrained by space. So used about 90%. So our objective of using MyLogs was saving space from a B3 to LSM. So LSM was very space optimized database. So our initial experiment shows that space could be saved to half compared to a compression ready. And still it used CPU a bit, but from our approach, the CPU and IO utilization was pretty comparable compared to in-level. Then after saving space, technically it could mean that we can put another MySQL instance in the same machine. So by running two instances in the same machine, in ideal, so we could save the number of machines by half, so which was quite a big deal for us. So that's why we started this project with spending many resources. So MyLogs features were, so since we were targeting to switch from in-level to MyLogs, we tried to create as many compatible features as in-level. For example, we decided to use cluster index format, which is same as in-level. And technically the LSM database like RocksDB is slower for these and faster for lights. So slower leads is apparently disadvantages. So RocksDB has several features to make lead less slow like a bloom filter. So we utilize these options. And we added support for transactions including consistencies between Bindog and RocksDB so that even though the MySQL instance is crushed, it can be recovered. Since this is very important feature for in-level and we heavily rely on that, so we wanted that same feature in MyLogs. So faster data loading is, I'm going to talk about later. And so RocksDB has many tuning options, so hundreds of tuning options. So we used to change the option by restarting MySQL, so which was very painful. So we made many options can be changed without restarting MySQL. So TTL is a time-to-leave feature which was comparable to HBase TTL feature. So this means by specifying TTL, then any data that are old as times can be removed without like a deletes, manual deletes or dropping functions. So this made several DB operations a bit easier. And we added support for online logical and the binary backups. Since we take logical backups for daily disaster recovery purposes, and we take physical backups for creating replicas. So we both heavily rely on these features. So we added for MyLogs as well. MyLogs was seen in the DB. So there are several advantages. MyLogs has advantages and disadvantages. So let me go through several points. So one biggest advantage of the MyLogs is the smaller space. So from our production environments, it's basically around half compared to a comparison DB. So most of our in-a-dibi deployments we are using compressed in-a-dibi. So for almost all deployments we have seen about half. So if you are not using regular, the compact non-compressed in-a-dibi format, you can save even more like 3x or 4x smaller space. And it also gives better cache iterate because space could be half. And lights are faster, this is LSM, Database is Characteristics, so which means replication is faster. So the replication tend to lag smaller compared to compressed in-a-dibi. And another advantage is the bytes written to storage is smaller compared to in-a-dibi. Like 10x or 20x smaller bytes written to flash is not uncommon. So since flash has a right endurance, so if you write too much to flash, then the flash burns out. So writing less is important. So if flash writes less, then it might be possible to use more affordable flash devices as a database. And MyLogs, this one advantage is it lacks several features. So most notable one is that MyLogs does not have a gap lock yet, so which means you cannot use statement-based replication. So we are using low-based binary logging to support MyLogs. And it doesn't have a foreign keys or full-text index or spatial indexes. So these are missing. And to make a better performance, it's recommended to use case-sensitive collisions. And these are slower, especially if all data fit in memory. So when you do benchmark with all data in memory, then you get slower results with MyLogs compared to in-a-dibi. And it's more dependent on file system and operating systems. So we use MyLogs and LocksDB without all direct. So since LocksDB is direct IO support, it's very limited. So we use buffered IO. So it's heavily depending on reactance kernels. So it's highly recommended to using a newer 4.6 reactance kernels, which fix many buffered IO problems. And there are many tuning options beyond buffer pool, such as bloom filter or confactions. So we are trying to give a best solution to very good default parameters so that users don't have to worry about that. These are improbable. Okay. How we migrated from in-a-dibi to MyLogs in your main database. So first step was creating... Okay. First step was creating... Is it on? It's off? First? Okay, fine. So first step was creating MyLogs instance without downtime. So this was pretty straightforward in MySQL world. So just pick one of the slaves, then stop the slave, then run MySQL dump. Second step was creating a second MyLogs instance without downtime. So this was pretty easy because the first MyLogs instance was ready. So just copying another MyLogs instance, we created a tool, an online backup tool called a MyLogs sort backup, which is also open source. And we used these two to copy from MyLogs, first MyLogs to second MyLogs. The third step was promoting this as a master. So, okay, okay. So this was as easy as just change master statement. So from a degree point of view, it's just one command. But from our deployment perspective, this was the hardest step because when MyLogs got fucked, then the data might be broken, corrupted, that's a huge problem for us. So we tested a lot. So for example, we created an audit framework, audit programs that captured all the graphics, then tested replaying shadow graphics to test MyLogs instance so that to verify that data was not corrupted. So after verifying several data consistency tests, then we could finally make MyLogs as a master. Then the final step was just copying MyLogs everywhere. So our current production status is, so we completed in a way to MyLogs migration in our biggest UDB data steers. And we could save 50% space in UDB, so compare to compressing it. So we could achieve original goals. So we started working on migrating other large database steers. So UDB is by far the largest database at Facebook, but we have several database steers, so we are trying to gain benefits. So developer and load maps. So I'm going to talk about what we are going to next and from a log-state development point of view. So we are trying to help MariaDB and Percon server to release with stable MyLogs. So Percon server did announce earlier this year that GA version, the MyLogs was released. So we are helping MariaDB to do the same thing in the near future. From performance point of view, matching read performance is most important. So from algorithm point of view, the LSM is slower than victory. So we are trying to fill the gaps. So my colleague Mark Karahan did lots of benchmarks and we are trying to make it better. I'm talking about supporting mixed engines and better application and bigger instance size. So mixed instance is basically goal is allowing people to run both in the labian if the data is in memory. And MyLogs is slower for these, for the very small tables, but it's much specific, efficient for big tables. So it's pretty reasonable that people want to use InnoDB for very small table and using MyLogs for the rest tables. So currently we do not support these configs, but we are trying to support in the near future. So targeting MySQL 8. So there are several challenges like making online or binary backups support both engines or making sure that benchmarks are not going bad. And the current plan is extending extra backup to integrate MyLogs hot backup. And we are also considering back porting several MariaDB features like GTRD, both auto engines to support crash recoveries. Better replication. So better replication is basically most of the benchmarks like Percona or Amazon is doing showing that QPS, the light QPS, improves significantly when the V-Log is disabled. So when enabling V-Log, which is almost all production environments, then suddenly light QPS drops. So it's because the 2PC of the V-Log and the engines. So we think this was mainly caused by 2PC. So we are researching if it is possible to use just one log, which means the V-Log. So just using V-Log and not using RocksDB Lighthead Log, which is leader log, and doing RocksDB's recovery from my in-review. So by having several format changes in the RocksDB side so that it can read from a V-Log and when my scale is crashed, recovering RocksDB by reading from a V-Logs. If this works, then this can significantly improve light throughput. So this is what we started researching. And parallel replication applies. It's basically my scale eight has a lot of progress. So we are looking forward to that. And RocksDB has several features to speed up even faster. For example, MyLogs supports not using internal transaction APIs, so it doesn't lock records. So this doesn't make sense on masters, but on the slaves, it should be fine. So we are trying to use those options on the slave side. The last part is supporting a bigger instance size, which means, so in most cases, even for us, we shard my scale databases. We have as the presented and the BTS talk and several sharding talks. So there are several, many my scale master instances and it's sharded, then using a sharding framework like a spider or BTS. So this is good if you have a specific purposes, but it makes it harder to use several features like secondary indexes or cross shard transaction support. So what we are looking for is how about, now we have more than 10 terabyte flush strategies. So why not just supporting a very big like 10 terabyte single my scale instance, then putting everything there? So you can use secondary indexes and you don't have to worry about distributed transactions. So these bigger instances may help several general purpose that small to medium sized applications. But these have several challenges like a parallel transaction, transactional my scale dump since the my scale dump from a 60 terabyte instance may take days, which is pretty painful for the BTS. So this has to be lying parallel and parallel queries or parallel copies, parallel DDLs or reasonable DDL. So if DDL is failed after one week, then it's restarting from the beginning, then that's really painful. And we have several challenges like a beta join algorithm, faster replication or handling many more connections and resource controls. From other point of view, shared strategies might be needed to support bigger storage. So there are many challenges, but this is probably the next interesting thing that we are looking for. So this is summary of my talk. So we finished deploying my docs in our production user database and we started deploying, you can start deploying my docs on slave side first with consistency check after verifying it works, then let's promote to masters. So we have added many status counters or monitoring metrics so that people can easily use and the more interesting features will come this year. So that's all my talk. So thank you very much. Yes. So, the question is, once working with Percona MariaDB release GA, so what will Facebook do? So we are not doing software business. So what we like to do is continue to push our features to open source and upstream, which is a MySQL 5.6 GitHub branch. And we don't expect people to use the branch because it's painful for people to maintain that. So we like to continue to innovate and push features to upstream. Then we like to work with Percona MariaDB to take the useful features, relatively stable useful features to their distributions. So, does it answer your question?