 Okay, it's time. Good afternoon ladies and gentlemen. And today I will tell you how I use MySQL test framework for troubleshooting. MySQL test framework is kind of unique test framework which is run every time when MySQL server is built. And this is not something like we usually would use as support engineer who works with customers in customer's cases. But I found it can be useful because it can solve a lot of time and can also help me to make run my customers experiments very effectively. So I am support engineer as I told you but what does it mean? Well, it means I can sometimes explain my customer pretty basic things. Sometimes I guide them while on call how to resolve their issue. But most time I just test. I don't know how it's seen on the backside but this is six different terminals. Shows doing exactly same select but running different versions from a few versions of Percona server, a few versions of MySQL's Oracle server and also MariaDB. Why is so? Because when customer has issue like for example, they SQL works wrongly. What I need to check? I need to check first is it true all the options. So like for example, if it's back, it's fixed or if it's like something which is MySQL was famous for being very forgiving, now it's not so forgiving. Is it true with all branches? For example, it's possible that MariaDB or MySQL doesn't have this issue. So I need to test this and I need to tell the customer. So practically I need to test with customer server version. It may be something pretty close like 5.6.10. Then I need to test same major branch but latest version. In this case of Percona server, this is 5.6.38. Then I need to check if it's fixed in newest branch, newest version. Again, in case of Percona server, it's 5.6.20. Then I check with upstream. This is 5.7.21. And then I make consignment checking with other forks such as MariaDB, especially for features which MariaDB implemented differently than MySQL. So it's exactly same scenario but all these kind of versions. So my manually test is, sorry, I won't do it manually. Which options do I have? Actually, when I started using MySQL test framework, MySQL sandbox just started as project. And in other solution, it did not exist yet. But still many of my colleagues use MySQL sandbox which is perfectly creates for complicated installation within command and includes scripts to start, stop and restart. And I use it like for such big test, such benchmarks and things like this. But this is doesn't, sandbox doesn't enter code for you. So you have to type manually. So like you can pretty easily start nine versions of different servers, but you still have to type. Docker, you can script everything, but with Docker, like you have the issue, then you need to test on something which is unusual, like build, which you just built yourself with custom patch created for this customer and that developers ask me, please test this patch in your environment. So this I cannot do. And of course I can script everything, but it will take a lot of time. So I started looking for test framework, for unique test, so that for unique test, we just script test. So I just started thinking about how can I do it for my customers. So actually getting started with MTR, my school test framework can be called MTR. It also has started with script called MTR. So I will use these words interchangeably. Just to get started, it's very easy. You just type any SQL you want. Just it's a understand SQL, so you can type everything which my school understands. You must to save it's V file, which name we have any name here. Just put here dot and test. That's mean it would be test. You will also include prerequisites check, like I do here. I'm ensuring in the DB is installed. This is very important for versions prior 5.7. I can have option for creating option, you should have, it's your test name. It's exactly like your test, dash, and then master dot opt. This way MTR understand this is option file. And now you'll put our options exactly same like you do on command line, like dash dash and any options, you can put any number of options here. Usually when we test, it's not just like single connection. It's often useful to check what happens if we have more than one connection. MTR has this possibility. You can, there is command. It has very small language. Easy to understand, easy to learn. It's like you can learn it in couple of hours. So you can just use command connect, put name of connection, and then connection parameters. This are for default. Here you, this way you connect to your new connection. That's, you change default connection. So here we are default connection. And here we are our new connection doing something. And then we do again default connection. And we can change for example, like here is transaction is not closed. We can check what we will see here in different transaction isolation level, for example. Many more. Sometimes by default, my school test framework when it encounters error, it's just stops. But sometimes we don't need it to stop. It's okay. This is expected behavior of this particular statement should return error. And you can just tell it don't stop and put option error number. Here you can tell NTR to skip all errors. And also you can, you can just have warning, just ignore all errors or warnings. When we are speaking about concurrent execution, concurrent execution, which is important, it's usually not like when you are, usually don't test transaction isolation level. But usually you're doing some job which cannot be stopped and you are trying to run parallel query, for example, to see if it's locked and if it's not locked or something, that's also something. For this, there is possibility to do this kind of test. You need to use send command. And then write command you want to use here. Then you can run in other command. You should know that it tells test framework to don't wait when this command finishes. But then when you need result, you just connect to the same default connection and just call rip. This mean get result set. For my case, case it should fail with error, but it may not fail with error, but you just will get results out and you will continue. You may have this send command only once per connection because you cannot execute two queries in parallel in same connection. But you may have multiple connection and multiple connection can run send commands. And this is absolutely okay. External commands. It's also test framework, it's capable to call external commands. It has predefined it, I guess it's for standard MySQL routine, just like MySQL dump or MySQL command line client. They work this way, like for example, if you have system wide MySQL 5.7 and you are testing antique MySQL 5.5, this command will point to MySQL dump 5.5, not one you install system there. This is a yes for directory, but you can call here any command, just put command name, like any Linux command, whatever you want. Flow control, of course, it's useful to have loops or condition processing, check variables. MySQL test framework has commas for flow control. It allows you to do well. This can be interesting, it's evaluation, it's calls to evaluate SQL statement using variables, using your custom variables. For example, in this example, I put generate 100 random rows for the table and I can do it again in my, yes, yes. Okay, so it is a MySQL test framework has a possibility to create replication very easily. So what you need to do is just, but actually I don't remember if it's true in 5.7 or 8.0, but in older version, you have to name your test case with RPL prefix, underscore prefix. Then you have to put here a source, include master slave, it's predefined script, which creates a master slave setup for you. And now you are immediately on the master, so you can rank queries on the master. If you need to connect to a slave, you just run connection slave and do whichever you want on slave. Then you may go back to master and do something called master and so on and so forth. There is also what we usually carry about, that's master and slave are in sync. So there is common sync slave with master, which returns control to slave. So like after this, it's in source that slave just got all updates from master and we are on slave connection. And this is how it's work. Replication, you can put options for both master and slave. For master, you are doing it the same way as for standalone test case using this exception. And for slave, you may use exception slave, slave.opt. So this is again, it's pretty easy, you put everything you want, it's a standard option, I think. But what is amazing, you can have any kind of topology. For example, here it's circle. So we are doing here, it's one, two, we just tried. We are doing two, but what is more amazing we can do, we can have any number of servers. For example, to do this, I can do just put here, three, four. I can have this. In science, we have multi-master in version 5.7, we can do this. You can go crazy and I can make a picture of colors and I can do it. This is only line here, it's a topology, which I need to create this kind of pretty crazy replication setup. And then I have to include this server file, which will create this setup automatically for me. Then there is a little difference between my previous example. So to connect to first masters to this server, you need to set special variable, RPL connection name, server number one here, and then include this file. So you will be connected to the server. If you need to connect to second server, you just need this. And same for server, whichever you want. You also have options for them. Options for such tests are a bit different syntax. Test framework expect dot.cf command, and they are usually the same syntax as mySchoolD multi-use it or mySchoolD save. So you first include standard replication options, so you don't need to redefine them. Then override whichever you want. So like for mySchoolD.1 you do these options, for mySchoolD.2 these, for mySchoolD3, et cetera, et cetera. And then for servers, which is number greater than two, you need to define port and socket, so test framework knows which names to use. Okay, we just created such test case, but how to run them? It's pretty easy. You have to put them into your, usually it's easiest way, it's like linux.tar.jz packages, go to mySchoolTestSuite directory, mySchoolTest, and then it has directories t and which holds test. You can also create your own suite, in case if you don't want to mix your test with test which comes with mySchool installation. And just like you run this test and if you have no other options, it just, I would put result of the test. If you want to save your test, test results, you can use option record. You see one output, it would be recorded in the directory R, sub-directory R of mySchoolTest, with name yourTest.record. What I actually do, so like when I started this talk, I talk it about I have to run on multiple servers. For this, it's very easy, so you see these kinds of things, it's very easily scriptable. So just very, so I created couple of scripts which is in my GitHub project, mySchoolScripts. So I have like scripts called doTest.sh, who just runs a test on a multiple version. So I have a predefined directory where I held all my tests. And then I just run doTest to define which build I want to test to run. And then test will be copied into t-directory and just run and will output result. If I need to run these on multiple versions, I just put B all these versions here. So this is my pre-compiled Perconocer. It works both with source directory, of course after you build binaries, and both with your installer directory and format of tar.jz packages, these scripts. It's also you can specify source upper directory for all tests and it has a few other options. After I finish tests, I archive them and put them into archive directory. And also have script which copies them to remote box. So I just like run do it with on command. Okay, so to summarize, mySchoolTest.frame.bog understand my SQL SQL syntax. It supports customs command, any complicated setups. It runs everything automatically for you and checks results. So this is a great tool which helps to when you need to run same test multiple times. Here are a few links. First is documentation for test suite. But I recommend you to just open existing test and read them. You will find something which is not documented like variables or how do you stem flow control. This is GitHub address for mySchoolScripts repository. This repository contains all scripts which I use it on this talk. This is my slide share and Twitter account where I put this talk. And also I want to announce, there is still not yet announced official on Percona website, but at April 12th I will be doing bigger webinar about one hour webinar on mySchoolTest.frame.bog. If you are interested, join. Thank you. It supports Linux command, so you can do it. It also has like for killing the master gracefully. It also has restart include script which calls restart server. It will restart it for you. What you mean? Which mySchool start box, which is how I use it, you will just like to start. You can start sandbox, you can have everything crying, but if you want to run SQL, you have to type yourself. Yes, I can do it, but for single connection, but for, I don't have these facilities for multiple connections and things like this. I'm not saying mySchoolSuntbox is better. I'm saying mySchoolSuntbox is just like for different kinds of tests. It's great like I use it for like for example, which is why it's, this does not fit. By default, mySchoolTest.frame.org gets killed after 900 seconds. You can never write it, but that's still. Also, it comes with pretty, with many defaults, which are not standard defaults. For example, all performance schema instrumentation is on. So like if you are testing performance, you will have disable it manually. So it would be too boring like if you are going to run a long running benchmark. So sandbox is good like if you want to have like run tests for hours, forget about it, do it, leave it for days, for hours, like, and this is very nice. If you are going like maybe try to experiment with something, but if you are no, you have to run same SQL code like multiple times, you can check it with, so these are just different kind of test tools. Okay, so I think that's all. Thank you. If you are interested in MySQL Sandbox, Giuseppe is the owner, creator, whatever, and he also released Friday, DVD player, so MySQL Sandbox, written in Go. So if you're interested, go to Data Charmer and it will explain you everything about that. So at the same time, I will start with Woospits mouse. Okay, so I will do the demo. Can you make a talk while you do the demo? I guess. So you will do this. It's a motion, you'll mute it, you get the mute. This is the real music, so you just take it, and you're seeing it, and you do it. Well, like that is perfect. T-Borg, can you hear me? Can you see? It's okay.