 I am senior web engineer from TenApp and today I would like to talk about automated testing of WordPress projects. Before I start, I would like to say that it's quite a big topic for one conference, so I will skip some details to not say too much and give you too much information. So if you find something you want to know more, come after my talk and I will explain you better. So I think everybody here knows that any change made to large and complex code base requires testing to ensure that we not introduce any new bugs in our system. So in all testing manually, each time we change something, it requires significant amount of time and energy. And not always can guarantee that we find all bugs. So that's why we need to automate our testing so we can easily check everything each time we change something. Today we have three types of testing used in web development. The first type is acceptance testing. Second type is functional testing, also known as integration testing. And the third type is most known unit testing. If we talk a bit more about unit testing, this testing level is intended to test all the smallest units of our application, like functions and classes. And the purpose of this testing level is to figure out if all our units works as we expect. Talking about WordPress, we need to test on this level all functions and class methods which we have in our application. And especially we need to pay attention and try to execute this test in isolations and write tests for one behavior. Second type of testing is functional testing. On this level, we combine our smallest unit into modules and test how these modules work together. The purpose of this level is to figure out if our modules' plugins have compatibility issues between each other and if they work correctly or not. Once again, in the WordPress reality, we check if modules work together, if plugins work correctly, if we have any compatibility issue between themes and plugins, etc. The final level is acceptance testing. This testing level is intended to test the system as a whole. And we need to check if our system is compliant with business requirements. So it means that we need to check our site without understanding what plugins we use, how it's built, all what we need to check is how it works in the user's browser to make sure that the user can come to our site and do what he needs to do and be happy. Okay, what we have today and how we can create our tests. Today we have a lot of different frameworks for testing, but the most known is PHP Unit. It's a good framework and oriented for unit testing, but unfortunately, it's a bit hard and maybe almost impossible to write acceptance tests and functional tests just with PHP Unit. So a new framework has been developed over PHP Unit, which is called Codeception. It's PHP Unit-based framework. It extends it beyond just unit testing and gives us quite flexible tools to write functional and acceptance tests. As you can see, it has a lot of different modules which you can use in your tests. All of them gives you different abilities to check, for example, database, file system, REST API, XML RPC calls, etc. But unfortunately, it doesn't have WordPress module which will help you to write your tests for WordPress projects. That's why we decided to create a WordPress plugin which will integrate Codeception into WordPress. We called it WP's Codeception. It's a WordPress plugin which integrates Codeception into WordPress and allows you to use WP CLI commands to create and run tests. Excuse me. Okay. So as I said, it uses WP CLI commands and let's check the basic commands which we have. To create testing environments in our projects, we need to run WP Codeception Bootstrap command. It will create in your project global config and by default, it will create three suits, each suit for each testing level. So you will have unit suit, functional suit and acceptance suit in your folder. To create tests, you can use one of these four commands. I don't want to go into details and explain the difference between them. You can check it in the documentation. It's like the difference. So let's go to the next command. It's Codeception run. It will run all tests which you have in your projects. You can specify which suit you want to run and what test you want to run. You can also define different environments and run your tests in different environments. That's all about theory. Let's take a look at the demo. It will explain you how you can use codeception. So here we have just simply basic WordPress installation with WooCommerce and MyStyle theme from WooCommerce. Let's assume we have a site which we built for our customer. And customer wants us to check each time when we develop a new feature that everything works. And users can go to his site and purchase products. Let me open a sample. So here is just really basic site with default products. So what are we going to do? We need to run terminal. Okay, I will make it a little bit bigger. Okay, let's assume we have a folder called MyPlugin. And let's assume we do everything here. So to create a new testing environment here, we just need to call WAP, codeception, bootstrap. It will create all required configuration files. And if you open it here, it will create a folder with tests. And here we have acceptance suit, functional suit and unit suit. So let's create one suit for, sorry, one test for acceptance suit. To do it, we need to run W codeception, generate, test, acceptance, MyTest. It creates a new file in acceptance folder which we can open and start writing tests here. Each public function which it doesn't start from underscore will be considered as a test and will be executed. So before we start writing the first test, let's create a scenario for our test. Let's try to automate checking if users can purchase products in our e-commerce shop. To do it, we need to go to the shop. Let's do something like first step, go to the shop. The second step, select premium quality product. Here we will change the quantity and let's grab a price to check total in the future. So third step is grab the price and change quantity, add to cart and go to cart. After going to the cart, we will check the total price on the fourth step and proceed to checkout. So on the checkout step, we need to fill in billing form, proceed to PayPal. On PayPal, we need to check if we can pay with PayPal account and login and pay. And finally, we need to check if we purchase a product. So it will look like this. We click login and click pay now. After that, we will be redirected to thank you page. We will click here link. And we will check if we see order received, which we assume that we purchase something. So as you can see, this is not quite huge scenario, but it will take a lot of time if we will do it each time manually. So before we start, we need to adjust our configuration for acceptance suite. As you can see here, we will use web driver module, which will allow us to send commands to our browser and manipulate it. But by default, we will use phantom.js browser, which is good, but I will change it on Firefox. So let's set windows size, maximize. So let's start. Actually, I forgot to say that each test methods receives acceptance tester objects, which has a lot of helper functions, which will help us to test our site. So if you take a look here, you can see that we have a lot of different methods. Like I am on a page, I am going to do something I can see, I can't see, I feel filled, I select option, I can see link, I can wait for eject response, etc. It has huge amount of different helpers. So how we will do it? Okay. From the beginning, we need to go to the home page. So I say I am on page, on the home page. And I want to select, not select, but click on shop link in the navigation menu. So I say I click and pass as first parameter, I pass selector. In our case, it will be just shop text. But as a second parameter, I will tell context of this selection. And context will be just our navigation menu. If we go here, we can see that navigation has ID navigation. So I just pass ID selector here. And conception will determine that we need to click shop in navigation. So after clicking on shop, we will see this page. And let's assume I want to click on premium quality product. So I want to say I see premium quality in the products list. So I say context is products list. After clicking on it, I want to see the title of the product. Okay, sorry. We need to click on it to go to the product. Actually, we will use the same here. So now we need to check if you see premium quality title. I see premium quality title in the product title header one. So now we need to grab our price. So we have it here. So we need to say, okay, price. I grab text from price amount. And now I need to convert it to float variable. So I need to remove dollar sign. And so to do it, I need to convert it to float wall value. And break replace everything what is not numeric dot. So now we have price. Now we need to change quantity. Let's use something like random from 1 to 10. To change quantity, we will use I fill methods. The first parameter is selector for this field. So we can say something like quantity input text, quantity input text and pass quantity. After that, we want to say I click on add to cart button. And I see something like has been added to your cart. In the context of WooCommerce message. And finally, we need to click on view cart button to go to the cart. Click cart button, uppercase. That's all. Now we need to check the total price. So we have our cart. And we want to check that total is equal to our quality and the price. Sorry, our quantity. So we need to say I see dollar sign number, format, quantity and price to digits in the order total row. And after that, if everything is okay, we need to proceed to checkout. So here we need to fill in all required fields. So when we call I fill field methods, we can pass as the first parameter field label. So I can say I fill field first name and say Eugene. I fill field second name as Manuilov. What else? I need to provide email address. And I need to provide phone number. Then I need to select country. Select option. Country, let me be Ukraine. Now I need to fill field and set town and city. Now state country and postcode zip. State country. Let me be something like this. And finally we need to click proceed to checkout. It should be something like click proceed to PayPal. Okay, here we can try to run our test and see how it goes. So if we open here, we can say, okay, codeception, run acceptance and show me steps. Sorry. Again, we started our codeception. It should launch Firefox now. Okay, so we made a mistake in selector because it's not second name, it's last name. Okay, if something wrong goes with your test, codeception will create a screenshot of that page. And you can see what happens when test fails. So we can see that I entered wrong selector. It's last name, actually. One more time. Yeah, proceed to PayPal button. Okay, let's try one more time. Okay, this time something wrong with postcode. Postcode zip. Okay, zip should be uppercase. And the final one. Okay, as you can see our test run successfully. But now we need to go to the PayPal and to pay for it. So the problem with PayPal is that it uses Ajax requests and to simplify our tests, I will wait a few seconds to make sure that all Ajax requests have been processed. So I just say I want to wait five seconds. And I need, after that, I expect to see this page. But I also need to make sure that I want to pay with my PayPal account. So I need to click on this link and wait a bit to make sure that PayPal switched to PayPal account. After that, I need to enter email and PayPal password. So I say I fill field email with my sandbox account. Fill field PayPal password is something like this. And I need to check option which says this is a private computer. After that, I need to wait a few seconds and click on pay now button. After that, we will have thank you page with click here link. And finally, we should be redirected to other received page. And we want to make sure we have this header on the page. Okay, let's run it. As you can see, each time we have a different amount of t-shirts which we want to buy and we have address. I forgot about address, sorry. Guys, if I remember correctly, it was something like billing address one. Let me correct this. I forgot to click on login button. I hope this is the last one. But you can see that if we spend one time, ten minutes to write a test, it will run each time you change something. It will take 15 seconds and you will be sure that everything works fine for you. Okay, actually the last one is probably something wrong with selector. But in general, you can see that we passed all checks from the beginning till the end and we checked that all the process works correctly. This is the only one scenario where we can use our conception. We have a lot of different cases when it could be used. And for example, if you have large project and you use, for example, staging server, you can run all tests on staging server each time a new version appears. And you can run it in CLI without any needs to install Firefox, etc. Also what we have is a module for browser stack. I'm not sure if everybody knows about it, but browser stack provides us ability to use automated testing using its service. So what we have, we can run our tests using browser stack on quite range of devices. For example, we can run all our tests on mobile devices. We can select, okay, I want to run it on iOS. I want to have iPhone 5S and I want to have there, for example, Chrome browser. And it will run. For example, you want to run your tests in Windows and XP and have Internet Explorer 6 or something like this. So let me show a quick example of it. So actually, let's do not close it. You just do something like this. We will create a separate test for it. We will go call it test browser stack. So to use browser stack, we need to change our configuration. I don't want to spend too much time on it. Let's just do a quick overview of what we have here. As you can see, I have defined another environment for staging server, which uses browser stack module and configures it to use Windows 7 version and Internet Explorer 10. And the URL will be Paris Worldcam.org. So what we will check. Let's see what else can we check here. So let's check if our start responsive and if menu for mobile phone will be different than for desktop site. So, for example, if we grab this button, I can say I am on home page. And I see this text in the primary navigation context. And I don't see this button. I don't see menu toggle. Sorry, not menu toggle. I don't see this text. So I don't see this text in the same context. So now I want to change the size of our windows, for example, to the mobile size. I say I resize windows to this dimension. And now I want to see mobile version and don't see desktop version. So I say I don't see it and see the second one. So it's really simple, just as an example, because I don't have too much time. And we will run it on browser stack. So to do it, we need to say conception, run acceptance, show me steps and use environment staging. Now we go to the browser stack and run our test. So we started browser stacks, started new instance. Launch windows 7. Actually, it works faster, just interconnections slow for some reason here. Okay, time out. Unfortunately, I can't show you it right now because internet connection is slow and we can't demonstrate it. But in general, we can just run any operating system and any browser we want and check in different browsers the same site and to make sure that everything works in different browsers if we need to support something old and crappy. That's probably all for me today. It's just an introduction into automated testing and I tried to show you what you can do and how it could be easily when you create small tests and it will save you a lot of time each day. Okay, back to my presentation. And thanks for listening. Thanks, Eugene. I have a question for you. How about dynamic content like Ajax? For example, instead of using PayPal, I use Stripe and when I click on the process to check out, I have the Stripe pop-up. Yeah, you can use different methods like, for example, if you take a look here. We can say I wait for JavaScript. So you can check if jQuery has active connection, active Ajax connection, then it fails or wait while we finish all connections like this. So it could be done. Actually, for JavaScript, for text. Yeah, we just can wait for JavaScript to be executed. Thank you. A question? Thank you for your presentation. It's very interesting. I have two questions. The first one is your project free? Yeah, it's free. We actually don't have a code exception project. We have WP code exception plugin. It's free. It's on GitHub and you can grab and use it. Okay. The second question, is your project working only with WordPress with any other web application? Code exception works with any PHP application. It has a lot of different modules for different modern PHP frameworks like Symphony, Laravel, et cetera. Native application. Yeah, even native PHP application will work fine with code exception. WP code exception is just a plugin for WordPress to help WordPress developers use code exception in other projects. Okay. Thank you very much. You're welcome. Thank you for the presentation. I'm more of a project manager, so I have a more global question. What kind of workload would you say this adds up to a project as compared to the usual development? How long do you think the test would be? Sorry, taking to create or write? It depends. It depends on what you want to test and how complex is it because if it's just simple front-end, it won't be too hard to write a test for it. If you want to test something in the admin side, it will be more harder and might take a little bit more time. But I would say maybe 10 minutes, maybe 15 minutes. But in future, it will save you a lot more. Sure. I think the other part is who do you think should be in charge of this? Is it better to have the developer who did this section write his own test and run them, or is it better to have a new pair of eyes and someone outside of that who does the test? Once again, it depends on the project, type of project. If enter resources which this project has. Because in general, I think it's more proper if developers write their own tests when they develop something. It's my opinion, I think. Thank you. Thank you. In your code, do you have some tests to test speeds or several latency or response in generating pages? No, I think test speed and performance is a bit different testing level. It's like system testing or something like this. And it is more related to DevOps guys, RAISers and PHP guys. So you can try to generate a page for two seconds or five seconds. It depends. It's not including your... Actually, maybe I misunderstand your question. Do you mean you want to test the performance or what? Just testing page generating. For example, in your projects you have your page generating in few seconds. For example, after upgraded WordPress in newer version, pages are generating in ten seconds. Do you... Yeah, you can extend your tests to check how quickly you go from one page to another page and compare probably to some value which you expect is fine for you. Thank you for the presentation. I want to know if there is any code-hint extension for other code editor. Sorry. I see that you use IDA. Yeah. For instance, for brackets, is there any extension for code-hint? For code completion? I think all modern IDE has code completion and can understand what you have based on your... For example, if you take a look here, I have WAP code-ception folder. It's a plugin which we use. And I installed it and activated it in my WordPress admin panel. And my IDE understands what files and classes and functions exist there and just use it for completion. So if you use something like PHPStorm or anything else that has code completion, it will also understand it and will help you when you write your tests. Thank you. Hi. Thanks for the presentation. It's very good. I have a question. I understand you have a script for testing a module and then you have script for testing the device, the interface. But how many scripts do you need to test one module? Because you've done just one script. So what is the average number of scripts that your developers have to design to have a full test of the script making sure that you can buy a t-shirt but you can say, well, I'm wrong and I go back? Yeah. Once again, it all depends on the project, on your resources, on your deadline, on how many developers you have and how deeply you want to test your projects. Probably something could be sacrificed to make the project faster. And probably, for example, for acceptance tests, you can just test the main features if you want to develop the project sooner. So it all depends on situation and projects and what plans we have and what resources we have. But in general, it will be enough if you test the main features which you need. Another question from Bishai. Hello, Philippe. I have a question on how can we improve the recording of a test scenario? Would it be possible, for instance, to use the Chrome developer tools and the network tab, you know, you can just check a checkbox and preserve log. And you can get... The result is that H-A-R, which is basically a recording of what you just do at the traffic level under Chrome. Would it be possible to get this log to import a script? Like in... Well, for use case, you can do this in Gatling, which is a stress tool. And I don't know if you know it. Yeah, it's a good question. And to be honest, I haven't faced such requirements and I'm not sure at the moment if it's possible to do or not. But thank you for your question. I'm sorry. So we can congrats again.