 Hello to everyone. Good early morning Thank you very much for coming to my presentation. My name is Yuri Kirasimov. I'm During my career. I was holding different positions, but within development. So my background is back-end development But this talk is kind of interesting and challenging to me because I don't want to talk about tools for the testing the idea of the talk is more on conceptual level and we will talk about different types of testing and The main question like how you can improve by using those and actually how you can implement them So usually when I start talking with people about testing it happens like well a lot of people heard about testing But when you ask about like okay, but do you actually implement testing on your real life projects and that's the Problem moment because usually people say like yes, we know Testing is great. It's it really helps, but not that many people use it by different reasons So one of the big points about using testing in your real life projects is of course how to sell it because this is very sensitive thing and When you start talking with clients like hey, this is like set of features We estimated it. I don't know 500 hours and we would like to spend another hundred hours on automated testing They will say well, that's very nice Very good, but what I'm going to get out of it Am I going to get any features like no? Then you start talking about quality and then you say well product is going to be very high quality and Very good argument you get is like well, but you're very good company you deliver very good quality by default, right? So why do we need to spend extra money for you to put something that we don't have much control of that? We don't have much Benefit of to get something that is called testing so I would like to to to tell a couple of stories from my practice and One story that I had that was at Drupal six time when we had like first big massive project and that was like first big project for our team and team was like six seven people and we supposed to finish the project in six months and Supposed to finish is the key word And what happened is Project was so big and that was the website for municipality. They have many many integration points They had a lot of features and it was very critical for every single feature to work Otherwise people couldn't make much things about website. They they supposed to apply different Like regulations different permits like it's all very very important. They cannot tolerate something not working So we spent half a year Developing and half a year debugging and fixing things and that was like huge disaster. I mean we were very lucky and because municipality had gone through this process one time before and That time they were delivering website in two years. So we've done it in one So that was great success. But still we went above the budget like really a lot and Well, we were thinking like how we can get better all the testing that we have done was manual and what we end up doing Was like manual testing sessions. So we finished the functionality and then every developer We had a meeting Every developer stood up explained like what functionality he has built how it's supposed to work and all other team members were Testing it and like fixing bugs on the spot and that was a very very time-consuming process But it was only the way to go unfortunately fast We didn't have like dedicated to a people and by that time there was no be had and all these new technologies so that was like big miss in our process and There was another project that happened like half a year later and The project was very interesting. It was building a Drupal 7 backhand for For like tablet application and that was for people who have problems with hearing And speaking so the idea was that they were able to communicate in restaurants in cinema and doctor by clicking some icons They were like recorded Speeches of actors so they were able to build their sentences and express themselves by just clicking them But on technical part that was just the backhand with rest API's But we did a different approach in this project. We baked in automated testing upfront So we were discussing like okay, how this API is are going to work and the system was really complex It was by default Composed with at least four or five servers. We were moving metadata around and we We build it to be scalable because we expected a lot of people using it and Our QA process Was like nothing. I mean we just showed to the customer. Okay carry your tests We click them 100% run everything is green like done That was like wow how we can apply this process to our usual development because like and and the size of the project was nearly the same so having automated testing and acceptance acceptance criteria for Project is like huge benefit, but still let's go back about selling things So when we try to sell to customer Testing we usually talk about quality and it's very hard to sell something when you cannot measure it, right? Because like you can you can understand the situation from client perspective He is choosing the vendor and like well he heard about different companies and he know and the They probably know some references, but like it it's all good on paper Here you don't have this measurements where you can say okay this company delivers better Quality or this team delivers better quality than the other So you need to think about some metrics that you can put on the table and you can describe okay This is exactly what we are talking about and it can't be understood by the customer So we can tell okay. We are going to deliver no bugs But it's default expectations of the customer to deliver software without bugs So it's not the big point to sell Then you can say okay. It will be easy to maintain, but still it's like If it won't be easy to maintain we won't be dealing with you So this is where we need some things like code quality metrics or Performance metrics or things like color like test coverage or something that you can measure in numbers So when you can say okay We are going to deliver this project and it's going to have this metrics for and I will talk about metrics a bit later in more Details, but for example you can say everything is going to be to Drupal coding standards It may sound to customer like well, what does it mean to me? And then you can say about like hey It's open source if you're going to deliver a project with this coding standards It will be very easy for other developers to be pulled in and to understand the code much better you can You can do the code audit like I did a lot of code audits for different projects And it's like the first thing and easiest thing to check like believe me it's just like one drush command and we have huge report and If you get this project from very unexperienced team It's very easy to get your customer to buy in your company because you can say hey This is the project you are dealing with like for two years and it has one thousand Like code style errors like probably you are dealing kind of with amateur team and Well, they can see all the warnings in like they cannot there is no much arguments about it It's like bad. So You really need to think about quality as some metrics that you can measure and you can sell so Some types of testing right so there are like so many things that you can think of but the Very important thing is to build something that will not take you too much efforts and that will gain you as much benefits as possible so usually when people start talking about testing it's automated testing and It is very important to have things like infrastructure for your team like development culture like all these details and It is very important to understand that it's big upfront investment It may be not upfront investment for each project because once you have set up everything It's like it doesn't cost much to build it again and again and again but if you're doing it first time you should understand that well probably I need to spend a few weeks building the system and If you do it you should really take a look at what other companies or other teams or other software industries do because it's not a new problem I mean automated testing for See languages around like many many years It's like like it has been sold like so many times You shouldn't think about like oh, I need to install or I need to write my continuous integration server No, they're like so many solutions and even if you are choosing a solution Maybe it's good idea to try something that already exists. So just as a service I mean you can spend $50 a month for something Instead of spending two weeks of your time or time of your developer to build something from scratch and Another extremely important thing about testing Automated testing is that if you don't run it. It doesn't bring you any value So it's not enough just to build a test and to report to the customer. Hey, we build those tests. Everything works like great No, you need to run tests like every day or every week or depending on what time schedule or delivery procedures you are going to have with this particular project So let's talk about Infrastructure so in like Drupal community or software development community like things like continuous integration continuous delivery They were like around for a long time I Like last few days I went through like many agencies and talk about like their processes and what I found out that These techniques are being used more on the client's basis than on like General scale of the company. So it's more like hey This customer has budget for these things. We build it for them Oh, we were able to sell it like we build continuous integration server or this is this project is too small We don't bother like it doesn't work this way. Unfortunately, you either go with it and you build the system that it's very easy to scale and Build for new projects and then you have this culture of continuous integration and testing and checking your code and code reviews Oh, you're still in this age where you like one project is the other and then like you don't have much cooperation between the projects So this is very important thing like I Will talk about tools that I'm using a bit later But there are definitely a lot of solutions that you can just pick up spend I don't know a few hours and then you can start getting benefit out of it And another big thing in quality when you start improving it in your projects is the Code reviews. It's not about like automated testing But this is something that will bring your quality and interaction between your developers to like different level Guess like usually developers. They just work on their thing and then they get another task They don't share much knowledge. Well, of course, they do Friday beer sessions, right? They need to talk about something that It's it's much better to encourage people to talk about what they're doing And that's the best way to do it Like if you see that some of your colleague do some not nice things or the things that you would do in different way You start communication and then you Gain both of you will get some knowledge out of it So it's it's very good idea when you're building infrastructure to think about this part as well Because like if you're using like github or Bitbucket or some github or some things that systems that allow this and you are making Decision which one to pick like go with the one that has this feature already Another big thing is your development culture and like deployments procedures. So in In my experience the best quality we were able to get is when Deployments are not tied to any team members. So there is no such situation like only one person can do deployment That means that Everyone on the team understands how it's going to be done and everything is automated Like we managed to do it in this way that our project manager did the releases Like and I I remember I read a couple of books about startups and one of startup They had it like crazy culture of automating things. So every time new developer gets into the team Like his first job to do is to do production development Deployment that is like that sounds very crazy like he can break and put production down But if that happens, it's better him to do it and people will recover fast Then it will happen on very important moment like press release or something That will be much better much worse disaster than Somebody will just break something in the middle of the night So it's it's very interesting but it's something to think about when you're deploying to you new processes in your company and Also when you have When you need to build your infrastructure for running all the tests you need to think about maintenance cost Like do you have enough people to actually maintain the system or do you have? Dedicated sysadmin or devops person or somebody who can take responsibility of maintaining it because you can think that well It's something that we build once and like we never come back to it until it's broken of course, but no, it's not going to happen Different projects. They have different requirements one project need these versions of software. I mean Apache solar, I don't know whatever you use PHP versions my sequel database and then the other project is going to need some something different So you will need to maintain your system. So when you pick one of them like think about these things as well So let's start talking about like real testing now so the first and from my perspective the easiest one to pick up is the static analysis of the code or Code checks first of all they run like really fast It does like one second and you will run all the tests possible in the world on your code base Well, unless you have hundreds and hundreds megabytes of the code But I hope you don't get it. Oh, at least if you have this case probably you are not on this session So what kind of code style checks? I would really encourage you to use so first of all of course is the Drupal coding standards We were talking about that part. Also, there are checks for Drupal Security-based so they can catch things like if somebody is using get like globals or like session variables or putting values into forms without sanitizing I mean something very basic, but like you can have many junior team members on the team You need to review their code and if they will see oh, I did something wrong You will save the time of senior developer doing code review and pointing them to that problem Also, very nice things are different kind of linters and this means like JavaScript linters. Yes, yes Ask linters what they do they check syntax and like this JavaScript the That's kind of my problem because I was I'm not front-end developer I usually miss like comma in the end of array and then Internet Explorer doesn't work So like this little things that can make your life much easier and you will learn much faster. They really help another thing that Can be interesting selling point when you're talking about the code quality is different complexity metrics and There are several types of complexity in the code You might heard about cyclomatic complexity and that means if you have like If you have a function and then you have like For each cycle there and then you have if and then you have another if and then you have another if and a couple of more So you get a lot of levels of nesting that is the problem So if you have this kind of thing, it's it's maybe obvious for you But it will be completely this is like horrible experience for somebody else to review that code So this complexity metrics they can point you can set the budget Okay, I don't want to have any code that has more than five levels of nesting and in this way You will need to break your things. You will need to like isolate things into separate functions So you need to exit early from your cycles or if statements I mean that will make the code much more readable and also there are some things like a copy paste detection and so if somebody is just like Don't want to take any decisions. He just copy paste from different examples or different parts of the code Without thinking of refactoring These tools can help as well. So they will identify a you have this piece of the code That is in that function in that function in that module Well, maybe I need to build just general function and just reuse it instead So it will give you more points to start thinking about the code quality and it will bring your quality higher And the nicest thing it's very very like it's so fast that you You can run it locally. You don't need to push the code to external repository to do these checks It takes just fractions of the second another big thing is the unit testing and This is very interesting topic because like a lot of people know What unit tested is unit testing is I mean it they've been taught in Universities on different classes like unit testing is great. You have different sets of arguments. You need to check what results but The ones who do Drupal development as they know that there are a lot of tests for the core for country modules But how I can implement unit testing in my real life projects I do just form alter here form alter there like what what do you need test that? It's like it doesn't make sense. You will just rewrite your function of altering the form like in different way but it it just doesn't make sense and What is the good part you don't need to unit test every single piece of the code You just need to pick the places where it makes sense from my experience, of course the best places is the functions that actually calculate something so for example, if you get set of arguments and You need to do some calculations you can write some unit tests You can feed it with some arguments to get results you check that it works What is really nice for unit testing that it's great source of documentation nothing can beat it like when I Take the external libraries and try to integrate them with my software first thing I look at is the unit tests because they show me how software is supposed to be called How functions supposed to work like what do we expect from them and this way? I don't need to read all the commutation on the website different versions like it can be hard to read But if you go through the unit test, you know that people well you say they supposed to check that they run So you will have the most current documentation you can find another thing that is great about unit testing It's good to check things like parsing of the data. So when you have regular expressions and Well, if you will have some time that you will not know where to spend and you will have Drupal I take a look at the regular expressions being used there. I mean there like this I mean some parsing like info files for Drupal 7 for example like these things are Easy to write but very hard to understand sometimes Like Drupal is great. It breaks the regular expression into lines and you have all the comments But you don't need to expect that from other developers. So When you have regular expressions parsing, I really encourage you to write unit tests So people will understand like what kind of scenarios this regular expression is parsing and how it's going to work And Another thing that Really brings a quality up is to writing the code that is able to be unit tested That is extremely hard thing But it makes you to think in a very different way about how you write the software Because like it's very easy to do like some hard coding like hey, we get this. I don't know Form API array we change it and then we pass it further And this is my sequel database. I do direct queries. I don't think about like unit testing at all but when you start thinking like hey This is direct call that I Cannot use if I want to unit test this function. So I need to Pass it some way or I need to abstract it in some way so In Drupal 8 and in modern PHP frameworks, you have things like dependence injection container That's like the best thing for unit testing ever because you can swap services out of it For example, if you work with database or external service, you can swap it with some mock and then You will just check that special methods were called in this mock object in your function and that will be a result of your unit testing so I Was exposed to unit testing a lot when I tried to refactor services module for Drupal 7 So what what was happening there? It was like classical code in Britain in Drupal and I Decided to put unit test there and the places where I've seen it's possible was about like okay we can parse different arguments that are coming make sure that they are being parsed properly or routing within the services module like how it works and it was very easy if you start thinking about that you pass All the arguments to the function that it needs so it doesn't need to call any global objects or any static variables outside and in this way you just Change the code and it makes Life much easier it gets very granular. It's very easy to test, but it takes a lot of efforts So I mean It will be great if you will get to this level when you will start unit testing a lot of things in your code and there are a lot of Articles about that and like you can see the Matrix of test coverage, but it's not easy program to solve as for sure and Drupal 8 is much better in this regard. So maybe we will get there sometime Another big thing the next like level right is functional testing and by functional testing I mean The way of testing how actually users are going to interact with your software with your website What things you can do with functional testing? I mean you can test every single thing on your website if you have Ideal scenario that never happens like you have unlimited budget unlimited time you can Test every single piece of your website, but of course it's never going to happen and because it's so expensive to do Like to write unit test probably you need maybe 10 20% of your development time I mean to start with something to write functional test if you're just starting from scratch You will need to spend at least like a couple of days to build it Well, unless you just check that your front page is working or something like that So it will take some effort and usually like in my practice We were selling functional testing and this is where all the arguments with customer Starts like hey, we would like to make sure that this piece of functionality works no matter what no matter what time of the day or night Like let's do it. So we had success selling the functional testing in commerce projects. That's for sure every Commerce project owner will really love to have his check out to work No matter what so that is very good selling point I Encourage you to try one day the tricky part of course is the credit card details, but you still have this Test environment with most of payment providers And this is something that will they will really love to have because they don't need to Wake up in the morning check that that check out works and then when they get to the office check out that it works And then when they go back home to check out that it works So they they just click one button in your continuous integration server and they see okay It works. I don't spend much time. Oh, they get report if it breaks. I mean you can do this way as well Functional testing take a lot of maintenance because It is based on the dawn structure of your website. So when you change something you need to adjust your tests So it will take you some time to get out of this problem But in the beginning you will need to rewrite your selectors few times So expect this and when you start doing functional testing, you think that it will take eight hours Don't be shy. It will take 20 So it's when you just starting doing something like Don't be shy with your STPs. That's for sure Very important thing is of course automation of this process. You shouldn't Make the system complex for your customer or for your developers or for your project managers Or whoever is responsible to work with the project to run the tests and this is like very important If tests are not being run They don't bring any benefits There are multiple tools, of course Most popular in Drupal Area is Behad. There are a lot of articles about implementing Behad. There are like Behad extensions and so forth But I'm getting into tools discussion, but Behad is not the only solution Maybe you can start with something that is SAS based So you just like Selenium IDE if you heard about this tool is just like you can record your clicks. So like very Junior person in testing can do that within maybe an hour and you already can start running those tests So like start very small don't have the ambitions Oh, I read about this cool technology Behad and I will write the tests and I will cover with tests like Every single thing and I will have no bugs like don't do that start with really small Then we will identify the things that break often and maybe Then you will implement some more advanced testing techniques to that particular problem And of course budget is the biggest consideration in this area. So User flows you like story about check out maybe in your particular Project, there will be some others that will need test coverage But it's definitely good point to talk with the customer identify what you would like to cover with tests And another big thing that is happening is the front-end testing and by that I mean You can test the performance of loading your pages You can have like JavaScript is a separate language with all its infrastructure of testing like unit testing like everything is there I'm not going to talk about that because it's definitely not the part of my expertise But I know it's safe. We will need to we will definitely look at that direction And there is a if you would like to read more I encourage you to Google for front-end ops And this is where like a lot of testing tools will pop up Another part that is getting a lot of attention nowadays is visual regression testing and the idea of this testing is to To catch bugs that are related to how website look like so if you have some CSS changes and on one page everything looks fine, but on some other page button just flew away and like something going wrong with the layout It's it's impossible to test with other tools because like it's functioning. It's It's fast JavaScript works. So like no problems But for the user especially for your customer that is a problem So the basic idea of visual regression testing is to create screenshots of the pages and to control them So there are like several approaches to this problem. One of them is to create Like to create baseline set of images so while you're doing development for example, you created one page Landing page or listing page. You say, okay, this is how it looks like. I would like to all other Deployments or other builds to compare to this Baseline image and then when you do changes you see like, okay, it's changed, but it is expected to be changed so I Replace this baseline image with a new one and in this way you actually evolve like you you get more baseline images You develop them you change them and it gets much more consistent and you catch all the visual bugs There's another approach That is comparing already existing website. For example, that is very nice for support projects So if you need to support something sometimes you don't know much about the project So it's very important for you to know exactly what pages have been changed So you can have your live environment You can have your staging environment where you deployed all your new fixes and code and then you compare them So you need to run screenshots for like thousands of pages sometimes but you need to know exactly what has changed so That is something that is also nice to have and Comparing environments is definitely easier technically problem because you don't need to it It takes less time for you to start using it because it's just like hey, this is one URL This is another URL to screenshots like done You don't need to introduce this technique to developers You don't get you don't need to fight with their resistance and things like that Some problems with visual regression testing that you need to think about is of course the performance it takes 10 20 30 seconds for each screenshot usually to take even with modern tools because like it's real website It will not give you a page immediately and screenshot means all CSS JavaScripts To be loaded and then to wait for all Ajax effects to be done like it will take some time So if you're running something like thousands and thousands pages, it's not going to be really fast So you need to think about that problem. Also, you need to store those screenshots somewhere Also, there are a lot of things like false positives because of the change of the content So you really need to think about the whole deployment infrastructure when you do visual regression testing So you need to sync the database. You need to make sure that content is going to be the same Otherwise, you will get the difference just because of some new articles appeared on listings and things like that And but definitely this is very important tool and it can be used in like multiple use cases One of the case it was really not about the quality But I would like to mention we had a project where we need to migrate the website from proprietary system to Drupal, but without changes of design So that was hard requirement customer didn't had budget for doing great design So they decided just to like move one's website from one platform to the other and The problem was that automatic migration didn't work well because of the data structure of the old website So we were using this visual testing to compare the websites I mean the old one and the new one to find the problems with migration and That was really great success like customer told us without this tool we wouldn't be able to get there because it had like few thousands pages and We were able to iterate through them Three four times during one week with one person working on the project SQA So like even to just look through two thousand pages it will take you more than that and There are a lot of use cases where visual regression testing can be very very handy Another type of testing is performance testing and this is something that You definitely need to sell because it will take some time to build it By that I mean, okay, we have this website. We need to make sure that it can handle 30,000 visitors like at the same time. It's not that easy to emulate 30,000 visitors I mean if you need to emulate 5 or 10 or 20, well, you can install jmeter and probably do that If you need more or especially if you need to emulate them to be authenticated users that is way more common thing You need to have proper investigation and proper budget for that But it's easy to sell because it's much easier to sell because it's like something that will customer understand like How I can make sure that my website will not go down after I launch like this is the way we will emulate all the traffic So that works fairly well Important thing you need to run it on production environment or environment that you know How many times it less powerful than your production? So sometimes we run it on staging environment and assume that our production will be twice more performant I mean, it may be not the best scenario, but it worked usually production is way more than two times more performant also one of technique if you have your website running for some time zone for example, it's only for United States or America in general you can run it at some time that don't have too many visitors So they will have delays in their responses, but it's not critical. So you can run it at 2 a.m Hopefully higher education website don't get much traffic during that time and There are a lot of tools for performance testing. That's for sure. So don't try to invent your own will just go through already existing tools and The last type of testing that I usually do before we launch we do some pre-flight checks and This is something slow, but it doesn't take too much time to set up if you go through this process several times So what you usually do you check all the URLs of the site map that they actually give 200 response Another thing that you can check the size of the each web page because some Editors will upload 5 or 10 megabytes image. So technically everything works, but your Visitors on your mobile phones. They will not get the page anyway. So you can check these things and Also your automated migrations may find some problems during these checks There are a lot of Actually tools for that but this is something for you to consider before you deploy and it's also a nice thing to sell Because you can sell another four or eight hours for this and in this way, I mean you do more work You show your professional level to your customer so it's definitely a nice thing to do and Now the tools so For continuous integration I would if you're just starting like try to use something that already exists try to use like Travis CI or Like some Drupal companies deliver continuous integration tools like Probo CI talk about like some other things If you have your internal knowledge, you can go with Jenkins bamboo. Maybe something else. There are like so many out of there Code checks for PHP. There are like two main products, but I'm sure they way more So PHP code sniffer it has different plugins and there are plugins for Drupal code style checks security style checks Like all of them. I'm sure there are ones for different other frameworks like symphony and others PHP MD is PHP massive detector. This is the one that have plugins for complexity checks So it will give you analytics about your complexity and you can use it For unit testing like Drupal has its own system. There's PHP unit for other frameworks So but there are a lot of things a lot of articles about it. That's probably one of the best Described part of software development functional testing Of course be had it's getting so popular. You can find a lot of sessions on Drupal cons and Drupal camps about it It runs on Selenium or other type of tools that actually emulates real-life browsers I Would encourage you to try some SAS platforms like host inspector This is the tool that you can download the plugging and actually record your actions And then you can run them from the cloud Maybe it will be easiest one to try and then you will see how hard it to be used and then You will develop your be had test based on that For the front-end testing there are a lot of them I really encourage you to take a look at front-end ops presentation of Chris Rappel He's he's really great front-end developer and he did great presentations. I learned a lot about it I cannot say I implemented most of things, but still like principles wise. It's very good For the visual regression testing I am biased because I am Developer of the tool that called backtrack This is the tool that you can easily compare to environments. So that's the main problem that it solves, but There are a lot of them out there. So you might heard about shoof. This is the project developed by Amitai There are a lot of open-source projects shoof is one of them. By the way, there is right There is web drive SCSS. I mean you can dig into a lot of articles about it and depending on how much efforts you can spend on it like Try something that will definitely help you to Stick like to keep your website stable Performance-wise the kind of standard tool is Jmeter and every all other tools are based on it So for example, flu.io or flood.io or blaze meter. They are based on Jmeter So you can you with Jmeter you can emulate the traffic already You can record you can write you can script the actions of the user, but with These services you can scale. So you don't need to think about multiple servers Maintaining them you can just buy the time and then you run those scripts from their infrastructure and for pre-flight This is the This is my hobby project. I wrote some script and go to do that But I found a lot of projects to do this I mean to do the checks of URLs and size of images and all those things I mean, it's important to remember these things when you launch the project And I'm sure you will find out way more tools than I mentioned By the way for the continuous integration. We were working on open source project. It's called CI box It's based on Jenkins So if you we did the talk at Drupal con Barcelona and Los Angeles about it Maybe you might be interested in that one as well But like it's just tools. It's way more important to understand the principles and to try to implement them and I'm very happy to take any questions. So for recording purposes We have microphone, but you can shout. I will just repeat them. Oh Then it's great. Thank you very much