 So welcome back to this last part of the testing lecture. I'll now go through a number of system testing tools. And this is just to give you examples of what you could do. It's not a representative thing necessarily. So in many cases it's also perfectly possible to use unit testing tools to do system testing. It's just that some tools might be more suitable. And I'll start with one that is fairly popular in the web industry that's Selenium. And there are two versions of it. There's Selenium IDE. There's something called Selenium WebDriver. I'll mainly focus on the IDE. I just mentioned the other one shortly afterwards. What Selenium IDE allows you to do is basically record and replay. So you can record something you're doing in a website for example. And then you can replay it later on. So you can kind of run it again and see if it still works. This has to be some kind of either regression testing that you test the bug or something that tests functionality that already exists. So it's hard to write a test that is not yet implemented like in a test-driven way. It works through the UI, so through a website for example. And it tests of course the entire thing. If you click on a button that calls something on the server and that checks something on the database and sends the response back all of that is of course checked through the UI. So you press a button and you see what comes back and you check the result. This is really good for acceptance testing because as you will see in a moment this is actually quite easy. It's also great for exploratory testing. So you can just explore, you can click around the website, you can try out things and you can record that. Later on you can just replay it. So you can say this was a good test. Maybe this is something I should keep for the future. What Selim IDE does is it records what you click on by checking the HTML code. So it basically checks are you clicking on a diff tag? Are you clicking on an element that has a certain class or ID? Or are you clicking on something that has a certain text? So it tries to find the unique identifier for whatever you're doing. It's mainly a black box thing so it's not so easy to check the internals of the code. So it's at least, I'm not an expert on Selim IDE but it's at least not straightforward to check whether something has been sent to the server in the right way. But you can of course check what comes back. If you're interested you can look up the tool on this website. I'll just do a basic scenario by recording something on Piazza and I don't yet know what to do but let's see. And I have installed the Selim IDE plug-in for Firefox so I'll just click on that. And I can say let's open an existing project that has no idea where I saved it so let's do a new one. Project name with sys-testing. So this is just my project. Okay. And I can do new tests here so this is one test. I can do plus so then I just add a new test. For example canvas test. I want to test out the canvas tool. And now once I'm ready I can just do, I can press the record button. So I say please start recording and I think it will ask me what is actually the base URL. So what is it you want to test? Well I want to test this website. I'm not sure this will work now because it probably requires me to log in but we can try. Now it works in this case okay. And you see there was the message that IDE is recording so now everything I'm doing here is being recorded and I can for example go to the assignments. Now I click twice. I can go to assignment one and I can have a look at the rubric. So let's say that's good for now then I just go back to my tool and say stop recording and you'll see that it has recorded a number of actions. It has first recorded that we open a certain URL so raykjavik.instructure.com slash courses slash 3398 that's just the URL of the course in canvas. Then it has set a certain window size maybe that's not important but we can say okay the window should be whatever thousand times 800 pixels so we have controlled conditions and then it says click on the target click on the element that has linked text assignments so that has the text of the link the text between the start and end tag is assignments. And then after that click on assignment one the text assignment one which is here. So that's a scenario of recorded and I can just replay this so if I do run current test you'll see that it just does the different things it goes very quick so you don't really see it but that's the intention of this testing that it goes as quick as possible because you want to run a lot of tests in a short time. Now we can do something here that we can also do in unit testing we can do assertions so for example we can say assert text well let's first do assert title we want to know that the title of the website I'm honestly not sure what I'm doing is correct but the title should be assignments colon F42 assignments colon and now I actually have an assertion in here let's see again if I run this test where it actually works and it says no this is not true and we can look at the log and it says well the actual value is assignment one because I'll actually check the wrong title I'm on this page and the title of here should be assignment one so actually I have written the wrong test here but you can already see you can check certain things like what is the title now if I rerun hopefully everything is fine again now still not probably I just have a typo actual value assignment one did not match something is strange probably it's just what I'm typing in here this is just me not being familiar with the testing tool enough but you can do these assertions like everywhere else so now it worked we can add another one just for completeness for example you might want to say assert text so the text of a certain element and we can now select it in the page I can say for example this one here should be assignment one rubric and now it automatically has found me a way to identify this element in HTML this is quite a complicated selector in CSS actually so it says find the element with CSS ID rubric 5676 and then it uses the child selector say select the child with class rubric title and class title and then it finds this one now we're checking that that one should be assignment one rubric and let's run this one last time to see what it works so it worked and now I've essentially written a test that says from the starting page of this course try to navigate to assignment one and check that the title is assignment one and that the rubric has title assignment one rubric that I've selected the right rubric maybe so this shows you and this was not too difficult it shows you that it's quite easy to record these scenarios and it's also not too difficult to add these assertions here so it's very effective to write tests the negative thing about it is of course as I said it's limited what you can look at at certain internal things is not too easy the other problem is it depends for example on your HTML code how the elements are named how they are structured if you change that it will mess up your entire test suite so if I somehow play around with the HTML design here then everything breaks so that was a short excursion to Selenium IDE we have as I discussed also Selenium WebDriver which is a similar thing it allows you to write tests similar to the recordings so they work in a similar way they just go through things but it's more programming if you don't record you just write a program and you can do the same things we just did in Selenium IDE you can find elements you can say give me the element with IDEX you can do things you can enter a text, you can click, you can wait but you have more powerful options to do checks compared to what I just did in the IDE so this is quite cumbersome it takes a lot of effort and it's more powerful than the record and replay so it's sort of a trade-off what you can do with these things now the last option I'll show you is a very different approach it looks almost too simple but actually it has been used very successfully in the industry by some of my colleagues so this is definitely something that can be done if you're in the right situation now the tool that we're looking at is called SikuliX originally it was called Sikuli it's essentially an automation tool so it's a tool made for automating tasks on your computer and you could see Selenium as a similar thing it automates using websites but it can also be used for testing the difference between this and Selenium is that it uses image recognition to locate elements so we don't say find the element with IDEX we use image recognition find the element that looks like this and then click on it it has some strange syntax elements like click on something but otherwise it's essentially based on Python so you can write complete testing frameworks test suites with Python and that's something that's very popular with practitioners of course they don't have to learn a new language and for example the web driver is a very different kind of programming this is essentially just Python with some extra stuff and similar to Selenium we can do stuff like enter text, click, wait and so on there is no whatsoever access to the system so the system is just running it's a black box you don't look at any variables basically all the checks you do you do based on how it looks like so instead of saying check that the text equals X you say check that there is an image that looks like this and this is also something that you could easily go through with a customer it's quite easy to understand what is going on and again I'll just do a demo of that I do a very basic one what you can do is much more complicated let me close all the other stuff let's go back to where we were before and I'll just write a SQL script so SQL comes in its own weirdly looking tool but I've seen sort of power users of this typically don't use the tool but they actually use just a Python regular editor like VS Code now what you can do here and you can look that up in documentation but there are special things like click so click on something and as I said we want to say click on a certain thing that looks like something else and for that we can do screenshoting so just say take a screenshot and say for example click here and now what the tool will do is it will try to find this on the screen and then just click on it and then we can click somewhere else for example we can again click on assignments so we want to go here what happens then is that we load so you actually have to wait for example 2 seconds or 3 seconds I don't know what's going on here I don't think there is a problem but apparently the tool thinks so and then afterwards we are on another page so we can click somewhere new for example here wait and then we're here and we want to check something for example we want to check that this assignment is published so we could say find find the image that looks like this and that should be it I don't know what's the problem here there's probably none we can just try to run this now and the cool thing here is it's actually quite by just looking at this is quite obvious what will happen you don't have to be very good at programming so let's try this run clicks here it clicks on assignments it waits it clicks on assignment 1 see this did not work it should have clicked there it didn't and because it did not it didn't find this image so our test failed here and the problem is simply that it clicked sort of under the actual link and there are tools to change that to say well within the image you find click on a specific spot so now it should click on the text instead of the line in between let's try this again this looks better and now the script completed and everything is green so the test has passed but you already saw before that given that it hasn't opened the window this statement down here failed so it basically said no I cannot find this so the test case fails and since this is python you can just use regular regular code as far as I remember if you do python and write all of that in here it works should work reasonably well so that's another example of what you can do and there are other things you can do here this is really just super quick run through Sikuli so this is cool it has of course disadvantages one of them which is because it uses the library OpenCV is it's really bad for color differences so if you want to distinguish colors it's not a good choice the other thing is of course if you have a website where you have a lot of identical things so identical buttons with the same text it's really hard because you use image recognition so if you suddenly find four times the same button then you have to somehow do something most of these things can be solved and this tool has been used on productive systems for example system testing for air traffic management solutions so it's perfectly viable to do that it has shortcomings of course to other things the great thing is really that it does not rely on any language or any it doesn't matter whether your system is written in C++, in javascript in assembly in anything else as long as it has a graphical user interface so that's good so overall now we discussed a bit of testing tests are really needed and nice to have debugger is good to identify error sources and then we have a whole bunch of stuff that we want to use to avoid mistakes so before we even need the debugger or tests we already use things and one of the elements is for example a strict mode in javascript that simply flags certain things that are known to cause problems and then there are other things for instance a very common thing is to have style guidelines that tell you how to format your code and it's not just to make it pretty but it's simply because it's known it has been shown often enough that readable code pretty code basically is less prone to errors so if the code is readable you will not make as many mistakes then there's another thing and we'll look at that in the other testing lectures and that's linters so there are tools that similar to compiler errors that show you what is wrong in your programming they just underline things that are bad practice or that are not good so they can help you already while writing the code to avoid certain things but we will not go into that now so to conclude this we have seen that there is a large variety of testing techniques in the first part you can use that at different levels different contexts and there is really no best fit so the best thing is to do as much testing as you can and somehow try to do it in a smart way that you cover different parts for example I've only covered a couple of examples mainly unit testing mocha and chai and we'll probably use that more in this course and we'll also reuse that later when we get to the back end to test our server code and I've just done a quick run through system acceptance testing through the user interface just so that you have seen that that whatever this becomes relevant to you you can go back and maybe have a look at these tools there were two differences that I've shown you in the system testing part in selenium we interacted with elements by identifying them through their html or css secoli which was in contrast to that just uses pure image recognition so it does not rely on the software being html or any specific programming language then we had mocha as a typical unit testing framework in this case it supports several styles but it's just an example of a very common unit testing library that you will encounter probably many times in your career and unit testing comes with assertions and in this case we use the specific assertion library which is called chai so that's it for this part in the next lecture we'll do a summary of everything we have covered and then probably do repetitions or clarifications of any of that material so thank you for listening