 The second part of this lecture 20 is going now into unit testing. Just as a short reminder there are tons of different things you can do, different ways of testing, different grades of optimization, different focus, different abstraction level. We focus mainly on unit testing here and now if we go into server-side JavaScript there are a whole bunch of things you can do. So you can do unit testing for single functions. You can do something that is called endpoint testing. So we test our HTTP server endpoints, the different methods we have been programming. Of course you can do integration testing. So once you integrate multiple functions, files, libraries and as a matter of fact endpoint tests are quite often integration or system tests. So they usually test more than one thing and you can do system acceptance testing. You can of course not do any UI testing since server-side JavaScript does not have any UI. And all of this is luckily highly automatable. To recap, when we did not use Node, we did not use NPM, we used Mocha through the browser. And as you of course remember we included our JavaScript files with a script tag and then to run Mocha we also had to include Mocha itself. So the Chai and Mocha libraries. We had to set up Mocha and we had to include the test file which was then run automatically. Luckily with NPM this is getting a whole lot easier and it's much more like traditional unit testing. So we simply use NPM to install Mocha and Chai, the two tools and then we use the executable. We use Mocha to specifically say please test the following file. The test file then is responsible for including all the JavaScript files that are relevant. And this is now a much neater separation of tests and application codes. So for instance you see that there is a .test subfolders. So all the tests are in one folder, all the production code is in another folder. So it's nicely separated. There is also no test setup code in our application code. Which is exactly as it should be. So that's much much neater. That being said you can run Mocha like this also on JavaScript files that run in the browser. It's just that back then we did not introduce NPM or note but you can also do that. So that's important. There's one last thing here that we haven't maybe mentioned that much is the minus minus safe def. So you have seen minus minus safe. That simply means that the dependency is saved in the package.json file. There is a difference between regular dependencies and def dependencies. So these kind of development dependencies are only installed for development purposes. If you are on the productive server they are not installed. So it's just less. It's less dependencies that could cause problems, security vulnerabilities and so on. It's less space that is being consumed and so on. So quite often in real cases you see this distinction between regular and development dependencies. Okay let's just jump into an example. I have shown you this module at some point when we did lecture 14 on some advanced JavaScript. I showed you that in note you can simply write a module and export individual functions. So we had for example this get SSID function. We had another function and we had a variable and because I used this export these two things were not accessible. Only the get SSID function was accessible from outside. And now we can write a test for that. So we simply write another file for example called test slash my module dot test dot js. So often you use exactly the same name as the file you are testing. You just add dot test. So that way it's clear. This is what you're testing. And now you start your test file by actually including the module. So you go one directory up. You include the JavaScript file. You assign it to a variable. And then exactly as you did before you write the mocha test. So you just do describe, give it a string, give it a callback function and then you have your individual tests. So for instance a test that checks that get SSID is returning a number would look like this. It would just say chai expect then there is the function call my module dot get SSID. So we call this function and we check that whatever is returned is a number. So that's just a type of check in JavaScript. So essentially checks that this variable here is a number. So let's look at that in VS code. I just go into the appropriate code here. I open it. Here is my module exactly the same that I just showed on the slides. So I have my get SSID that's exported. And here's my test. So it's down here in here. I called it tests. Let's rename that test like in the slides. I import my module. I also have to import chai. That's nothing that I have in the slides. And then I check whether the return is a number. Let's try to run this. Here we are. I still have to install the dependencies. So we could try npm install save death mocha chai. That's exactly what's on the slides. And now you'll see that there is a package.json file created in the moment. No, it's not. They directly install it because I don't have a package.json also fine. But if you look into the details here, you see that there is chai and there is mocha. So those things have been installed. And now if I go in here, I can simply run it. So this is just the executable. It's in my node modules in the mocha folder in bin mocha. And then I pointed to the test file. So in my current directory, it's in the test subdirectory. And it's called my module test.js. And if I run this now, this looks exactly like it did in the browser just in my console. And you see that my test fails because one, two, three, four, five is apparently not a number. If I look into my code, I'm actually returning a string. So if I change this, now it's a number. And now I rerun the test and the test is passing. So this is pretty much the same as before. It's just that it's much, much easier to execute these tests. You don't have to write any HTML code. You don't need any HTML code. And that's making the whole story much easier. Also here, this is our module. This is our application code. There is no reference to the testing stuff here. So it's really clean application code. The testing them is responsible for importing whatever is needed. So that's how we do it, basically. But now there's an interesting concept here. One thing I have mentioned a bit is that we are actually doing unit testing here. So we should only test a single unit, like a function. And the other thing is when you do that, what are you actually testing? And I've mentioned that a bit. One thing of classifying this is to say you have the success case. This is how the function should work. And then you have like edge and corner cases or something that is at the edge. If you have any input at the edge of an interval, like in assignment three, for example, the date has to be in the future. So one edge case could be to take the exact date. What happens then? Or the end date of an event had to be larger than the start date. So what happens if you make them equals, for example? So these kind of things are typically edge or corner cases that you should be very careful with. Those things you can check. And then there's finally failure cases. So that has to do with input checks, input validations. What happens if in the date field instead of a number I put in a string or I put in undefined? What happens then? Those kind of things you should test. There are a number of other things we should be looking at. One is don't do random values because then you get something that is called a flaky test, a test that fails sometimes, a test that doesn't always fail. This is in practice one of the biggest problems companies have when they have large test suites. So that's what takes them a lot of time that there are so called flaky tests. Instead of random values, you should really think about the cases, like as I said before, you have edge cases, failure cases, success cases, and then make the test deterministic. One test only tests one thing and it does it in the same way every time you execute it. If you want any kind of random behavior, there's other stuff to do that. For example, you have things like monkey testing, which is just doing random things, pushing random keys on your keyboard if you have UI testing or so. So the aim of unit testing is always that a single test should as precisely as possible test something that has to do with the cases, but and that's now where we get into the next step. It also has to do with other functions. And if we look into the example here, if we look into our module, the get SSID function is calling another function. So it actually depends on what this function is doing. And that means our unit test is not a real unit test, but it tests two functions together. And this is not good because imagine this other function is developed by someone else and it changes behavior, suddenly your function fails or your function changes behavior, but it's not because there is a problem within your function that it's because someone else has done something wrong. And that's something you do not want to have in unit testing. In unit testing, you really want to assure that your function works assuming that all the other things work. So you do not want to care about what happens here. And that's that's an important thing. And this is why we have something that is called mocks or stubs. So we want to replace other functionality by some kind of fake behavior that does exactly what we want. So we want to limit external resources, not only other functions, but also database access. So you don't want them in your unit test, you don't want to start querying the database, you don't want to start doing HTTP requests, because those things can be slow, they can behave in a strange way. So it could be that sometimes your test passes because the database is quick, sometimes your test fails because the database currently has a lot of load and that's why it doesn't reply on time. There might be bugs in those things. So again, this is nothing that has to do with your functions. It's something else. So that you want to avoid. And the main reason why you want to avoid that is if your test fails, it's very hard to tell what exactly was the reason. If you actually only test your function, then you can tell exactly if the unit test fails, it's because of your function or because of the test, one of the two things. But if you have a lot of dependencies, you have to start looking at all the details. Remember also in lecture nine, we discussed system testing, UI testing, you saw that there were a lot of different error sources. For example, we don't wait long enough after clicking and so on. So everything that is extra that our function depends on, we would like to reduce. And one of the main ways to do that instead of programming in a certain way is to use certain things for testing specific facilities. And there are essentially two things we use. One is called stubs, which means we are replacing a function with some kind of fake thing. And mocks, those are essentially things that replace external resources. So instead of doing an actual HTTP request, you pretend to do an HTTP request. And we'll just look at two libraries here. There are lots of tools for this. But one is called sign mjs. That's what we use for stubbing. And then there's something called knock, which we'll use to replace external libraries. So stubs are there to replace other functions so that we can really make sure that we only test our function. This is something I adapted from one of the labs, I think. So that's a piece of code where we first want to check a URL. If the URL has somehow the right format, then we want to load a file from there. And then if the file is being loaded, we say, okay, if the file is not being loaded, we say file could not be found. And as you see in that code, I'll quickly open it up here as well. Yeah, it's from lab four. As you see in this code, the function we want to test depends on two other functions. And it could be that we are in here, we're doing everything correct. But someone is messing up check URL. So for example, the check URL function assumes someone does a bug and it always returns true. So it does not have the right behavior. So suddenly if we would test this, our test might fail. But it's not because we have done something wrong, it's because someone else has done something wrong. And then we have to check where exactly is the problem. So what we would like to do is we would like to exchange these actual function calls for something fake, we just assume that this thing works in the way it does. And one way of doing that is to use sign them. What we could do is we create a so called fake function, that's these two lines here. So we say create a function that always returns true. So this thing just returns true, very basic. And then we replace a call to mod dot check URL. That's just our function call. Down here, you see mod dot. So mod dot check URL is replaced with this fake thing. So now every call to check URL is simply returning true. The function is not any more called. And that makes sure that no matter how messed up this function really is, our behavior is really just testing the check and get URL method. And we do the same with the other function. Here it's slightly more complicated because load file async is an asynchronous function. So here sign and offers the yield option. So instead of returning, it means the callback function, the input parameter is called with null. But otherwise it's exactly the same. So we basically replace two functions with a specific behavior, replace check URL always returning true, replace load file async with always calling the callback with null. So if we go back into our code now, what will happen is this thing will return true. And this thing will take the callback function and put null in there. And then we can actually check, is this behavior here correct? There is not a lot of behavior here, but at least we have gotten rid of our dependencies. So then the test looks like this. Before each test, we create these fakes. And then we simply do the call to our function, we put something in as not valid. And we assume that the return value is null. Let's run this. I again have to install mocker and shy and also see not knock. Those sort of things I'll be using. Take some moment. And now I can run mocker on my test app stopped. And there's lots of stuff happening. I don't know what's going on. Cannot find module. Okay, there's a lot of other stuff I haven't installed. Let's hope it's not too much. Okay. Now this looks better. Now this is passing. So you see that the behavior is exactly as it is. If this thing returns true, so it goes into the next call, this statement returns null. So it goes in here. And then the callback is actually null. And that's exactly what we test for here. Now assume that I do a mistake here, I accidentally do not equals, for example, then our test would fail. So that's what we want to test. Instead, if I now do a mistake in my check URL. So for example, it returns false. Now you see that this would be not true. So it would go in here and will actually give me a different callback. So our test should fail. But you see that the test still passes. And that's because I have replaced this function call with something else. So even if someone messes up this function, our unit test for the check and get URL still runs because we have not made any mistake in here. So that's an important concept. Stopping. This is as I said, in in sign in, it's called a fake. But you'll see similar concepts in other programming languages. Sometimes it's also called mocks. So there is no clear distinction there, I would say. But in this world where we're using here, stubs of fakes means replacing a function. And then the next thing is mocking, you kind of replace some kind of external service. For instance, if we look at the load file as a function in more detail, let's go into our code. What happens is, I'm using a different library here, but we basically do an Ajax request. So someone puts a URL in, and I do get request to that URL. It's asynchronous. And then, if we get a reply, we just send the text back, whatever the URL replies, if the status code is 200, we send the response back, otherwise we send null back. So that's just some kind of asynchronous call. But this is an HTTP request. So it's again, something external. It's something that has unpredictable response time. So this is really tricky to test. So if we want to just unit test this function, then we would like to replace the actual get call with some kind of fake thing. We want to check that this works, but we don't want to check that our network is quick enough. So we replace, we don't send an actual HTTP request. We mock the HTTP request. And this is the test. There's nothing special here. We call the function and we use an asynchronous check. We check that the loading this resource sends back, backend is online. That's what we check. And we're done. But now we use knock, we use this other library to say replace any call that goes to this address that uses a get function to test. So if anyone sends a request to a URL slash test, then you should always reply status 200 back end is online. So we don't care what this thing actually does. It always replies this is just a fake in our code. Each get request to this URL will return 200 back end is online. And they will do so immediately. So we don't have to wait any certain time. So let's look at that. That's the other test here app mocked test. So I am saying any kind of request that goes here, flash test should reply this. And then I try to trigger this and I get exactly that result. Let's try it. It would be surprising if this now behaves strangely. And it passes. And this is interesting because if you go to the real URL. So this is what I used last year in the course, this URL does not anymore exist. So if you do an actual call to that URL, it would not work. No such app. So you would get something else left. So if I actually remove this, if I let the test run the actual HTTP request, then this would fail. Because I get null back. And I expect that back end is online. Again, this would be more of a system test because I'm testing my functions in addition to external things. In this case, some kind of back end that runs somewhere. So if I really just want to test the load file as a function, I should replace these calls. Okay, that's roughly what I wanted to cover in this lecture. So we have gone through linting, static analysis of our JavaScript code, what are common errors, what are bad smells of code that might lead to problems. It's not a problem in itself. We looked at debugging. Do try it out. You will have to try it out in the assignment for the issue here is our note file, our note application runs somewhere it doesn't run in the browser. So we somehow have to connect to it. We can use Chrome Inspector VS code for that. But we have to run it in inspect mode. We did unit testing much much smoother than we did it in mocha. Because we don't any more mix application and test code, we not any more mix HTML with JavaScript and whatever. So it is nicer. But then we looked into the details. If we want to do proper unit testing, if we really don't want to have any dependencies on functions, databases, HTTP requests, whatever, you need to use techniques such as stubs and mocks. So you need to replace external resources. And I've just shown you two very basic examples. I've replaced two functions. I have replaced an HTTP request. But you can probably imagine based on those five minutes, that this can get quite complicated in an actual application. If you have a lot of calls, if you have a lot of dependencies, this is not easy at all. So that's another skill that you will have to develop over time. And this is not just in web development, this is everywhere the case that you need these kind of techniques to write good unit tests. Okay, this concludes lecture 20. We'll continue with with testing a lecture 21. And there we look specifically at endpoint testing. So we look at how to test our restful APIs, what should we be checking for? Overall endpoint testing can be seen as a form of integration system testing. So we test the entire backend through the HTTP request. But more on that in the next lecture.